HomeGeneral

General StudioLine topics

MANUAL CLEANUP OF DATABASE 

Wayne Hewitt
Wayne Hewitt
Posts: 34


11/19/2015
Wayne Hewitt
Wayne Hewitt
Posts: 34
I have deleted all photos, webs, and collages from what was a db of several hundred gigs.
I then compressed the db but was surprised at how large it still was so I looked at the folder structure (.png attached) and noted that the folders Temp, Images, and Cache are contributing the size.

Is it safe to manually delete their contents?

What about going forward? Is the large Temp folder most likely from my deletion actions?

I could just delete the whole empty db and start over but I want to keep the tag structure.




Attachments:
SL_Database.jpg
0 link
Andy Schmidt
Andy Schmidt
Administrator
Posts: 85


11/19/2015
Andy Schmidt
Andy Schmidt
Administrator
Posts: 85
Hi,

When you refer "db of several hundred gigs", what specifically are you referring to? An actual database file is restricted by Microsoft to be 4 GB - so it could not possibly be "several hundreds" (unless you use a SQL server version of StudioLine for our professional users).

The entire "Database/" subfolder can be larger, because it contains backup copies, transaction logs, journals, and various other transient files that allows StudioLine to recover/rebuild a damaged database after system crashes, program faults, hardware problems, and the like.

Then there is the "/StudioLine Data" main folder that holds your entire Image Archive (including the various subfolders). THAT of course will continue to grow to whatever size needed to securely archive all your original images at the maximum resolution your camera originally produced - but also holds smaller resolution screen-sized and thumbnail-sized versions that are dynamically created and crucial for snappy performance. Some of our pro users and very active semi-professionals have image archives that are many Terabytes (thousands of GB) in size.

In that context it would be crucial to know the "Settings" for your StudioLine Image Archive. Are you storing your originals "internally" by moving them into the image archive, and StudioLine has the only originals? Or are you keeping your originals "externally" where they are NOT copied into the image archive and StudioLine just remembers their location elsewhere on your Windows system.

Now, you mentioned you had "compressed the db" - how exactly did you go about doing THAT? Using some Windows utility, a function within StudioLine, MS Access? Once I understand what you compressed, and how, I have a better idea of what that would accomplish - or not.

If you deleted any pictures within StudioLine, they end up in the "recycling bin", from where they can be recovered. If you want to free disk space, you can EMTPY the recycling bin inside of StudioLine.

What do you consider the "tag structure"? If you are referring to the "image tags" (descriptors) for each image, these are specific to individual images. When you delete these images, that kind of metadata is lost as well. If you want to preserve/backup images and their "image tags" (descriptors), you can export them in StudioLine and restore/import them later or on a different computer. The export will create two files - and image file holding your original image and a "sidecar" (XMP) file holding all the descriptors (whether they originated from the camera or were added by you later).

If with "tag structure" you are referring to your list of "keywords" and "categories", you are able to export those to a text file.

Finally - can you catch us up on what your problem was on the outset that you are trying to solve, and what you are hoping to have accomplished at the end. Maybe you can also describe your you have integrated StudioLine into your image editing/archival/sharing work flow. This way we can figure the best ways to go about it all.

(If your image archive settings keep all your originals "internal" to StudioLine, deleting them would mean a permanent loss - so I imagine that would not be your intention. Instead, if the totality of all your images is reaching the limit of your primary hard disk, it may be time to consider a much larger, possibly external, disk drive accommodate your future growth and the fact that every new camera will likely be capable to producing even larger picture file sizes than the one you had before).

Best Regards,
Andy
edited by Andy Schmidt on 11/19/2015
0 link
Wayne Hewitt
Wayne Hewitt
Posts: 34


11/19/2015
Wayne Hewitt
Wayne Hewitt
Posts: 34
Andy,
I see I did not really give enough information.

My original issues were
1. The 600G drive I use for SL (one of two in a laptop) was very nearly full.

2. I had multiple(s) of many pics and had multiple sources arranged in a structure which had grown unworkable due to a few abandoned starts at reorganizing it over the years.

3. Dragging and dropping to a new structure would take much more work than an export with import to a newer structure.


All originals were stored by SL.

I exported the entire set in what I considered to be logical pieces; Once with the original structure intact, once as individual pics - both sets with xmp files.
I tested export/import before starting to ensure that it did in fact restore properly. I did not see how to export the tag/keyword tree but I did know that the ones I had used were in the xmp files.


The internal recycle bin was cleared after each export. I noted that it would only delete a percentage of its contents each time even though I had selected the entire contents.

I 'compressed' using the SL menu item 'Compress Database'. I could not see that it did much other than perhaps re-indexing.

I deleted all web and collage activity outright as I had no ongoing need for those which had been created.

The image I sent with my original question shows the current size and file count of the folders within the dn name within the \Studioline3 Data folder.
I am running StudioLine Photo Classic 4 v4.2.14

So, in short, I have no Image Archive pics in the database but the folder sizes still total some 250G. I could export the tag/keyword structure and start over but I still would like to understand why the larger part of the 250G is still there, and assuming it would reindex itself, whether or not it can be manually deleted.
0 link
Andy Schmidt
Andy Schmidt
Administrator
Posts: 85


11/19/2015
Andy Schmidt
Andy Schmidt
Administrator
Posts: 85
Wayne Hewitt wrote:
I had multiple(s) of many pics
Not your issue right now - but let me just point out that the image archive has TWO ways to create "copies" of images, that will NOT cause duplication of space. This is helpful whenever you want the same photo to be found in more than one location. In both cases, only a second REFERENCE to the original picture is stored - in one case as a link to whatever current edited state, in the other case before edits.
Wayne Hewitt wrote:
I did not see how to export the tag/keyword tree
A screenshot is attached.
Wayne Hewitt wrote:
I 'compressed' using the SL menu item 'Compress Database'. I could not see that it did much other than perhaps re-indexing.
Yes, the actual JET (or MS SQL Server) database contains mostly metadata and typically is only a few GB large. Compressing it will not result in major disk space savings - it just optimizes the database engine by creating continuous blocks of free "pages".
Wayne Hewitt wrote:
I still would like to understand why the larger part of the 250G is still there
Those are likely preview and thumbnail images that were created dynamically and have not yet "expired".
Wayne Hewitt wrote:
assuming it would reindex itself, whether or not it can be manually deleted.
I admit that I have not investigated this in the program code - partially, because I couldn't guarantee that I might not overlook some logic branch that might be crucial.
Basically, I definitely advise against making ANY changes to ANY files/folder/subfolders inside the "Studioline3 Data" path. While I know for a fact that certain files and/or subfolder contents are indeed dynamically created I cannot rule out that some of those files are cross-referenced in the database. Thus deleting the file from Windows Explorer would result in "orphan" entries in the database and I don't know if and how the program code would account in every possible combination of scenarios - either now or after future program changes.

Therefore, if you are certain to have full backups / exports of all the data, images, keywords, categories, etc. that want, I would rather you rerun setup and choose for StudioLine to create (add) a NEW (parallel) data folder, e.g. "Studioline Data". Then, with the new/empty image archive being the "current" one, import your keywords and categories in there, start importing your pictures, etc.


If anything is missing, you can toggle back to the old image archive, and take more exports.


Once you are satisfied that everything is carrying over perfectly, you can then use the data management options of StudioLine to "detach" the old "StudioLine3 Data" image archive, close StudioLine and then delete that entire folder structure.
edited by Andy Schmidt on 11/19/2015

Attachments:
ExportKW.jpg
0 link
Wayne Hewitt
Wayne Hewitt
Posts: 34


11/19/2015
Wayne Hewitt
Wayne Hewitt
Posts: 34
OK, I will make a new iteration and start reloading. I likely did have physical duplicates as most were the result of having changed image names externally and then reloading (Its a long sad story smile ).

I did compare file counts on in SL and the backups (allowing for XMPs) so I am confident that I have all the images. It a moot point as they are no longer in SL other than in the Temp folder.

Thanks for the quick help.
0 link






Powered by Jitbit Forum 8.3.8.0 © 2006-2013 Jitbit Software