Corrupt eflibrary - Network Backup Approach

I corrupted my EagleFiler Library (user error) and have some thoughts about how to lessen the impact if it ever happens in the future.

I’ve been casually using EagleFiler for some time, using test data to become more acquainted with the application. Recently, I must have done something wrong without knowing it - like maybe moving a file in the finder to a different folder while EagleFiler was open. Anyway, I got some errors off and on and did a Verify on all records and Update Checksum on all records. Next time I opened the library, EagleFiler tried to open it, but said it was unable to open it. No errors were reported in the Errors window. Now, when I try to open it, EagleFiler launches, but no library is opened and no errors are reported. So, I’ve lost my library (the file is still there) - but It’s not a big deal because I was just testing. By the way, I didn’t have a backup, because it was just test data.

Any way, I then created a new empty library and re-imported all the files that were still intact from the old library. The fact that the individual files are still valid and also organized in the proper folder is a HUGE advantage of EagleFiler over other applications that store the data in a proprietary database. I’ve lost my tags, but that’s not the end of the world.

My library resides on an external disk drive connected to an AirPort Extreme. That way, I can use the same library on either my desktop or my laptop (not at the same time).

Backing up the library and all the files is the best solution, but I just don’t do backups often enough. So here’s what I’m planning to do. I use FoldersSynchronizer to backup my user folder on each Mac to the network drive on the AirPort Extreme. I plan to use FoldersSynchronizer to backup the whole EagleFiler library (xxx.eflibrary and all files) from the AirPort Extreme to each of my Mac’s, and never open those copies. After I’m done using EagleFiler on either Mac I’ll run FoldersSynchronizer to backup all the data - it only backs up files that have been added or changed. That way, I’ll always have two very recent backups of my EF library.

I’d welcome any other approaches to backing up a network drive (short of RAID). Thanks.

I think these are actually two separate issues. Moving files behind EagleFiler’s back means that it won’t know where they are, but if you move them back or delete them everything will be fine. The library being corrupt and not opening would have a different cause, e.g. a disk error, or a crash or error in the middle of EagleFiler saving the .eflibrary file. This is much more rare and probably not due to anything that you did.

Are there errors reported in EagleFiler’s log or in the All Messages section of the Console application?

Because of EagleFiler’s metadata backups, you shouldn’t lose the tags or notes, either. I can only guess that since this was a short-lived test library EagleFiler never had a chance to back up the metadata.

I’m not familiar with FoldersSynchronizer, but it sounds like a reasonable plan.

Another option would be to put the library on an iDisk and back it up to the Macs using iDisk syncing.

Errors and Recovery
There were hundreds of lines in the Error Log - I’ve put the entire log out at www.carlit.com/eferrors.txt

Here are a few summarized:

2008-10-02 00:33:54,578 - appcontroller.pyc:125 FSPathMakeRef error -43 getting creation date for: /Volumes/FreeAgent/Eagle/EagleTest1/Files/fw4.pdf

Lots of the above type of error.

2008-10-02 00:43:34,960 - library.pyc:515 addFileWithImporter
Traceback (most recent call last):
File “wash/library.pyc”, line 507, in addFileWithImporter
File “wash/records.pyc”, line 297, in copyIntoLibrary
File “path.pyc”, line 922, in copyTo
error: TaskFailedException - 1 - cp: chown: /Volumes/FreeAgent/Eagle/EagleTest1/Files/Web/NoteBook - Organization for a creative mind-1.webarchive: Operation not supported

2008-10-02 00:46:47,701 - appkit.pyc:66 validating clear:
Traceback (most recent call last):
File “mjt/cocoa/appkit.pyc”, line 64, in validateItemUsingCanMethod
File “mjt/cocoa/tablecontroller.pyc”, line 53, in canClear_
File “mjt/cocoa/tablecontroller.pyc”, line 69, in canDelete_
File “wash/itemscontroller.pyc”, line 321, in objectsForDeletion
File “wash/sources.pyc”, line 512, in objectsForDeletion
File “wash/records.pyc”, line 548, in canDelete
File “wash/records.pyc”, line 534, in isInTrash
File “mjt/basics.pyc”, line 241, in wrapper
File “wash/records.pyc”, line 372, in parent
AttributeError: ‘NoneType’ object has no attribute ‘recordForID’

Lots of the above errors - with variations.

Again, this was a test library, so I don’t need to recover any info.

Recovery

I wasn’t aware that the metadata was backed up. This is HUGE! So, based on 5.2.10 Backup Metadata in the EagleFiler 1.3.8 Manual, would the following be the process to recover the metadata?

  1. Have a backup of the metadata - This is done automatically behind the scenes by EagleFiler, or if in doubt, manually do Option, File, Backup Metadata.
  2. Create a new EF Library.
  3. Close the EF Application
  4. Copy all the files (and folders) that were in the old Files folder to the new Files folder (probably don’t copy the Trash folder)
  5. Copy all the files that were in the old Notes folder to the new Notes folder.
  6. Launch EF with the new library
  7. EF will associate the backed-up metadata with the files in the new EF library (this is kinda like magic…)

By metatdata for a library record, do you mean: Tags, Title, From (maybe), To (maybe), Dates (all 3?), Label, Tag Names?

Thanks for your response.

This is harmless.

This means that there was an error copying the file into the library, specifically when setting the file’s owner. I don’t know whether this has to do with strange ownership information on the original file, or perhaps with the destination AFP volume.

The trash-related errors are actually a good sign. The only known bug that causes corruption of the .eflibrary happens when there’s an error emptying the trash. The next version of EagleFiler fixes this bug and can recover library files that were damaged by it.

It’s never correct manually add files to the Files or Notes folders. That’s making changes behind EagleFiler’s back. Instead, you need to import them. The Backup Metadata section says:

If you import a file that has backed-up metadata, EagleFiler will restore all of the metadata from the backup. For example, if a .eflibrary file were damaged, you could create a new library, import the Files folder from the old library, and all the metadata would be preserved.

So, in other words, the steps would be:

  1. Have a backup of the metadata.
  2. Create a new library.
  3. Drag the contents of old library’s Files folder (in the Finder) into the new library (in EagleFiler). (It’s up to you whether to import all or some of the files, skip the Trash folder, etc.)

The backup preserves: tags (names, colors, abbreviations, and hierarchy), notes, title, from, to, dates (added, created, modified), count, label, source URL, and checksum (so EagleFiler can tell if a file is damaged).

Not preserved by metadata backup: indexes (will be automatically rebuilt), indexing settings for phrases/words, viewing state (window and split bar positions, which columns are hidden, view settings, which sources were expanded and selected, etc.).

Recovery
Thanks so much for your insight. In the unlikely event that the eflibrary becomes corrupt, I feel very confident that not only the files I put into EagleFiler can be recovered, but also the metadata.

Your support of this product is amazing.

This is fixed in EagleFiler 1.4, and it will automatically recover library files that were damaged in this way.