workflow problems: moving into EF, modify save dialog, metadata EF/FS consistency

Hi,

I really like EF and the approach of only implementing sensible features.
However, there are a few things that make working with EF slower than necessary. For the following list, it would be great if you could either tell me whether and WHEN you plan to implement this feature or give me some hints how to implement that in applescript (if there does not exist a script yet).

Here are the issues (only the most important ones, I will probably write more requests in the future):

  1. moving files into library:
    Except by using the ‘to import’ folder there is no way to automatically delete the original file after importing (=moving instead of copying). Using this folder is no real alternative because it is not possible to import with options (i.e. option-F1) and sometimes it’s just not confortable.
    I’d suggest to a) make a checkbox in the import with options dialog to move instead of copying and b) provide a hotkey to do the same without options dialog.
    I think b) is also scriptable, so this might be possible to implement without waiting for the next version, but I don;t have much experience with applescript yet…

  2. saving directly into library (modifying save dialog)
    This would really speed up the workflow of many users and make EF much more useful because people will import many more things into EF.
    I want to have a modified save/save as dialog in all applications that directly asks for all the options you get when pressing options-F1.
    If this is too much work, it would also help greatly if one can at least specify the folder. At the moment there is no good way to save directly from an application because using the ‘to import’ folder or F1 creates a copy, so that the application does not know the new file location, so it’s not possible to modify and save the document again without reopening it beforehand.
    A quick solution (that would be sensible anyway but should not replace the modified save dialog request!) would be to allow apps to save directly into the library, so that one can at least specify the folder. This should be very simple: create a folder action (or sth similar) that notifies EF when there is a change somewhere in the library folder. ED then determines what change has happened and imports the new file or updates the library accordingly. This would already greatly simplify saving because one can continue using the application to further modify the file without the need to reopen.
    (Actually I don’t understand why EF does not use sth like folder actions to get notified of new files in the import folder. That’s much better than polling every 30 seconds or when the EF window gets focus).

  3. library / filesystem consistency
    This is more of a concept than a single change and will probably require many internal changes. Also note that the quick solution for 2. is a subset of this feature request.
    By library/filesystem consistency I mean that all information in the library is replicated to the filesystem, and all changes in either the file system or the library should trigger an update in the other place (synchronization).
    Currently only the folder structure (and filename) is replicated. There is also an options to put tags into spotlight commets (very useful!).
    But why stop here?

I suggest using HFS metadata support (this is also what spotlight uses) to append the following information to each file:

  • tags (using the appropriate field(s))
  • author, title, comments (=notes), URL and library
  • all dates (modified, creation, added to library)
  • probably also all containing folders (that’s already there implicit by location, however this may help spotlight to find the file)
  • anything else that EF saves (the idea is that all informations in EF’s database is also in the file system - using either custom fields or - better! - already existing standardized fields that other apps could also access)

Look at http://www.fluffy.co.uk/spotmeta/ for a description of how to add custom fields to file metadata (look at notes for developers) and about spotlight importers (this could help to keep metadata IN the file in sync with EF and real HFS metadata).

Synchronizing should work both ways:
a) if the EF library changes, update the filesystem immediately (easy)
b) if the user makes some changes in the filesystem, update EF accordingly, e.g.:
- if the user moves a file from one folder to another (within the library folder)
- if the user modifies author, title (e.g. by using the finder or an app to edit the file) or even tags using a third party application (of course there needs to be a standard)

  • if the user deletes a file (EF should show this file its trash)
  • if the user saves a new file directly into a library folder

Note that this (sync FS -> EF) can be done very efficiently, there is no need to use polling to detect changes. Use e.g. spotlight or folder actions. If you implement smart folders it is even possible to replicate them into finder smart folders!

Advantages:

  • transparency!
  • everything in EF can be searched for using spotlight, even if EF is closed
  • safety (if the database fails it can be recreated fromt he FS)
  • openness / future-proof: less dependence on EF because all information is also in the FS, allows usage of other metadata apps
  • changing the library using the FS can be much more convenient sometimes (no need to switch to EF); for operations like adding or deleting files te finder or open/save dialog is sufficient
  • modify metadata when EF is closed
  • display metadata using the finder (e.g. get info or adding new rows to the finder list view)
  • allows sending a file to someone else with all metadata attached
  • this is a step into the direction of desktop metadata integration, the next step might be to allow the user to add custom fields to a file (e.g. project, due date); this new fields could be displayed/sorted/searched in EF (record list or get info) and the finder (list view or get info). fields could be automcatically added based on folder or tags (e.g. tagging a record/file as receipt might add the fields ‘financial year’ (string), ‘total amount in EUR’ (real number), ‘submitted to tax office’ (yes/no); allowing the user to add fields or tags and folders gives metadata much more structure; and spotlight already supports this! (e.g. searching for ‘total amount in EUR’ < 100 to display small bills); it would be GREAT to be able to store bibtex entries directly on the pdf file, this would enable so many ways to make organization much more efficient (e.g. automatic generation of the bibtex library file from a folder containing pdf’s with such a bibtex entry).
  1. smart folders
    I know this is planned, but when (in how many months approx.) can we expect it this?
    Also, when is the next version due?

If I had enough time I would probably code an information management app that supports 3. by myself, but there is too much work at the moment…

Btw, is there a student discount for poor students like me? :wink:
This would be great!

Thanks!
Danny

Thanks for the detailed feedback. I’ll try to give you a general idea of what’s in store for EagleFiler. Please note that software development is unpredictable, and external factors can change my priorities, so I try to avoid making promises about particular features, versions, and time periods that I might not be able to keep.

I’m planning to add an option so that files can be moved into the library rather than copied.

The Capture With Options dialog is not scriptable. However, you could write a script that accepts dropped files, imports them into EagleFiler (using the import verb and the deleting afterwards option).

I don’t want to get into injecting code into other applications. However, I believe it’s possible using Default Folder X to set the Spotlight comments when saving. When EagleFiler supports importing the comments as tags, this would enable you to set the tags when saving.

Agreed. I intend to address this by (a) letting you save directly into the library folder structure, and (b) making it easier to create new documents from within EagleFiler.

I don’t think folder actions are suitable for this. However, Leopard has added some filesystem infrastructure that may help.

I agree that in principle this would be nice, but I don’t think the technology exists to do this synchronization completely. That said, I’ll be moving EagleFiler in that direction as possible.

I don’t think that’s what Spotlight uses. That said, I agree that it would be useful to export as much metadata as possible using extended attributes. Please note that the modification and creation dates are already stored using HFS.

As far as I know, SpotMeta works by hooking into Spotlight in an unsupported way and overriding all the built-in importers. It might make sense to make EagleFiler’s metadata available in a format that SpotMeta can use, but I have no plans for EagleFiler itself to hack into Spotlight in this way.

I don’t think this will be possible.

Which apps can edit a title stored in an extended attribute?

This would break the consistency where the folder structure in EagleFiler matches the folder structure in the library folder, because the Finder’s trash is not EagleFiler’s trash.

I don’t think that either Spotlight or folder actions provide the necessary support for doing this.

This can already be done, since EagleFiler backs up the metadata as invisible plist files.

Sure, but the Finder currently has no support for this. Maybe Path Finder will someday.

This already works with folders, due to the metadata plists.

Probably in 1.4, but we’ll have to see whether any complications arise during development.

Yes, please send me an e-mail from your student address.

“Agreed. I intend to address this by (a) letting you save directly into the library folder structure, and (b) making it easier to create new documents from within EagleFiler.”

This should be very simple: create a folder action (or sth similar) that notifies EF when there is a change somewhere in the library folder. ED then determines what change has happened and imports the new file or updates the library accordingly. This would already greatly simplify saving because one can continue using the application to further modify the file without the need to reopen.
(Actually I don’t understand why EF does not use sth like folder actions to get notified of new files in the import folder. That’s much better than polling every 30 seconds or when the EF window gets focus).

I also missed this feature, in my opinion this is the feature that would make EF really get integrated into any workflow flowlessly.

Actually I tried looking for a kind of “sync this folder” feature or similiar like in Expression Media (former iView MediaPro) or in Adobe Lightroom within EF.

Being able to save, let’s say a Word document, directily from Word into the desired folder in the EF library would be great, because otherwise you have to save the document with F1 and then moving it later to the desired folder within EF manually.

Any plans or are any working in progress for this?

What is “this”? I said in the previous post (which you quoted) that I was working on letting you save files directly into the library. If there’s an additional feature that you want, please describe it specifically. I’m not very familiar with Expression Media or Lightroom.

What is “this”? I said in the previous post (which you quoted) that I was working on letting you save files directly into the library. If there’s an additional feature that you want, please describe it specifically. I’m not very familiar with Expression Media or Lightroom.

Sorry Michael, actually I wanted to mean, how was going the development of the saving files directly into the library feature (sometimes a write faster than I think).

iView MediaPro is a very well know DAM program which was bought last year by Microsoft and renamed to Expression Media. It is capable of reading more than 100 diferent types of files, and if has among many other things, a “folder sync” feature. This application is a very well known among photographers around the world, although Apple Aperture and Adobe Lightroom are taking its market by storm. I use Expression Media and Adobe Lightroom in my daily work, I thought you knew this applications… my 2 cents about this.

Sorry, but I don’t want to get into stating the development status of various features.

I used to use it long ago when it was iView. I know what Lightroom is, but I use Aperture myself. I’m not familiar with the folder sync feature that you’re referring to.

This should work with EagleFiler 1.4.

You can do this with EagleFiler 1.5.