Zen and the Art of Workflow Maintenance

I have received a number of emails asking how I manage the workflow of images from my camera to my library and how I go about protecting and backing up my photos. With this post I hope to share some of the tips (and pitfalls) I’ve discovered in trying to simplify, reflect, readjust and further simplify the way I manage my images.

First things first and this is the most important point before we continue; There is no right or wrong process when it comes to workflow! Every photographer approaches their workflow in a way which makes sense to the type of shooting they do and their own personal preferences. TWIP podcasts #62 and #63 on workflow illustrate how diverse each of the podcast members’ workflows are. I continually seek to refactor my workflow, looking for the simplest set of processes which meet the requirements I have for my images.

Antoine de Saint-Exuper once said:

“Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.”

This is my ideal, thus this post encompasses a snapshot in time and, if I work hard and keep looking to simplify and improve further, it probably won’t be relevant in a month’s time – it will have changed into something which works better for me.

Setting the Scene

Initially I recommend getting a feel for what resources you have at your disposal. Do you have any external hard drives or USB memory keys lying about. What computer do you use? How many photographs and what type of photography are we talking about; are they family snaps or client’s wedding portfolios?

If you have a lot of photographs like I do then you probably have some of them on your primary computer and the rest floating around in some sort of disconnected storage; an external disk, a set of DVDs or some file host on the Internet. Questions I’d ask here are: Does your primary photo management software (be it iPhoto, Photoshop Elements, Picasa, Aperture or Lightroom) know where all the photos are? Are offline files referenced properly and can you search for information associated with them?

Do you have some way of keeping the important photos offsite? What if, God forbid, the house burns down or someone nicks your hardware in the middle of the night?

Do you have some sort of archival strategy in place? In 10 or 20 years time will those scratched DVDs in the drawer still work?

Below is a diagram of my hardware and storage situation. It may seem very complicated at first glance but it is a breeze to set up and is automated to the point where I can forget about it and get on with the business of taking pictures. I will talk through aspects of the diagram later in this post.

Getting my Photos off the Camera

This is the easiest part of the process. I pop my card into a reader and plug it into my computer. Aperture opens and asks where I want to save my files. I import all new files into the Aperture library. On import Aperture renames the files to reflect the date and time the image was taken and applies an import preset which stamps the files with basic metadata such as Copyright information.

Keywording

This is the next task I perform and I have it first because for me it is the most important way to find information about my photos and search across years of archives. It was the most neglected aspect of my workflow for a long time and the sweat and effort to keyword 4 years worth of photos taught me the valuable lesson ‘keyword on import’. If there are any recognisable events in the photos I can quickly bundle them up in a selection and use the Aperture album creation hotkey to turn the selection into an album to which I can refer later.

First Pass gut feel

 Next comes a quick pass over my import. I go into full screen mode and quickly move through the photos not paying too much attention to the details. I trust my gut feel here enough to know if the composition is good or bad. In Aperture I flag photos I want to mark as picks with a 4 star rating. Lightroom users can do the same or use the ‘flag as pick’ option to do the same. I prefer the star rating because the graduation means something to me which we’ll get to later.

At this point I can leave the process and carry on with it later if I like. I have my photos in a managed environment. They are keyworded, in albums as required and I have identified the couple which are important to me.

 I also usually click the button to backup my vault at this point. The vault backup contains my library (keywords, metadata, adjustments, file library and any managed files). As this process is a synchronisation rather than a full backup every time it is usually really quick and updates only the files/information which has changed.

Adjustments: Time to whip out the Loupe

At some point I’ll want to take a closer look at the technical aspects of the photos. I usually go through my picks with the loupe looking at the finer details and making minor adjustments to contrast, sharpness and exposure as required. Photos which I want to retain as picks but need more serious adjustments I will rate as 3 stars and then attend to them in a later editing session. The photos I will use for my portfolio I will rate as 5 stars. The rating system allows me very quickly to move between all the photos that need work (3), are part of the pool of picks (4) or are my very best shots which will make it into books or onto walls (5). The majority of my Online photos (Flickr, Ipernity etc.. ) come from my 4 star pool. At any point if you feel the photo doesn’t make the grade you can simply remove the star rating.

Second pass gut feel

As this step can happen at any time I recommend you leave a bit of time between your import and adjustments and the second pass on your non picks. In this workflow you will be working only with your picks for most of the time seeing the non-picks only when you look at the contents of an album for example. It is usually a good idea to take a little time at a later date, sit down and browse through photos you felt didn’t make the grade. I am constantly surprised by the little gems I find which then finally get to make it as a pick.

Non picks become referenced files, picks become managed

Photos I’ve selected not to be picks leave my laptop hard disk – there really is little point in them eating up my precious free space. In Aperture you can easily move files around using the ‘Manage referenced masters’ option. Select a group of pictures and put them on a drive of your choosing – in my case above my 1 terabyte drive. The files are still referenced by Aperture and you can still browse and search on the files. If I need to use any old files such as a group of bracketed photos for a HDR image then the ‘Consolidate masters’ option sucks the files back into my library (alternatively I can plug back into the big drive and export the masters to a local folder).

I have set up a couple of smart folders which make the identification of files that need to move easier:

• identify files to reference: looks at any file which has not been rated and exists on the local volume
• identify files to consolidate: looks at any file which has a rating > 3 and exists as a referenced master. This is useful in bringing any photo identified as a pick in the second guess gut feel back into my library.

So that’s really it, a simple set of steps which have ensure I have a set of photos sorted into albums and picks which are easy to browse and search on and are easy to backup.

Make sure your photos walk out the door

The last point I want to make is ensure you have some sort of offsite backup available. You can have the fanciest systems, terabytes of storage, a wall of Drobos but putting all your images in one place leaves them vulnerable to loss through natural disaster or theft.

I rsync (a sync command in Leopard or other UNIX based systems) my referenced photos to a seperate portable Hard drive and ensure that Aperture also keeps a vault on that disk. This disk goes everywhere with me, when I’m going out for the day it is either in my camera bag or in the cool centre console of my car. At work it lives in a bag in my desk drawer. I know that a meteorite can hit my house (in this scenario my family and pets are vacationing in Fiji) and that all my expensive computer equipment can go up in a ball of flame and I will not lose a single image. I can buy new furniture, I can build my house again brick by brick from the foundations but I cannot replace my wedding photos, or the pictures of my son or my portfolio.

Cloud Backup

 Call it simple paranoia but my portfolio photos are very important to me and represent a tangible emotional and dollar investment. These photos are usually converted to DNG as my long term archival file format of choice and a copy of the photo with keywords and adjustments are then saved as a full size JPEG. These files are exported to a local backup folder where a little piece of software picks them up and uploads them to secure cloud storage.

My cloud backup of choice is Jungle Disk. Jungle disk is part of Amazon.com’s S3 storage services. They allow you to create an encrypted drive using a key you provide. You can setup the software to automatically backup files from selected locations to your cloud drive. The main reason I like them as a provider is that you only pay for your files in transit (bandwidth is charged for upload and download only) and having the online drive encrypted means the files are totally safe from being viewed or stolen. I also use Jungle disk to backup my important documents putting them safely in a place where I can access them from anywhere.

Wrapping Up

I won’t hark on much more on this topic of workflow and backups, you have a good idea of one possible workflow which may work in part (or entirety) for you. I am more than happy to answer any questions on actual details of this workflow or my storage solutions; just leave your question it in the comments and I will get back to you.

Now stop worrying about backups and get out and take some photographs!

Aperture to DNG to Adobe Camera Raw Workflow

I’ve just made a really useful Automator workflow for fans of Aperture. The workflow allows me to select any number of .NEF raw files (either managed or as referenced masters) and with a single click perform the following actions:

• Generate XMP sidecar files with all Aperture metadata and export them with the .NEF to a working folder
• Combine the .XMP and the .NEF into a .DNG using the latest DNG converter from Adobe
• Remove the temporary .NEF and .XMP files leaving only the DNG files
• Open a finder dialog, allow me to select one or all of the DNG files
• Opens my selection for editing in Adobe Camera Raw

This is important to me as it allows me to use all the power and flexibility of Apertures photo management and retain the use of ALL the brilliant non-destructive editing tools of Lightroom (which are all available in the ACR interface). Further heavy pixel pushing from ACR straight to Photoshop is a further button click away.

There is a little basic setup required.

• Go to this site, download and install the DNG Automator action.
• If you don’t have it yet, go to Adobe and download their free DNG converter. Install it.
• Go to my Skydrive and download my workflow (or application if you prefer), unzip it and run it.

You should see something like this:

The workflow should open in Automator. Replace the areas in the image above which say Adobe Photoshop CS4 with the version you have (if you don’t have CS4).

You can now either run the workflow directly from Automator or (from Automator) click: File > Save as plugin. Select ‘Script menu’ under the plug-in for section and choose a name. The script is added to your scripts folder in your taskbar and you can access it really easily when you’re in Aperture.

Done! The best of Aperture goodness with the heavy lifting of ACR and Photoshop for when you want it.

Time Machine saved me from my own impatience.

picture: Apple Time Machine

I am notoriously impatient, especially with hardware and computers. My abhorrence for slowness and lag has meant a costly exercise of hardware upgrades to keep up with the curve. Mix lag, slowness or system speed issues with my sanctified realm of photography and I grow horns and start breathing fire. And so, due to some weird quirk of software combinations combined with thrashing out some massive TIFFs through Silver Efex, my laptop started to disk thrash and throw up it’s electronic hands in surrender. It was 11pm … it had been a long day … I did not have much sleep the night before and now was faced with a computer which won’t compute; I did what any self respecting person would do and reached for the hard reset.

Not my shining moment.

On logging in I found my user profile was now horribly corrupt! I could log into other profiles no problem but not into mine – I could see my desktop background but that was pretty much it. Thankfully it was late at night with the family tucked up safe in bed for the expletives that resounded around the living room were a hark back in time to the harbour back-streets of 18th Century Amsterdam.

It was here that I realised my backup strategy combined with the use of Time Machine would pay dividends. I had never had to do a full system restore on my Mac before and was nervous whether it would work – thoughts of hours lost doing PC Windows reinstalls provided a bleak outlook for the rest of the weekend.

As an extra precaution I booted up with my Leopard disk, opened a terminal and backed up my entire user directory to my external backup drive – I figured at the very least I could get restore the important bits in the event the full restore didn’t work. The following is actually a one line command broken over 3 for simplicity:

tar -cvf

/Volumes/[my_backup_drive]/userprofile.tar

/Volumes/Macintosh HD/Users/Stuart/

I didn’t really need to perform this step but the extra safety net was of some comfort. For more info on the tar command look here or type ‘tar –help’ from your Mac command window.

I then selected the option to restore my system from a Time Machine backup, was presented with a list of all recent system snapshots (I selected one from two hours before), clicked ok and went to bed. In the morning I followed the prompt to restart the computer, held my breath and logged into my profile – it was like the problem had never happened, full seamless restore, user profile working like a charm and I was back in business.

Time Machine is complete peace of mind, the only upgrade I have planned in the near future is to point my Time Machine backup, photo library and all photo masters at a Drobo.

It’s nice when software works as it’s supposed to.