Four wishes for Lightroom

It’s 2011, a new year, and it’s likely that Adobe will put out a new version of Lightroom this year. With that in mind, it would be wonderful if the Lightroom team could implement the following features in the next minor or major version of LR:

  • Find and Replace within metadata (details here)
  • Faster navigation and rendering when working with large catalogs (details here)
  • Filter catalog for metadata conflicts (details here)
  • More accurate time of capture for movie files captured with an iPhone 4 or a Nokia N95, or other video camera that doesn’t supply sidecar THM files (detailed explanation here and bug report here)

Thanks!

About these ads

File corruption rears its ugly head

During the last few weeks, I’ve seen the following error in Adobe Lightroom 3.

There I am, editing photos, minding my own business, and when I try to view a photo for editing, I get the error message you see above: “The file appears to be unsupported or damaged.”

When I try to view the file in the finder, I get the same error message, this time directly from the OS: “The file […] could not be opened. It may be damaged…”

Here’s my workflow:

  1. Shoot RAW
  2. Import RAW files as DNG into Lightroom (LR converts them on the fly)
  3. Process in LR
  4. Export as JPG or as needed
  5. Back up the catalog and check its integrity weekly

It’s pretty straightforward, and that’s the way I like it. I currently have about 85,000 photos in my LR library, whose catalog is stored locally on my MBP, with the files (DNG, RAW, JPG and TIF) residing on a Firewire Drobo.

Fortunately, the file corruption is only temporary, meaning there’s an error somewhere along the way:

  • It could be Lightroom
  • It could be the DNG file format (because I haven’t gotten the error with RAW or JPG files)
  • It could be OS X: I wonder how much testing Apple did for 10.6.5 with volumes greater than 4TB
  • It could be the Drobo: it’s a big volume (4.4TB) with lots of data that’s constantly being updated, lots of I/O traffic

I don’t know, and I hope someone reading this has an answer.

What fixes the error every time is quitting LR, ejecting the Drobo, cycling its power, mounting it, and starting LR. I’ve also tried just restarting LR, or just ejecting the Drobo, but those methods didn’t work.

Time of capture metadata bug in iPhone 4 movie clips

Updated 9/12/10: I’m not sure any more if this is an iPhone 4 glitch or an Adobe Lightroom 3.2 bug. A thread has been opened in the Adobe Lightroom Forum, if you’d like to chime in there.

After upgrading the iPhone with iOS 4.1, I recorded a new video clip, imported it and some new photos into Lightroom, and the same wrong date and time appear for it.

According to a comment on my thread in the Apple Support Forums, the correct time of capture is displayed for iPhone 4 video clips elsewhere but Lightroom. And I also noticed that Lightroom displays the very same incorrect date and time of capture for video clips taken with the Nokia N95.

Updated 9/27/10: I’ve been in touch with Adobe, and it turns out this is a “designed” behavior. That is, because movie clips do not have EXIF data (there is no standard for EXIF data when it comes to them), they are assigned a random date and time as they’re imported into Lightroom. HDSLR video files are accompanied by a .THM file which stores the necessary EXIF data, and that’s why they show up properly.

Quoting from Davide M.’s (Adobe) response:

So I then had a look at our bug database and it turns our this is a known issue with mobile phones although somewhat out-with our control. Movie files do not technically have EXIF data or at least the standard has not yet been established. Since the import process can assign a timestamp to a movie file, we ignore this time stamp since it can be inaccurate, as shown by the example of your video file being changed by the simple process of email. Other applications while appearing to work fine, in fact are simply showing you the files creation date. If you were to duplicate the file, you will see that the timestamp in these other applications will change to the time the file duplication took place.

The reason why most DSLRs work is because they create a sidecar file containing that information. Files with no timestamp, such as the ones from the iPhone and the Nokia N95 do not create this and hence default to 1/1/04 when looking at the Loupe information overlay.

In the example you used, the Canon 7D creates a .THM sidecar file with the same name as the video file it generates. This contains all the data associated with the video file.

Still, this is problematic behavior, as it introduces erroneous times of capture in these movie files. So I asked Davide if it would be possible for Lightroom to be updated so that it writes a more accurate time of capture for these movie files. Thankfully, he agreed to log it as a feature request. Time will tell if this will make it into a future LR update. Quoting him below:

That’s certainly something I can log in our feature request list. Because this has been deemed ‘as designed’ by our engineering team (due to the lack of EXIF data in movie files) it is not technically a bug. None the less, I can see that this would be a useful addition to our application. Thanks for bringing this to our attention.

Thank you, Davide!

After downloading a few movie clips taken with an iPhone 4 (running iOS 4.0.1) onto my computer, I saw right away that their time of capture was incorrect, even though the iPhone’s time had been set up correctly. I took a few screenshots of the movie clips in Lightroom, which you can see below. Click on each to view them large.

This time metadata error happens when using either the main (back-facing) HD video camera, as shown above, or the front-facing VGA camera, as you can see from the screenshot below.

It looks like iPhone 4 records the same time for all video clips recorded with it, set at 1/1/04 1:44:24 AM.

It goes without saying that any digital video camera worth its salt will record the time of capture properly. The question, naturally, is when Apple will fix this glaring bug?

For comparison purposes, here is a screenshot of a Canon 7D movie clip, also shown in Lightroom. The time of capture was recorded properly, as was to be expected.

SmugMug, are you listening?

I’m disappointed with SmugMug over their continued lack of support for proper export and maintenance of photographs directly from Lightroom. Back in July, I wrote about the Flickr Publish Service in Lightroom, and wondered when SmugMug would introduce their own.

What I was really looking for (and I said this in the post) was a way for the publish service to identify what I’ve already uploaded and allow me to re-publish those photos where I’ve made changes to the metadata or to the processing. The official Flickr Publish Service didn’t offer that option.

A few of my readers (Gary, Chris, Russell, thanks!) pointed me to Jeffrey Friedl’s excellent plugins for Lightroom, and I’ve been using them ever since. As a matter of fact, I’ve switched over to them completely. I use them for all four web services where I currently publish photos (SmugMug, Flickr, Facebook and PicasaWeb). I don’t know what I’d do without them. Wait, I do know — I know for sure I’d be doing a LOT more work and spending a LOT more time uploading and maintaining my online collections.

With Jeffrey’s LR plugins, I was able to identify about 90% of the photos already uploaded to SmugMug, and about 75% of the photos already uploaded to Flickr. In the case of Flickr, I then did manual updates and re-identifies so I could get it to know 95% of the photos already uploaded. This means Lightroom now allows me to quickly identify, update and replace almost any photos I’ve got at SmugMug, Flickr, Facebook and PicasaWeb. This is huge.

There is a catch, though, and it’s a BIG one. I keep running into the same “Wrong Format ()” error with SmugMug, which means I still haven’t been able to straighten out the photos I’ve uploaded to them. Here are a couple of screenshots of the error messages I get. It starts with a “TimedOut” error, then I get the “Wrong Format ()” error, then the upload process aborts.

I get these errors almost every time I try to re-publish an updated photo, but I don’t get them as often when I try to upload new photos. To give you an idea of how bad things are, I’ve currently got 109 photos to update in one of my galleries at SmugMug, and last night, I had about 167 photos. I’ve had to restart the re-publish process about 30-40 times since last night. You do the math, but I think it works out to 1-2 photos per error. This sucks. I should be able to just click the Publish button and walk away, knowing all of my changes will propagate correctly.

I’ve contacted Jeffrey, and I’ve contacted SmugMug. I’ve had extensive email conversations with each. SmugMug alternates in their replies. They’ve said the following to me:

  • It’s a fault with the plugin
  • It’s something on their end but they’re working on it
  • There’s nothing they can do about it
  • I should use something else to upload photos
  • They blamed my setup, which we ruled out after some internet connectivity tests

Jeffrey says there’s nothing he can do about it, and I believe him more than I believe SmugMug. Want to know why? Because his other plugins work just fine. I’m able to re-publish updated photos to Flickr and Facebook and PicasaWeb without any problems. Only SmugMug somehow can’t handle my uploads.

I’ve tried reloading the plugin, installing it anew, removing and re-adding the publish service, upgrading the plugin, but nothing. I still get the same errors.

My question for the smug folks at SmugMug is this: how is it possible that Facebook and Flickr and PicasaWeb have worked out the re-publish issues, but you haven’t? What’s taking you so long? Why can’t you work out the same problem on your end?

I was hoping that with the release of Lightroom 3.2, and the release of the official SmugMug Publish Service for LR (hat tip to David Parry for the advance notice), that SmugMug would work out the kinks in their API, but it looks like they still haven’t done it. I tried their plugin, but of course they took the easy route, like Flickr, and haven’t introduced any functionality that would identify photos already uploaded to their service. Only Jeffrey Friedl’s plugins offer this feature.

This leaves me terribly disappointed. As a SmugMug Pro, I don’t want to bother with error messages. I don’t want to bother with posts like this. I’d rather post photographs and update my SmugMug galleries in peace, but I can’t.

If you’re having the same problems with SmugMug, please, write to them, and ask them when they’re going to get their act together. This problem’s existed for several months. How much more time will it take until they deal with it?

Where’s the SmugMug Publish Service for Lightroom?

I love the Flickr Publish Service in Lightroom 3, and would love to see SmugMug make their own.

The only thing missing for the Flickr service is for it to know which photos I’ve exported and uploaded to Flickr before the service became available, in previous versions of Lightroom. I for example have either tagged the photos uploaded to Flickr with, obviously enough, “Flickr”, or have added them to a Flickr collection in Lightroom, so I could easily find them.

Here’s where SmugMug has the chance to shine! I’d love to be able to publish my photos to SmugMug directly from Lightroom, using the Publish Services functionality, so I could always sync up any photos that I’ve re-developed or where I’ve updated the metadata. But for this service to really stand out, it needs to know which photos I’ve already uploaded.

You can see where this is going, right? I’ve already tagged all my SmugMug photos, and have already placed them in collection sets and collections that match my SmugMug categories breakdown exactly. With a little bit of computing power and some smart algorithms, the folks at SmugMug could put together a killer Publish Service for Lightroom that incorporates all the Flickr functionality and bests it by matching my already-uploaded photos.

What about the cost? The Flickr Publish Service is free to use for all Flickr users, but you cannot re-publish uploaded photos if you’ve changed them in Lightroom. (You can, but if you’re not a Pro, it’ll wipe out any comments and faves on the photo, so it’s not advisable.)

SmugMug could use a similar approach. Their Publish Service could be free for basic SmugMug users, with limited functionality, and it could offer full functionality to Power and Pro users. (I myself have the Pro membership.) I’d even be willing to pay a one-time fee to download and install the service, because I think the functionality would be amazing.

Stop the headache – generate 1:1 previews before editing

One of my gripes with Lightroom ever since I started using it was the image blurring that took place as it generated image previews or re-rendered images while in Develop mode. (I started using LR in February 2007.)

It looks like Adobe listened, and the image preview rendering that takes place as I develop photos isn’t noticeable anymore — that, or my faster laptop has something to do with it, too. However, the very noticeable lag in generating either standard or 1:1 previews still occurs as I browse through images in my catalog, and that can’t be helped even by my zippy MacBook Pro. As you move through images, Lightroom will blur them until it generates a standard preview, then blur them some more you zoom in, until it generates the 1:1 preview.

Fortunately, there’s a solution for it. I’m not sure if this existed from the start, or if it was introduced in later versions of the software, but you can choose to generate 1:1 previews for a set of photos before you begin working with them. (I found this out thanks to this article from Steve Paxton at O’Reilly.) Just select the images you plan to work with on a given day, and go to Library >> Previews >> Render 1:1 Previews.

lightroom-generate-previews

If you choose that option, Lightroom will also render Standard-Sized Previews in addition to the full ones, allowing you to work with the photos right away, in standard or loupe view, with no lag or blurring (well, your computer’s specs might also have something to do with it). Still, if you’re sorting through a large set of images (hundreds or thousands), this pays off handsomely, in ways that you cannot even appreciate until you start popping aspirins to deal with the tension headache caused by all that screen blurring you could have avoided if you planned ahead.

If you’ve been a long-time subscriber to my site, then you may know about an article of mine written in January ’08, entitled “The next stage for Lightroom“. In it, I described the need for Lightroom to:

  1. Allow the storage of photos from its catalog on multiple volumes
  2. Allow people to work with photos from the catalog even when external volumes were disconnected
  3. Allow for the storage of the previews database (which can be very sizable) on an external volume, or for its splitting into two parts

Guess what? Most readers just couldn’t get what I was saying. But Adobe listened. Points one and two have already been implemented in later versions. Now I can store my photos on multiple volumes, and I can work with their meta-data even when I can’t access the image files because I’ve disconnected the external drives. As for the the storage of the previews, that’s now easily solved, too.

Because I’m now storing my Lightroom catalog on my laptop, where hard drive space is an issue, this means I have to limit the size of the previews database. I do this by giving it very little play. I tell LR to generate medium-quality 1024 px standard previews, and to delete 1:1 previews after a day. The previews database is no joking matter. Just a short while back, LR was set to discard 1:1 previews after a month, and my previews database had ballooned to over 60 GB!

lightroom-previews-settings

So, even though I allow myself the luxury of generating 1:1 previews for hundreds of photos, as you’ve seen above, the size of my previews folder stays manageable, and the free space on my hard drive stays where it needs to stay, because Lightroom cleans up after itself. Instead of worrying about free space, I allow my MacBook Pro to flex its processing muscle for 15-30 minutes before I dig into a large set of photos from a particular location, and then I can work undisturbed and headache-free for that day.

Get the tiltshift look right from Adobe Lightroom

If you use Adobe Lightroom and want to apply a tiltshift effect to your photos, you can spend hundreds of dollars on expensive Photoshop plugins, or you can do it for free, with an Adobe AIR app called TiltShift.

If you’ve used TiltShift before, you know you can open any photo in it and apply tiltshift effects to it, but did you know you can do this right from Lightroom? Here’s how.

In Lightroom, open up the Export window and add a new Export Preset. See the screenshot below. I called mine TiltShift, so I can easily remember it. Adjust any of the settings, like color space, sizing, sharpening, etc. They don’t really matter, although it’s better to keep the image smaller so TiltShift can work faster with it.

The really important option is in the post-processing section — the very last one in the Export window. There, you’ve got to make sure you tell Lightroom to “Open [your photo] in Other Application…”, then click on the Choose button and browse to find the TiltShift app. This is pretty much it.

lightroom-tiltshift-setup

Lightroom will automatically pass your image to TiltShift, which will open it and allow you add tiltshift effects to it, to your liking. For example, I initially processed this image of a medieval water pump found on the streets of Medias, Romania, in Lightroom.

The old water fountain

Then I exported it into TiltShift using the export preset set up as described above, and adjusted the settings there to get the effect I wanted. This is how the controls and the image looked inside TiltShift.

tiltshift-screenshot

Once I did that, I saved the photo and uploaded it here. This is how the final image looks.

water-fountain-tiltshift

It couldn’t be easier, and again, let me remind you TiltShift is a free app.

[TiltShift home page] [Download TiltShift]