Tag Archives: dng format

Thoughts

File corruption rears its ugly head

During the last few weeks, I’ve seen the following error in Adobe Lightroom 3.

There I am, editing photos, minding my own business, and when I try to view a photo for editing, I get the error message you see above: “The file appears to be unsupported or damaged.”

When I try to view the file in the finder, I get the same error message, this time directly from the OS: “The file […] could not be opened. It may be damaged…”

Here’s my workflow:

  1. Shoot RAW
  2. Import RAW files as DNG into Lightroom (LR converts them on the fly)
  3. Process in LR
  4. Export as JPG or as needed
  5. Back up the catalog and check its integrity weekly

It’s pretty straightforward, and that’s the way I like it. I currently have about 85,000 photos in my LR library, whose catalog is stored locally on my MBP, with the files (DNG, RAW, JPG and TIF) residing on a Firewire Drobo.

Fortunately, the file corruption is only temporary, meaning there’s an error somewhere along the way:

  • It could be Lightroom
  • It could be the DNG file format (because I haven’t gotten the error with RAW or JPG files)
  • It could be OS X: I wonder how much testing Apple did for 10.6.5 with volumes greater than 4TB
  • It could be the Drobo: it’s a big volume (4.4TB) with lots of data that’s constantly being updated, lots of I/O traffic

I don’t know, and I hope someone reading this has an answer.

What fixes the error every time is quitting LR, ejecting the Drobo, cycling its power, mounting it, and starting LR. I’ve also tried just restarting LR, or just ejecting the Drobo, but those methods didn’t work.

Standard
Thoughts

Metadata: DNG vs RAW

Generally speaking, I prefer the Adobe DNG format over the proprietary RAW format given to me by a camera, because I like the fact that it’s more or less future-proof. With a DNG file, the meta-data resides inside the file — like with a JPG — but the format is lossless, same as a RAW file — and unlike a JPG.

In spite of the fact that it’s a “publicly available archive format”, I would like to see more camera manufacturers adopt it, so I can feel more comfortable using it. I realize companies like Hasselblad and Leica have already adopted it, and you can take photos directly in DNG format on some of their cameras, but until the big camera manufacturers like Canon and Nikon adopt it, it won’t have the mass acceptance it needs to ensure its long-term survivability.

Still, I have begun to convert the RAW files in my photo library to DNG. By my count, I have converted about 30% of my 77,000 photographs to DNG format, and I am converting more of them every day. Let’s hope Adobe sticks to its word in the future and I’m not left holding the bag, having locked my photos into a format that might become obsolete.

Long-term benefits and potential caveats aside, I should point out a more current disadvantage between DNG and RAW. It has to do with metadata.

Yes, it’s true that with a RAW file, you’re stuck working with your metadata in sidecar XMP file, and that file may get corrupted, or you may lose it, thus losing your metadata and the processing directives for Camera Raw or Lightroom or whatever you’re using to process your photos. With a DNG, everything resides inside the file. There’s no XMP file, which is a good thing, most of the time.

But when you’re backing up your library, and let’s say for the sake of the argument that you’ve got to back up 20,000 photos, which is what I’m doing right now, and you’ve made minute changes to the metadata of all those files — only changed one EXIF or IPTC field — the backup software won’t care. You’ll have to back up 20,000 DNG files, each (in my case) between 12-24 MB. That’s going to take a LOT longer than backing up 20,000 XMP sidecar files, each of which is only 15-25 KB, because those are the only files that will have been changed if I update the EXIF or IPTC data for a whole bunch of RAW files.

That’s one area where RAW trumps DNG. I’m willing to overlook it if DNG will indeed prove to be a future-proof format, but that remains to be seen.

Standard