A review of Google’s Backup and Sync

google-drive-to-backup-sync

Google launched this new service in the second half of 2017. I remember being prompted by the Google Drive app to install an upgrade, and after it completed, I noticed a new app called “Backup and Sync” had been installed, and the Google Drive app had become an alias.

Screen Shot 2017-12-26 at 14.08.28.png

The new app sat there unused for some time, until I discovered its new capability, namely to back up and sync other folders on my computer, not just the Google Drive folder. This was and is good, new functionality for Google, because it ties in very nicely with its Photos service, which has already been offering the ability to back up all of the photos and videos taken with mobile devices to the cloud through the Google Photos mobile app. I’ve been using Google Photos for several years, going back to when it was called Picasa Web.

I set it to back up all of my photos and videos, allowing Google to compress them so I could back up the whole lot. (It’s the “High quality (free unlimited storage)” option selected in the screenshot posted below.)

Screen Shot 2017-12-26 at 15.01.14.png

I already back up all of my data with Backblaze, which I love and recommend, but it doesn’t hurt to have a second online backup of my media, even if it gets compressed. Having lost some 30,000 images and videos a few years back, I know full well the sting of losing precious memories and when it comes down to it, I’d rather have a compressed backup of my stuff than none at all.

Screen Shot 2017-12-26 at 14.15.12.png

The thing is, there are shortcomings and errors with this new service from Google, which I will detail below. The backup itself was fast. Even though I have several terabytes of personal media, they were uploaded within a week. So that’s not the issue. After all, Google has a ton of experience with uploads, given how much video is uploaded to YouTube every single day.

Screen Shot 2017-12-26 at 14.08.51.png

As you can see from the screenshot posted above, it was unable to upload quite a few files. The app offers the option of uploading RAW files in addition to the typical JPG, PNG and videos, but it couldn’t upload RAW files from Olympus (ORF), Adobe (DNG) and Canon (CR2). They were listed among the over 2700 files that couldn’t be backed up.

Screen Shot 2017-12-26 at 14.09.56.png

I ended up having to add the extensions of RAW, PSD, TIFF and other files to an “ignore” list located within the app preferences. This is the full list I’ve added there so far: DNG, TIFF, RAF, CRW, MOV, PSD, DB, GRAPHDB, PLIST, and LIJ. It seems there’s a file size limit on images and on videos, because most of my large images (stitched panoramas) and videos of several GB or more didn’t get uploaded. That’s a problem for an app that promises to back up all your media.

There were also quite a bit of crashes. The app crashed daily during the upload process and even now, it crashes every once in a while. I set up my computer to send crash reports to Apple and to the app developers, so I assume that Google got them and will at some point issue an upgrade that fixes those bugs.

I also kept running out of space on my Google account. Given that I’d set the app to compress my images so I’d get “free unlimited storage”, and I’d also set it to back up only my images and videos, this didn’t and doesn’t make sense. Add to this the fact that it’s trying to back up unsuccessfully all sorts of other non-image files (see the paragraph above where I had to add all sorts of extensions to the ignore list) and once again, this app seems like it’s not fully baked. I ended up having to upgrade my storage plan with Google to 1 TB, so it’s costing me $9.99/month to back up most (not all) of my images and videos, compressed, to a service that offers “free, unlimited storage”. The app says I’ve now used up 408 GB of my 1 TB plan. Before I started backing up my media, I was using about 64 GB or so, adding together Gmail and Google Drive. So about 340 GB are getting mysteriously used by some invisible files that I can’t see in Google Photos or Google Drive, but they’re obviously stored somewhere by the Backup and Sync app.

Remember, this is Google. They have a ton of experience with apps, with images and with videos, so why did they push this out when it still has all these issues?

Fun with technology

I’ve had multiple Drobo units since 2007. To this day, I still enjoy adding a hard drive to a Drobo. It’s one of those things that can be an ordeal on other tech, but on a Drobo, it’s been made fun through proper planning and design.

It lets you that it’s low on space, you order a drive, and when it comes, you look at the app, which tells you exactly what size-drive is in each bay. Pressing a small lever on the side of the bay releases the drive, which slides out. You put the new one in, the Drobo immediately checks it and formats it, then begins striping the data set across it.

Screen Shot 2017-12-21 at 12.48.30.png

I love this process. It’s so simple and so fun! The Drobo doesn’t care what hard drive you buy, as long as it’s larger than what you already had. It allows you to grow the capacity of your Drobo in time, as the prices for newer, bigger hard drives decrease, without any sort of headaches. This is technology done right.

Permanent data storage

We need to focus our efforts on finding more permanent ways to store data. What we have now is inadequate. Hard drives are susceptible to failure, data corruption and data erasure (see effects of EM pulses for example). CDs and DVDs become unreadable after several years and archival-quality optical media also stops working after 10-15 years, not to mention that the hardware itself that reads and writes to media changes so fast that media written in the past may become unreadable in the future simply because there’s nothing to read it anymore. I don’t think digital bits and codecs are a future-proof solution, but I do think imagery (stills or sequences of stills) and text are the way to go. It’s the way past cultures and civilizations have passed on their knowledge. However, we need to move past pictographs on cave walls and cuneiform writing on stone tablets. Our data storage needs are quite large and we need systems that can accommodate these requirements.

We need to be able to read/write data to permanent media that stores it for hundreds, thousands and even tens of thousands of years, so that we don’t lose our collective knowledge, so that future generations can benefit from all our discoveries, study us, find out what worked and what didn’t.

We need to find ways to store our knowledge permanently in ways that can be easily accessed and read in the future. We need to start thinking long-term when it comes to inventing and marketing data storage devices. I hope this post spurs you on to do some thinking of your own about this topic. Who knows what you might invent?

How to create a Fusion Drive on a mid-2011 iMac

Yes, you can enable Fusion Drive on older Macs. I’m not sure how this method will work with Macs older than 2011, but I know for sure that it works on mid-2011 iMacs, and quite possibly on other Macs made since then. I have just completed this process for my iMac and I thought it would help you if I detailed it here.

I like Fusion Drive because it’s simple and automated, like Time Machine. Some geekier Mac users will likely prefer to install an SSD and manually separate the system and app files from the user files which take up the most space, which is something that gives them more control over what works faster and what doesn’t, but that’s a more involved process. Fusion Drive works automatically once you set it up, moving the files that are used more often onto the SSD and keeping the ones that are accessed less often on the hard drive. This results in a big performance increase without having to fiddle with bash commands too much.

The hardware

My machine is a 27″ mid-2011 iMac with a 3.4 GHz processor and 16GB of RAM. I bought it with a 1TB hard drive, which I recently considered upgrading to a 3TB hard drive but decided against, given the fan control issues with the temperature sensor and the special connector used on the factory drive.

imac-basic-specs

I purchased a 128GB Vertex4 SSD from OCZ. It’s a SATA III (6 Gbps) drive and when I look in System Info, my iMac sees it as such and is able to communicate with it at 6 Gbps, which is really nice.

ocz-vertex4-ssd-128gb

ssd-specs

The hardware installation is somewhat involved, as you will need to not only open the iMac but also remove most of the connections and also unseat the motherboard so you can get at the SATA III connector on its back. You will also need a special SATA wire, which is sold as a kit from both OWC and iFixit. The kit includes the suction cups used to remove the screen (held into place with magnets) and a screwdriver set.

2nd-drive-ssd-kit

You can choose to do the installation yourself if you are so inclined, but realize that you may void the warranty on the original hard drive if something goes wrong, and this is according to Apple Tech Support, with whom I checked prior to ordering the kit. Here are a couple of videos that show you how to do this:

In my case, it just so happened that my iMac needed to go in for service (the video card, SuperDrive and display went bad) and while I had it in there, I asked the technicians to install the SSD behind the optical drive for me. This way, my warranty stayed intact. When I got my iMac back home, all I had to do was to format both the original hard drive and the SSD and proceed with enabling the Fusion Drive (make sure to back up thoroughly first). You can opt to do the same, or you can send your computer into OWC for their Turnkey Program, where you can elect to soup it up even more.

The software

Once I had backed up everything thoroughly through Time Machine, I used the instructions in this Macworld article to proceed. There are other articles that describe the same method, and the first man to realize this was doable and blog about it was Patrick Stein, so he definitely deserves a hat tip. I’ll reproduce the steps I used here; feel free to also consult the original articles.

1. Create a Mountain Lion (10.8.2) bootup disk. Use an 8GB or 16GB stick for this, it will allow you to reformat everything on the computer, just to clean things up. Otherwise you may end up with two recovery partitions when you’re done. I used the instructions in this Cult of Mac post to do so. The process involves re-downloading 10.8.2 from the Apple Store (if you haven’t bought it yet, now is the time to do so) and an app called Lion Diskmaker.

2. Format both the original HD and the SSD, just to make sure they’re clean and ready to go. Use Disk Utility to do this, or if you’re more comfortable with the command line, you can also do that (just be aware you can blow away active partitions with it if you’re not careful).

2. List the drives so you can get their correct names. In my case, they were /dev/disk1 and /dev/disk2.

diskutil list

3. Create the Fusion Drive logical volume group. When this completes, you’ll get something called a Core Storage LGV UUID. Copy that number, you’ll need it for the following step.

diskutil coreStorage create myFusionDrive /dev/disk1 /dev/disk2

4. Create the Fusion Drive logical volume. I used the following command:

diskutil coreStorage createVolume paste-lgv-uuid-here jhfs+ "Macintosh HD" 100%

5. Quit Terminal and begin a fresh install of Mountain Lion onto the new disk called “Macintosh HD”.

6. Restore your apps, files and system settings from the Time Machine backup using the Migration Assistant once you’ve booted up. Here’s an article that shows you how to do that. When that completes, you’re done!

The result

Was it worth it? Yes. The boot-up time went from 45-60 seconds to 15 seconds, right away. And over time, the apps and files I use most often will be moved onto the SSD, thus decreasing the amount of time it’ll take to open and save them.

At some point, I expect Apple to issue a utility, like Boot Camp, that will allow us to do this more easily and automatically. Until then, that’s how I set up Fusion Drive on my iMac, and I hope it’s been helpful to you!

Hardware preview: ioSafe N2 NAS

ioSafe, the company famous for its line of rugged external drives that can withstand disasters such as floods, fires and even crushing weight, has come up with a new product: the N2 NAS (Network Attached Storage) device.

The N2 device comes at the right time. The market for NAS devices is maturing and demand is growing. Western Digital has even come out with a line of hard drives, the WD Red, specifically targeted to NAS enclosures. To my knowledge there is no such other NAS device out there, so ioSafe’s got the lead on this.

The N2 appliance is powered by Synology® DiskStation Manager (DSM) and is aimed at the SOHO, SMB and Remote Office Branch Office (ROBO) markets.

The high performance 2-bay N2 provides up to 8TB of storage capacity and is equipped with a 2GHz Marvel CPU and 512MB of memory. The N2 uses redundant hard drives as well as ioSafe’s patented DataCast, HydroSafe and FloSafe technologies to protect data from loss in fire up to 1550°F and submersion in fresh or salt water up to a 10 foot depth for 3 days.

Features:

  • Local and Remote File Sharing: Between virtually any device from any location online
  • Cloud Station: File syncing between multiple computers and N2 (like Dropbox)
  • iTunes Server
  • Surveillance Station: Video surveillance application
  • Media Server: Stream videos and music
  • Photo Sharing: Photo sharing with friends and family
  • Mail Server: Email server
  • VPN Server: Manage Virtual Private Network
  • Download Station: Post files for others to download
  • Audio Station: Stream audio to smartphone (iOS/Android)
  • FTP Server: Remote file transfers
  • Multi-platform compatibility with Mac/PC/MS Server/Linux

Hardware:

  • Dual Redundant Disk, RAID 0/1, Up to 8TB (4TB x 2)
  • 2GHz Marvel CPU and 512MB memory
  • Gigabit Ethernet Port
  • Additional ports for USB 3, SD Memory Card
  • User replaceable drives
  • Protects Data From Fire: DataCast Technology. 1550°F, 1/2 hr per ASTM E119 with no data loss.
  • Protects Data From Flood: HydroSafe Technology. Full immersion, 10 ft. 3 days with no data loss.
  • FloSafe Vent Technology: Active air cooling during normal operation. FloSafe Vents automatically block destructive heat during fire by water vaporization – no moving parts
  • Physical theft protection (optional floor mount, padlock door security – coming Q1 2013)
  • Kensington® Lock Compatible

Support and Data Recovery Service (DRS):

  • 1 Year No-Hassle Warranty (for N2 Diskless)
  • 1 Year No-Hassle Warranty + Data Recovery Service (DRS) Standard (for loaded N2)
  • DRS included $2500/TB for forensic recovery costs for any reason if required
  • DRS and Warranty are upgradeable to 5 years ($.99/TB per month)
  • DRS Pro available includes $5000/TB + coverage of attached server ($2.99/TB per month)

Operating Environment:

  • Operating: 0-35° C (95°F)
  • Non-operating: 0-1550°F, 1/2 hr per ASTM E119
  • Operating Humidity: 20% – 80% (non-condensing)
  • Non-operating Humidity: 100%, Full immersion, 10 feet, 3 days, fresh or salt water

Physical:

  • Size: 5.9″W x 9.0″H x 11.5″L
  • Weight: 28 lbs

The N2 appliance is being brought to market with funding obtained through IndieGogo. I know it’s hard to believe it when you look at their products, but ioSafe only has about 20 employees. Sometimes they have to be creative in the ways they fund their R&D.

The ioSafe N2 will begin shipping in January 2013 and will be available in capacities up to 8TB. Introductory pricing for the ioSafe N2 diskless version is available for $499 on Indiegogo ($100 off the retail price of $599.99) if you want to get your own hard drives.

I’ve also written about ioSafe Solo, the ioSafe Rugged Portable and the ioSafe SSD devices.

What’s next in data storage?

My recent musings on high definition and the state of the technology behind it have spurred me to think about data storage (not that it’s a new subject for me). But so far, I’ve commented only on what’s already been developed, and didn’t take the time to think about what’s next.

What’s the motivation behind this post? It’s simple. For Ligia’s Kitchen, it costs me about 10.5 GB for 5 minutes of final, edited footage of show, with a one-camera setup. What goes into the 10.5GB? There’s the raw footage (and sound files, if I use a standalone mic), the edits, and the final, published footage. When I use two cameras, the space needed can easily go up by 1.5-2.5x, depending on the shots I need to get. I shoot and edit in 1080p, and output to 720p.

My storage needs are okay for now. I’ve got plenty of space, and if I keep going at this rate, I should be fine. But… and there’s always a but, isn’t there… I have more show ideas in mind. And there’s the hypothetical possibility of shooting with a RED camera at some point in the future, if certain factors come together to allow it. So I’m thinking ahead.

Current hard drive technology (bits of data on disks) has certainly come a long way. Those of us who’ve been in the business long enough know what prices used to be like for capacities that are laughable by today’s standards. Back in 1999, I paid $275 for a 27GB hard drive. My laptop’s drive in college could store a grand total of 120MB. And when I began to learn programming, I’d load the code into memory from tape…

I remember being really excited about Hitachi’s new Perpendicular Magnetic Recording Technology, which came out in early 2006. They even had an animation on their website, which they’ve taken down since. That technology is behind all of the new hard drives that are on the market today, by the way. Hitachi came up with a way to get the bits of data to stand up (hence the term perpendicular) instead of lying down on hard drive platters, thus doubling the amount of data that could be stored onto them.

There are two roads ahead when it comes to data storage, of which one is more likely to succeed:

  • Optical storage (this is probably the future of storage)
  • Biological storage

Let’s first look at biological storage. One particular article made the rounds lately: researchers at the Chinese University in Hong Kong have managed to store 90GB of data in 1g of bacteria. While it sounds exciting, the idea of storing my data in petri dishes on my desk doesn’t readily appeal to me, and certain complications come up:

  • 1g of bacteria is about 10 million cells (that’s a LOT); one must start thinking about the potential for bio hazards when you work with bacteria.
  • The data is stored in a bacteria’s DNA, which means it’s encrypted (a good thing), but it’s also subject to significant mutation (a bad thing) and it takes a long time to retrieve it because you need a gene sequencer, which is tedious and expensive (a bad thing).

I’m not against this. Hey, if they can make it safe and fast, okay. But I believe this is going to be relegated to special applications. The article suggests the technique is currently used to store copyright information for newly created organisms (I wonder how many new bacteria researchers as a whole have created, and is it any wonder antibiotics have such a hard time working against them when we keep playing God). I also see this sort of data storage as a way for spies to operate, or for governments to keep certain secrets.

Okay, onto more cheery stuff, like optical storage. I’ve always thought there was massive potential here, and am glad to see significant work has already been done to make this a reality. There are two technologies which are feasible, according to research that’s already been done:

  • HDSS (Holographic Data Storage Systems), which so far can store up to 1TB of data in a crystal the size of a sugar cube, but doesn’t yet allow rewrites
  • 3D optical data storage, which so far can store up to 1TB of data onto a 1.2mm thick optical disc

These developments are very encouraging. Optical storage is safe, and its potential capacities are huge, possibly endless. And when you think about computer hardware, and how manufacturers are looking at using optical technology in the bridges and buses and wires inside the hardware, because it’s incredibly fast, you start to see how optical makes sense. Let’s also not forget fiber optic cabling, and its incredible capacity to carry data. It certainly looks like optical is the future!

So what’s going to happen to the standard 3.5″ form factor of today’s hard drives? Well, it’s likely that it will stay the same, even though it the storage technology inside it might change. We’ll have crystals and lasers instead of platters and heads, but they’ll likely be able to fit them in there somehow. I don’t think we’ll need to start keeping crystal libraries on our desks, like in Superman’s Crystal Cave, and sticking various-sized crystals into our computers any time soon, although it did look pretty cool when Christopher Reeve did it in the movie.

It really all depends on how soon this new technology will come to market. Right now, there’s clearly enough vested interest in the 3.5″ and 2.5″ form factors to motivate drive manufacturers to shoehorn the new technologies into those shapes, but if optical hard drives won’t be here for the next 5-10 years, then it’s possible that the form factor will change as well. We are after all moving to smaller, sleeker shapes for most computers, notebooks and desktops alike.

Hardware preview: the ioSafe Rugged Portable

I’m a fan of ioSafe‘s rugged hard drives, which you can trust with your data through very rough conditions. I was glad to write about the ioSafe Solo (their first product), and the ioSafe SSD, because no other company on the market offered such unparalleled protection from destruction.

Now they’re launching a new model, the ioSafe Rugged Portable — their first drive made for travel.

As you can see, they’re using a new design, with a machined Aluminum or Titanium enclosure that can withstand up to 5,000 lbs (Ti) or 2,500 lbs of pressure (Al). The drive is suspended on all six axes of motion, and can withstand a drop from 20′ (SSD version) or 10′ (HDD version).

Of course, the drive can still withstand immersion in water — up to 10′ (Al) or 30′ (Ti), both up to 3 days in duration. And it’s got a whole other bunch of protections built in as well:

  • ChemSafeTM Technology – Full immersion in diesel fuel, oils, hydraulic fluids, aircraft fuels, 12” depth for 1 hour with no data loss per MIL-STD-810G Method 504
  • EnviroSafeTM Technology – Continuous exposure to UV, blowing sand, blowing dust, rain, salt fog, icing or freezing rain, 24 hours with no data loss per MIL-STD-810G Methods 505.4, 506.4, 509.4 and 510
  • AltiSafeTM Technology – High altitude operation. 15K ft. (Alum.) and 30K ft. (SSD and Ti.) rated altitudes per MIL-STD-810G Method 500.4
  • Theft Resistant Kensington® Lock compatible slot solid metal construction – theft protection

As you’ve no doubt gathered so far, there are multiple flavors of the drive, with HDDs or SSDs inside, and Aluminum or Titanium enclosures. And it ships with USB 3.0, FW800 and USB 2.0 connections — your choice.

Possibly the best feature — given the drive’s title — is its weight, which comes to 1 lbs for the Al enclosure and 1.5 lbs for the Ti enclosure.