Tag Archives: backups

Reviews

Synkron and I don't see eye to eye

I’ve been using Synkron lately to keep several directories synced up across a couple of external drives — a backup of sorts, just in case one of the drives decides to kick the bucket. I’ve also been cleaning up my drive, and something drove me bonkers: I couldn’t figure out why, in spite of my best efforts to clean up the drive, I couldn’t get 100GB free or more on my MBP (total HD size is 250GB). It just didn’t make sense; after all, I’d been able to do it just a couple of months ago, before I filled up my drive with photos from Austria and Romania. Why couldn’t I do it now?!

I decided to download Disk Inventory X, a drive space analysis tool based on WinDirStat, which I’d already used on a couple of Windows machines. What do you think was the biggest culprit on my MBP’s hard drive? The Synkron cache.

It was eating up 30.6GB of my space, without me even realizing it…

Look, Synkron, you may have a nice, shiny icon, and you may be able to put a nice GUI on rsync or subversion (not sure which you’re built on), but I’m sorry, when you can’t clean up after yourself or at least take up less space on my drive, you’re no good to me — into the Trash you go.

If you’re using Synkron as well, and you’ve noticed your free space shrinking inexplicably, you may want to give Disk Inventory X a try, and to consider whether another utility to sync your files may be a better option. I’m not ruling out Synkron completely, and I hope that its developers decide to fix this cache issue at some point.

Standard
How To

If Time Machine doesn't work…

… and you get the little exclamation sign within the Time Machine icon in the menu bar, and Time Machine will not back up your Mac any more, then here’s what worked for me, twice so far:

  • Reboot the Mac.
  • Before doing anything else, go into the Time Machine drive, locate your Mac’s folder inside the Backups folder, and look for a single file that starts with a date and ends like this: .inProgress. Move it to the trash.
  • Tell Time Machine to “Back Up Now”.

That’s it. It should start backing up again. But if it doesn’t, you may want to visit the Apple support forums and see what worked for others. Some are saying you’ll need to toggle the backup disk to None, then back to the usual backup drive.

➡ Updated 8/14/08: Make sure you delete the .inProgress file once you move it to the Trash. If you can’t delete it, do a Get Info and make sure you have Read & Write privileges to it, then delete it. It may take a while to delete it, but let the Finder finish the job, don’t cancel it. If you don’t delete that file from the Trash, Time Machine may continue to give you errors and remain unable to back up your Mac.

Standard
Thoughts

What's on my desk?

The idea for this post has been sitting among my drafts for a long time. I always thought it’d be a good idea to share how I work and what equipment I use, for the benefit of those of you that want to put together a similar setup. Let me make it clear that I’m not doing this to brag. I realize my equipment is fairly expensive and out of reach for some. Believe me, I’m keenly aware of its cost, and have made certain sacrifices in order to acquire it.

The hardware I have now is the best hardware I’ve had so far, and it allows me to easily develop for my clients, write for my sites, process my photographs and edit my videos and movies. See the photo enclosed below. Each important piece of equipment is marked with a number, and each number is explained in detail.

Here’s what you see above:

  1. 15″ Apple MacBook Pro: I love its dedicated 512 MB video card, and its design and speed. It allows me to focus on my work, not twiddle my thumbs while the computer plays catchup to my commands.
  2. Drobo: I have two of them. I store backups of my photos on one, and my videos on the other. Both of them are shared over my home network, so that my other Mac, a 20″ iMac G5, can access the videos and play them in Front Row.
  3. 2TB WD My Book Studio Edition II drive, running in RAID 0 (striped) mode: WD was kind enough to give me this drive as a gift of sorts, considering the significant problems I had with my 1TB WD MyBook Pro Edition II drive. I think they got tired of my complaining, and sent it to me to shut me up. Well, it worked. This drive has done great so far, and I have no complaints. I plan to write a review for it soon. I’ve been putting it through its paces so far, and it’s held up nicely. I connect it to my MBP through the Firewire 800 port, and I love the transfer speeds.
  4. 250GB WD Passport drive: I bought this last year, and it has been very useful. It’s small, stylish, and it’s powered solely by the USB port. I use it a LOT to transfer big files between computers.
  5. Turbo.264: a nifty little gadget built by the folks at Elgato, it speeds up H.264 conversions significantly. I’ve been putting this through its paces as well, and plan to write a review soon.
  6. mStand laptop stand: I love this stand, and reviewed it recently. It’s the best-designed stand for the MBP.
  7. newerTech miniStack v1: This is the first version of the popular miniStack drive enclosure. It uses PATA drives, and it has built-in FireWire and USB hubs (3 ports each). I use it as a USB hub, and it sits underneath my laptop stand — that’s why it’s not clearly visible in the photo. Both my Drobos are connected into it, and I have a spare USB port that I can use for my CF card reader, etc. If you want to have a better look at this enclosure, you can see it better in my Drobo review, where it appears in both the photos and the video.

As you can see, I aimed for simplicity in my setup. I wanted the most amount of processing power and hard disk space, in the smallest amount of physical space. You’re looking at about 5TB of storage in the photo.

Most of the heavy lifting (in terms of data crunching) happens on the WD Studio drive. I use it as a staging area for video processing, and transfer the finished, edited ones to one of the Drobos. I also use it to store and work with my Lightroom photo libraries. Working with photos in Lightroom from a Drobo is too slow for me, and the WD Studio Drive offers a visible advantage as I process each photo. I then back up my laptop and the WD Studio Drive to the Drobo through Time Machine.

You may recall I initially moved my photo libraries to one of the Drobos because I kept experiencing Lightroom library corruption and thought the WD Studio drive was at fault. As it turns out, Lightroom itself is to blame. Once your photo library reaches tens of thousands of photos (I have over 40,000 photos in one of my libraries), Lightroom will experience library corruption on a fairly regular basis. It’s just an unstable piece of software, and I hope that the next version will be better.

Just a quick note for those of you that noticed it. There’s a wire that runs behind the laptop, along the wall. The photo isn’t crooked, the wire is. I use it to hang small wires or notes to myself on it, and being loaded down with a few toward the center (which is only partially shown in the photo), it appears slanted. Sorry about that. It bothered me, but I wasn’t going to spend an hour or so in Photoshop cloning it out.

Hope this helped give you some ideas! If you have any questions, let me know.

Standard
Lists

Condensed knowledge for 2008-03-11

Standard
How To

How I moved all my content from comeacross.info to raoulpop.com

A place in this world

Background info

As midnight approached this past New Year’s Eve, I was busy working on a long-term project. I was about to move all of my content (every article and post I’d written) from comeacross.info to raoulpop.com. There were many reasons for this, but consolidation was the most readily apparent.

As detailed on my About page, I’d already combined my content from other sites of mine onto comeacross.info, but there was one more piece of the puzzle that needed to fall into place. I’d alluded to it already. I was thinking about doing it in 2006, believe it or not. As a matter of fact, when I sat down and thought about whether to start writing at comeacross.info or raoulpop.com, I knew deep down I should choose to start writing on my personal domain, but worried it might be too difficult for people to remember and type the name.

After a year or so at ComeAcross, I realized that the subjects I was writing about were much too varied for a standalone site. I was writing in a personal voice, using a lot of 1st person, and it only made sense to have that sort of content reside on my personal site. Plus, there were so many splogs (spam blogs) on the .info TLD, that I worried whether I would be taken seriously if I stayed on .info. I’d owned raoulpop.com for a long time, I wasn’t really putting it to good use, and it didn’t make sense not to.

I set a deadline of 12/31, and got to work on planning and research. What better time for such a big change as this than New Year’s, right?

I’m documenting this for you because someone else might need to know how to do it. And I figure the thought process that went on behind the scenes is also worth knowing.

Planning and research

My biggest challenge was to figure out how to redirect all of the traffic from comeacross.info to raoulpop.com, reliably and accurately. I needed to make sure that every one of my articles and posts would redirect to my new domain automatically, so that a URL like

http://comeacross.info/2007/12/30/my-photographic-portfolio/

would automatically change to

https://raoulpop.com/2007/12/30/my-photographic-portfolio/

and the redirect would work in such a way that search engines would be properly notified and I wouldn’t lose my page rank.

I knew about 301 redirects, but I wasn’t sure how to accomplish them in the Linux/WordPress environment the way that I wanted them to work. I had worked mainly with Microsoft web servers until recent times, and Linux was and still is fairly new to me. I was using John Godley’s Redirection plugin for WP (it’s an awesome plugin btw), and I knew it could do 301 redirects quite nicely. I had been using it heavily when I changed post slugs or deleted/consolidated posts at ComeAcross.

I worked out a line of Regex code that I could use to create a site-wide redirection, I tested it and it worked fine. In case you’re wondering, you can easily test it by creating a 307 (temporary) redirection instead of a 301 (permanent) redirection. Here’s how to do it:

Create a new 301 redirection where the source URL is

/(.*)

and the target URL is

http://www.example.com/$1

Make sure you check the Regex box, add it, and you’re done.

Just to make sure, I contacted John Godley to confirm whether it was the best way to do things. He said that would certainly do the job, but there was a MUCH easier and faster way to do it, one that saves a lot of the overhead that comes into play when WP gets used. It works through the .htaccess file. He was kind enough to provide me with the code, which is reproduced below.

<IfModule mod_rewrite.c>

RewriteEngine On

RewriteRule ^(.*)$ http://www.example.com/$1 [R=301,L]

</IfModule>

Just paste that into your .htaccess file (remove all other code but make sure you back it up somewhere in case you need it), save it, upload it, and you’re done.

Don’t do anything yet though! Not before you’ve thoroughly backed up everything! Let me outline the steps for you, and keep in mind that I wanted to mirror all of my content from two separate WP sites using the same WP version, and to redirect from the first to the second. These two conditions have to be met in order for my advice to apply to your situation.

  1. Make sure both sites are on the latest and greatest version of WP, or at least they’re on the SAME version of WP
  2. Back up the database from the old domain
  3. Download all site files from the old domain
  4. Upload site files to new domain
  5. Restore database to new domain
  6. Make changes to .htaccess file as shown above
  7. Log into your new domain’s WP admin panel and change the site and blog URLs. Now you’re done! Check to make sure the redirection works properly and all of your content is there.

Upgrade your WP installs

The two sites have to be on the same version, or else things might not work as expected. Upgrade both sites to the latest and greatest, or at least make sure they’re on the SAME version before you do anything else. Go to WordPress, download and install the latest versions. There’s also an Automatic Upgrade plugin, but I haven’t tried it yet, so I can’t vouch for it.

BEFORE you do any sort of upgrade, you need to back up. Yes, you can’t get away from this… You’ll need to do two backups, one before you upgrade, and one after you upgrade, before you transfer the content.

Back up your content

This combines steps 2 and 3 listed above. Backing up your site files is easy. Use an FTP client to access the files on the web server and download them to your hard drive. I always keep a local copy of my site files. It just makes sense.

Backing up your database is a little more involved. Your database contains all of your site content (posts, links, comments, tags, categories, etc.) so you definitely don’t want to lose it. There are detailed instructions on backing up the database on the WordPress site. You can follow those, or you can go to your site’s Admin Panel >> Manage >> Export and download the WordPress WXR file, which you can import into your new site afterwards.

While this is great for backups, restores are another matter. I tried it and found that the import operation kept timing out at my web host. Given that I have thousands of posts, I didn’t want to sit there re-restoring the WXR file only to get a few posts done with every operation. I needed something quicker.

There is a plugin called WordPress Database Backup which lets you download a zipped SQL file of the database. You can use this to restore the database through the MySQL Admin Panel, if your webhost provides you access to it.

What I did was to simply point my new site install to my old database. This is a very handy and easy solution if you plan to host both sites with the same web host. But this still doesn’t excuse you from backing up the DB before you upgrade the WP install! 🙂

Restore your content to the new site

This is a two-step process (see #4 and #5 above) and involves reversing the steps you took during the backups. You will now upload your site files to the new domain, and you will restore the database to the new domain as well. If you’re in my situation, where you’re using the same web host, you can simply point the wpconfig.php file on your new domain to the old database.

Make sure all your content is properly restored before going on to the next step!

Make changes to the .htaccess file

You will need to make sure you don’t touch the .htaccess file before you transfer it to your new domain. Only the .htaccess file on your old domain needs to change. Remember this, or you’ll be wondering what’s going on with the redirects afterwards…

Use the code I’ve given you above, in the Planning and Research section, to make changes to the .htaccess file on your old domain, after you’ve made absolutely sure that all of your content is now mirrored on the new domain. Once this is done, the redirects will occur automatically and seamlessly.

Final checks and tweaks

This is very important. Surf to your old URL. You should get re-directed to your new URL. Do a search in the search engines for content of yours that you know is easily found. Click on the search results and make sure the links get redirected to your new site. Because you’re using 301 redirects, the search engines will automatically change their search results to reflect the URL changes without affecting your page rank, so you shouldn’t lose any search engine traffic if you execute the content move correctly.

There are a few more things you’ll need to check:

If you’d like to make changes to your site feed (and I did), you’ll need to handle that properly. I use FeedBurner, and there are people that subscribe to my content via RSS or via email. I needed to transfer both groups of subscribers to my new feed seamlessly. The FeedBurner folks helped me do just that, and I didn’t lose a single subscriber during the move. I detailed that process in this post.

What about internal links? If you’ve blogged for a while, you’ll have linked to older posts of yours. Those link URLs now contain the old domain, and you’ll need to change all of them at some point, or you’ll risk making those links invalid if you should ever stop renewing your old domain. Fortunately, there’s a Search and Replace plugin for WP that lets you do just that. It works directly with the database, it’s very powerful, and it’s very fast. That means you have to be VERY careful when you use it, because there’s no undo button. You can easily mess up all of your content if you don’t know what you’re doing.

What I did was to replace all instances of “.raoulpop.com/” with “.raoulpop.com/“. That did the trick nicely. I then did a regular site search for all instances of ComeAcross and manually made any needed changes to those posts. (Here’s a thought: back up the DB before you start replacing anything. This way you can restore if something should go wrong.)

Finally, if you’re using the Google Sitemaps Generator plugin, you’ll want to make sure you manually rebuild your site map. You don’t want to have your old site information in the site map as Google and the other search engines start to crawl your new domain.

That’s about all I did for the site content transfer. It occupied half my New Year’s Eve night, but it was worth it. It’s quite a bit of work, but if you plan it out, it should only take you 4-5 hours or less to execute the transfer, depending on your familiarity with this sort of thing, and the speed of your internet connection (keep in mind that upload speeds are a LOT slower than download speeds on most broadband connections).

Given how much work is involved, I was a bit surprised to see Matthew Mullenweg (founding developer of WordPress) talk about doing his own switch to a new domain in “2 seconds“. I think what he referred to is the changes to the .htaccess file and the blog URLs, which are the fastest parts of the process. There is, however, quite a bit of work that needs to take place behind the scenes before those switches can get flipped. And I also believe (someone correct me if I’m wrong) that he pointed both domains to the same web files — in other words, re-used his existing WP install — so he bypassed a lot of the steps that are otherwise required.

Hope this proves helpful to someone!

Standard