Run WordPress by itself or cached?

I’ve been running an experiment for the past three months. I wanted to see how well WordPress would do if I ran it by itself, without any sort of caching. So far, so good.

About four months ago, my web server kept getting pummelled into the ground almost daily, and I couldn’t figure out why it kept happening. After researching the issue, I found the prevailing opinion to side with the need for a caching plugin. People were complaining that it’s just not optimized well, and must be run with the aid of such a plugin, otherwise higher levels of traffic will bring the web server down. Trouble was, I already ran my WP install cached, using WP Super Cache, had been doing so for over a year, and my server still went down. (I should specify it had only recently started to go down.) What was I to do?

I posted a message in the WP forums asking why WordPress doesn’t generate static files. Were there any plans to do so in the future? To my surprise, Matt Mullenweg (WP’s founder) replied to my post, and told me that while there are caching plugins out there, WordPress.com doesn’t run any, and they’re doing just fine hosting millions of blogs. Others chimed in as well, and their replies got me to make the following changes:

  1. Made the switch to a VPS (Virtual Private Server) with SliceHost. Four months later, I’m still very happy about that move.
  2. Doubled the RAM on my web server (to 512MB from 256MB).
  3. Turned off WP Super Cache and started running my site by itself.

Each step followed the other in succession. I wanted to make gradual changes so I could see why my server kept having issues. Switching to a VPS host was good, and it was needed, but for my traffic levels, it wasn’t enough. Doubling the RAM was good and it was needed, and while the new RAM is enough for now, I’d still be having problems if I didn’t also disable my caching plugin.

Here’s where I think the crux of the caching/non-caching issue lies: it has to do with the load placed on the server as cached versions of the pages get created. Normally, that’s a non-issue. But as I monitored my server carefully, I discovered that it went down only as it started to get indexed heavily by search engines. Their bots visited my site in spurts, with traffic peaking, then falling back down. They spawned multiple threads, over ten at times, following links and slurping up the content. It’s when bot traffic peaked that an incredible load was placed on the web server. It kept generating cached versions of pages it hadn’t already cached, RAM and CPU demand increased to unsustainable levels, and it went down.

No amount of tweaking the Apache and MySQL config files helped with this sort of scenario, or at least it didn’t help me. You see, the difference between peak traffic levels with search engines vs. people is that people will go to a single article or a group of articles that are in demand. A caching plugin works great for those sorts of situations. There’s a limited number of pages to worry about caching, and those pages get served up time and time again. The load is acceptable. When a search engine bot starts indexing your site, it’ll call up any and all available pages that it can find. That can place a huge load on the web server as it scrambles to serve up those pages and build static versions for the caching plugin. I believe that it’s too much for most medium-sized servers to handle, and they will usually go down.

In my case, disabling the caching plugin and making sure no traces were left in the .htaccess file were the only things that helped. Now, I might have up to four different search engine bots crawling my site, each spawning multiple threads, and my server will usually not go down. Sure, there are times when the server will get dangerously low on RAM, and will be unresponsive for 5-10 minutes, but that’s an acceptable scenario for me. And if I should all of a sudden get huge amounts of people traffic to a post, it’s possible that the web server will also become unresponsive, at least for a time. But the great thing about running WordPress by itself is that Apache will usually take care of itself. As the requests die down, Apache will kill the extra threads, the available RAM will go back up again, and the server will recover nicely. That wasn’t possible while I ran the caching plugin. When it went down, it stayed down, and that was a problem.

I realize that what works for me may not work for others. I have not tested what happens with WP Super Cache on a larger server, for example one with more RAM. It’s possible that the larger amount of RAM there will offset the greater demand placed on the server as it builds static versions of the pages, although I’m not sure what to say about the CPU usage. That also peaked as the caching plugin went crazy. Not sure how that’ll work on a more powerful server.

WP Super Cache has some options that allow you to cache more pages and keep them cached for longer periods of time. Perhaps fiddling with those options would have allowed me to keep running the plugin, but I wanted to see how things stood from the other side of the fence. Like I said, so far, so good. Caveats aside, running WordPress by itself was the cure for my persistent web server outages.

About these ads

Just give me a good zoom lens, thanks

Greetings from Osttirol! My wife and I have been vacationing in Austria for the past week. It’s a gorgeous place to visit and, needless to say, I took tons of photos here. I’ve been carrying my Canon 5D and my lenses with me everywhere, and let me tell you, I’ve been sorely in need of a good zoom lens.

The lens inventory in my camera bag is woefully short at the moment. I started out with three primes: EF 24mm f/1.4L, EF 50mm f/1.4, and EF 100mm f/2.8 Macro. I sold the 24mm prime with the intention of buying the EF 24-105mm f/4L Zoom, but other circumstances intervened, and now I’ve only got the 50mm and 100mm lenses.

There are some who say it’s better to have prime lenses. I disagree. I’d like to see them carry five or six prime lenses in a backpack up and down a mountain in order to get the range that one or two good zoom lenses would give, and then tell me if they still feel the same way. And by the way, try changing lenses in swift mountain breezes, with insects buzzing around you and just dying to get inside the sensor chamber and leave smudge marks (which happened to me). Oh, and don’t forget to throw in a few other accessories such as polarizers and UV filters of various sizes for the different diameters of each lens, plus one or two water bottles and a fleece jacket plus an umbrella in case the weather goes bad, and then we’ll talk…

In a way, I was glad to only have to carry two lenses; I’d have really felt the weight of a third one. But I felt so limited in the photos I could take, because I could only use the 50mm or the 100mm lens to frame my photos. In some instances, I could walk back and forth to get a better view or angle, but in others, there was no way to get a better photo without also being able to fly — which incidentally, would be very nice, but I haven’t figured out how to do it yet. And no, I don’t believe in cropping. I only do it when I absolutely have to. I didn’t pay $2,800 for a full-frame sensor that can take 12.8 megapixel photos so I could crop them and get the same resolution I can get from a $500 camera.

To this day, I slap my head when I think that I could have had the 24-105mm zoom lens as a kit lens with my 5D for a little over half its usual price. I was such a fool not to get it! It’s a light and sharp zoom with more range than the EF 24-70mm f/2.8L, and you can easily walk around with it for hours without getting too tired.

So far on this trip (which ends very soon, unfortunately) I took 1904 photos with the 50mm prime, and 471 photos with the 100mm prime. If I had had (don’t you just love the English language) the 24-105mm zoom on my trip, it’d have stayed on my camera 95% of the time, because that’s the range I use the most, particularly on the wider end of that focal spectrum, which was not available to me, each and every day, how stupid could I be, ugh…

Look, I’m not knocking the 50mm prime, which is a great prime, and very cost effective given its low light capabilities and sharpness. And I’m definitely not knocking the 100mm prime, which is versatile and a fantastic macro lens with gorgeous bokeh. But I really didn’t need f/1.4 or macro capabilities for landscape photography, which is what I did on this trip. I needed a zoom lens!

So, if you’re not sure what lenses to get, don’t do what I did, or you’ll be frustrated to no end as well. First get a good, lightweight zoom lens, one that won’t kill your wrist as you carry your camera around taking photos. Later, as you find that you need more specific capabilities, such as being able to take handheld photos at dusk or dawn, or more bokeh, or macro photos, then spring for those primes that have the features you need.

Get iMovie '08 to import 3GP files

I recorded a few video clips with my wife’s mobile phone today, and when I tried to import them into iMovie ’08, I found out that I couldn’t. Apparently, I’m not alone, because if you do a search for this on Google you’ll find there are plenty of other people with the same problem.

Fortunately, there are two ways to work this out, but neither is necessarily obvious. Pick one of them:

  • Open up the clips in Quicktime. I was able to open them up and play them right away. Now, export each clip as an MP4 file. You must specifically select File > Export, then select Movie to MPEG-4 from the Export drop-down menu. Don’t just Save the clip, because it’ll package it as a MOV file, which iMovie ’08 will still refuse to import. You must Export it as an MP4 file. You may also be able to export to other formats, but MP4 is what worked for me.
  • Open them clips in Turbo.264, then export them to some format (try Apple TV or iPod Standard, for example). There’s a catch here though. If the resolution of the clips is too small, Turbo.264 may not be able to convert them, and will give you an error. Best stick with Quicktime then.

Now open up iMovie ’08, and go to File > Import Movies, browse for them, and import to your heart’s content. That’s it!

If Time Machine doesn't work…

… and you get the little exclamation sign within the Time Machine icon in the menu bar, and Time Machine will not back up your Mac any more, then here’s what worked for me, twice so far:

  • Reboot the Mac.
  • Before doing anything else, go into the Time Machine drive, locate your Mac’s folder inside the Backups folder, and look for a single file that starts with a date and ends like this: .inProgress. Move it to the trash.
  • Tell Time Machine to “Back Up Now”.

That’s it. It should start backing up again. But if it doesn’t, you may want to visit the Apple support forums and see what worked for others. Some are saying you’ll need to toggle the backup disk to None, then back to the usual backup drive.

:arrow: Updated 8/14/08: Make sure you delete the .inProgress file once you move it to the Trash. If you can’t delete it, do a Get Info and make sure you have Read & Write privileges to it, then delete it. It may take a while to delete it, but let the Finder finish the job, don’t cancel it. If you don’t delete that file from the Trash, Time Machine may continue to give you errors and remain unable to back up your Mac.

If iMovie '08 crashes…

You can try bothering with the preferences files, or the codecs, or the color profiles, as the forums tell you to do, or you can do the following:

  1. Go to your home folder > Movies > iMovie Events. Locate the folder with the last clips you imported. Delete the iMovie Cache and iMovie Thumbnails folders.
  2. Restart iMovie. For most of you, that’s all you’ll need to do.
  3. If it still crashes, that means one or more of the actual clips themselves weren’t closed properly by the video camera (perhaps you ran out of battery, space on the card, etc.). Open them up in Quicktime (if you have Quicktime Pro) or some other application (like the software that came with the camera) and snip 1-2 seconds off the end of each clip. Save them, then delete the iMovie Cache and iMovie Thumbnails folders (in case iMovie re-created them).
  4. Now restart iMovie. It should work fine now.

A few words to explain some things:

  • If a movie clip doesn’t open at all from the Finder, then it is corrupted. Perhaps only its header is messed up for some reason. At any rate, I have not yet found software that will fix those movie clips. I searched enough, and only found spyware that promises to do it.
  • If a movie clip opens and plays fine in Quicktime, but iMovie crashes when trying to import it, then its ending is corrupted, as explained in step #3 above. Cutting 1-2 seconds off the end of the clip in some software other than iMovie will restore its ending and let you import it without crashes in iMovie.
  • The method I described above has worked reliably for me on three occasions now, and this is after I wasted hours on the phone with Apple Support. It only occurred to them to let me re-import video clips after having me try all of that other stuff, including a re-install of iMovie ’08 and also the creation of a new account to see if the problems re-occur there. Pointless. I say go to the root of the problem and take care of it right away, don’t beat around the bush.

Hope this helps you too.

How to upgrade WordPress via SSH

If you know how to log in via SSH (Secure Shell Access), then you will be able to upgrade your WordPress site in three minutes or less by using the following lines of code.

I have to admit right away that I’m highly indebted to this pre-existing tutorial from Techtites. But that tutorial is a little dated for the newer versions of Linux, and one of the commands given there no longer works on my web server, because it’s been deprecated (I use 1and1). I thought it useful to provide the right commands in this post, and to keep it updated in case something changes.

A few words of CAUTION:

  1. BACK UP all of your site files and your site database before running the upgrade. Take the time to do both, or you may deeply regret it later. As a matter of fact, it’s a great idea to back up your site files and database on a weekly basis, if not more often, just in case you get hacked or the web server crashes, etc.
  2. I’m not a Linux expert. I’m just glad I found these commands and that they’ve made the upgrade process easier for me. Don’t ask me to help you configure this for your web server. If the commands don’t work there, or something gets screwed up, you’re on your own.
  3. Should you use the WP-Cache plugin, disable it and delete any cached files BEFORE running the upgrade process. Even better, disable ALL your plugins before the upgrade process. If you don’t do this, you may get a big, fat 500 error afterwards.

Now, initiate a SSH session to your web server (I use Putty). Your web host should have the directions on how to do this. Go to the root level of your site/WP install folder (this is NOT the same as the root level of your SSH login).

Once you’re at the root level of your WordPress install (the one where you can see the wp-config.php file), enter the following Bash commands, in the order they appear. Wait for each of them to execute successfully before proceeding to the next one.

This will download the latest version of WP directly from WordPress.org:

wget http://wordpress.org/latest.tar.gz

This will unzip it, creating a directory called wordpress:

tar xfz latest.tar.gz

This will delete the wp-includes and wp-admin folders:

rm -rf ./wp-includes/
rm -rf ./wp-admin/

This will take you inside the wordpress folder:

cd wordpress/

This will copy everything inside the wordpress folder to the root level of your site, overwriting any existing files and directories. This line is the only line that’s changed from the Techtites tutorial:

cp -rpf -f * ../

This will take you back out to the root of your WordPress install:

cd ..

This will delete the wordpress directory and the downloaded WP archive, since they’re not needed any more:

rm -rf ./wordpress/
rm -f latest.tar.gz

Hope this helps!

Condensed knowledge for 2008-03-12