Our Infrastructure

Ready for the gory details? This is your last chance to hit “back” and get back to happy-soft-marketing-speak.

We serve millions of hits and terabytes of bandwidth per day, so it takes a serious infrastructure to make sure we can serve lots of traffic, really quickly, with a minimum of down-time.

Here’s how we do it.

YOUR HOME SERVER

“Clustered” or “Grid” WordPress systems are slow. They’re slow because by distributing everyone’s traffic over many servers, none of those servers can keep anything cached. This is especially true when there’s a few large sites (which we also host!) because they tend to bogart a lot of resources on all servers in the cluster.

So at WP Engine every blog is automatically assigned to a “home server.” All your blog activity is served from this one server, shared with other bloggers. To ensure there’s enough resources for you, we maintain enough RAM to ensure that the entire PHP opcode cache and the non-media file-system cache can be kept 100% in RAM, and we keep CPU load averages at or below 0.2 per CPU.

This server is for dynamic traffic only. As discussed below, the MySQL database is separate, memcached servers are separate, and static content is served separately too. This allows each of those servers to be tuned for a particular use-case, which is much more effective than trying to run different kinds of services on one server.

If you’ve got enough, you can sign up now.

A CLUSTER WHEN YOU NEED ONE

Of course the problem with the “home server” strategy is that you have a single point of failure: If that server is down or even just overloaded with traffic, your blog would become unavailable. We can’t have that!

So when a certain home server is failing, we automatically reroute your blog traffic to other people’s home servers. In other words, for the one clear benefit of a cluster — robust uptime — we still have that too! When your home server is fixed, traffic will route back.

All of this is invisible to your web visitors and even to you. In fact it’s normal for us to take home servers offline on purpose for maintenance or testing or upgrading hardware.

MULTI-PART FRONT-END: SECURITY, RULES, CACHING, AND STATICS

All your web traffic comes through a sophisticated front-end system involving multiple components.

First is a DoS appliance — hardware that detects Denial of Service attacks where hackers try to make a website unavailable by flooding it with packets from servers around the world. This appliance blocks all such attacks from reaching the back-end system (keeping it healthy), and is rated for 1.5 million packets per second of attack while still allowing normal traffic through.

Next is a firewall. We don’t even have port 22 open for SSH! (We log in using a VPN.) ‘Nuff said.

Next is an IDS (Intrusion Detection System) — a hardware appliance that scans packets for malicious activity such as known application exploits, email-harvesting, cross-site scripting attacks, SQL-injection attacks, etc.. This comes from a database which is regularly updated by security agencies. Right now we’re blocking about 7,000 attacks per day!

Next is our rules engine. We look at the request and decide which customer it’s for (which can be fairly complicated with wildcard domains and domains configured for WordPress Multisite). This decides not just the home server, but a large set of behavioral rules, many of which are common to all WordPress installations but some of which are specifically tuned for your blog. These rules are part of our “secret sauce” so we can’t get into too much detail, but to give you a simple example, we handle various 301 redirects that WordPress will do anyway, but we can do them at this layer incredibly quickly and without touching the web servers or databases.

For static content, e.g. images, javascript, and stylesheets, we serve them directly off of disk from this front-end system, protecting the back-end system from having to do that byte-movement. This results in faster response-times for your visitors and off-loads that much work from your home server.

Next is our caching system. Again this is part of our “secret sauce,” but we run the most sophisticated caching system in the world, which we’re able to do because we support only WordPress. This makes common pages (like home pages and feeds) load in 30ms (not including the time to make it all the way to the end user’s browser), protects the back-end system from most traffic and specific traffic from popular plug-ins and 3rd-party services, and yet is tied back into your WordPress system (via our system plugin) so that when the cache needs to be refreshed (e.g. post is updated) we refresh immediately so there’s never stale data.

Finally, for requests that make through this gauntlet and really do need to be served up by your home server, a load-balancer does the job of routing your traffic to your home server, or distributing your traffic if home is unavailable.

Phew!  Sign up today, and get all this for your site too.

MYSQL CLUSTER

Of course we run beefy servers for MySQL, and again the rule is “everything must run out of RAM” for it to be fast.

On top of that, we shard reads and writes out of WordPress so that the reads — which is the overwhelming majority of database activity — don’t hit the master server but are distributed over read-only slaves. This speeds up queries, increases cache-coherency on those queries, and allows the master to scale further than it would otherwise.

The read-slaves can become a master if the master becomes unavailable, so this doubles as a live backup.

INSIDE WORDPRESS

That’s just the stuff outside of WordPress itself, but a lot of the optimizations happen inside the page-request, thanks to plug-ins, some of which we wrote and some 3rd-party ones we use and even contribute code back to (open source, of course!).

There’s a lot of parts here, some of which is again secret sauce, but you can get a flavor for the types of things we do by checking out W3 Total Cache. We actually don’t use this plugin because we have better ways of handling each of the things that it does, and in a way that doesn’t require you to learn how to configure it yourself, but those are the same categories of things that we do.

Although there’s big things we do here with caching database calls, pages, changing HTTP headers, etc., sometimes it’s the little stuff that makes a big difference. For example, WordPress comes with support for most versions of jQuery; plugins and themes can declare their dependence on jQuery, and WordPress will automatically serve up that library. However this means you’re serving this standard library yourself (through us), and Google has graciously decided to host jQuery for free for the whole world if you use their URL instead. This is better for your blog, not just because Google’s servers are fast, but because so many websites use Google’s URLs it’s also likely that the browser already has that file cached, which means it’s even faster to load your blog! So, one of the zillions of optimizations we make is to automatically detect when WordPress is supplying jQuery, and we re-write the URL to go to Google instead.

Little things add up! That’s our job. Are you ready to sign up yet?

CDN (Content Delivery Network)

A CDN is a set of servers positioned all around the world which cache a copy of your content — usually statics like images, javascript, and stylesheets. When a request comes in for content on the CDN, it automatically determines which of these world-wide servers is physically closest to the requesting browser, and the content is delivered from that location.

The effect is two-fold: Content loads faster because it’s closer, and your site is more scalable because content is coming from many different servers.

WP Engine is the only hosting company on Earth (that we know of!) who bundles a CDN as part of our base service. So this awesome technology that’s normally reserved for large sites who want to spend a bunch of money, is now accessible to everyone.

On top of that, we do all the configuration for you, so you don’t lift a finger. You just run your blog normally — uploading content to our servers, nothing weird — and we automatically change the HTML links to use the CDN and automatically ensure your content is loaded into the CDN network.

MORE STUFF

There’s more…

We run separate memcached servers to support W3 Total Cache in caching page content, database queries, and the WordPress “Transient API.”

We run daily backups which also get transferred off-site. We also give you a plugin that makes it trivial for you to make your own backups if you want. Also when you sign up with us we give you a Golden Ticket for VaultPress if you want to use Automattic’s fully-automated, live backup service. Currently, you can only get an invite from Automattic or through us!

We run third-party website uptime/speed monitoring services so we know before you do if your site is down or performing badly.

We run through your files once a week, using Yahoo’s SmushIt to squeeze every last byte out of it with no loss in image quality. Makes your site that much faster to load and hey, it takes less space on our hard drives too.

Our hardware is awesome too — 10,000 RPM drives in RAID-10 configuration, XEON processors, running in VMWare so we can switch off of bad hardware, and top-quality network infrastructure.

Reconsidering doing this all yourself? Good call.
Sign up now, see what else we offer, or contact us with any questions.

I MUST HAVE MISSED SOMETHING…

We do a lot, so maybe you still have questions and maybe I’ve missed something in this long description.

If you have questions, just ask!  If you’ve read all this, and you still want to learn more, it’s time to give us a call: at 1-877-WPENGIN (1-877-973-6446).  Ask your hardest questions. We studied for the test.