Submitted by patrick on Sun, 02/28/2010 - 22:48
Posted in
Where you ever hit by the social media? Making it to the frontpage of some large social network and getting thousands of extra visits in just a few hours is probably every bloggers dream... At least until it actually happens and the sudden traffic burst turns into a nightmare as the webserver grinds down to a halt.
In case you ever experienced this, or want to prepare for such a thing happening, I put together a few tips that should help to prevent the worst on standard LAMP type root servers.
I won't bore you with the basics here. You have probably already found the performance settings in the admin menu, activated caches, aggregated CSS/Javascript files and enabled page compression. Those things certainly do make a difference, but when push comes to shove, you'll have to do a lot better than that.
Before tuning your server, it is important to understand what happens to Drupal during a DDOS attack (after all, DDOS and social media attention both technically boil down to the same thing).
To simplify matters, let's assume that you are using Apache with MPM prefork that is, having a pool of server processes, each handling one client connection at a time. More processes are spawned as needed.
The problem with this model is obvious: The more connections there are at a given time and the longer it takes a client to finish, the more processes must be spawned in order to serve the requests. Spawning additional processes is generally undesirable. They cost extra RAM and the startup time adds latency. In case a page, instead of a file is requested, even more RAM is required to load and execute PHP, while each database query, required to build the page, creates further latency.
The server runs into trouble as soon as it runs low on memory. Once all physical RAM is exhausted, the operation system will have no choice but to start swapping out memory blocks. In worst case, this means that some (active!) webserver processes are swapped out in order to allow others to run. Once that happens, latency kicks in big time and HTTP requests get extremely delayed. Since unfinished requests block their associated processes and cannot be reused in this state, the server will start spawning more and more processes to serve incoming requests which leads to even more competition for RAM.
Eventually, the operation system will be busy with nothing but swapping, essentially locking the machine up.
So in essence, the key aspects of avoiding being DDOSed is to keep both memory consumption as well as latency as low as possible. Keeping this in mind as the golden rule, let's get down to it.
Prevent disk swapping at all costs
Once your webserver starts touching the swapspace, it is as good as dead, so make sure this never happens by setting the memory limits accordingly. Remember: It's always better to simply deny some requests right away than to have your system brought down and have even more users not see what they came for.
You can limit the number of processes using the MaxClients Directive. To figure out the maximum value for this setting, determine the size of your average Apache process, by looking at your process list via a tool such as top, and divide this into your total available memory, leaving some room for other processes.
Install APC
APC is the Advanced PHP Cache. You can find an installation instruction for Debian here. Installing APC is generally a good idea. It'll dramatically reduce page generation time. The faster a page can be served, the sooner the webserver can answer another request without having to spawn/fork additional workers to handle the load.
Of course, you should install APC before being hit with a massive traffic increase.
Provide static versions of your pages
Traffic from social media websites is largely anonymous. This means, large quantities of users will be served identical pages. By keeping the fully rendered pages around and not rebuilding them from scratch each time they are requested, you can tremendously cut down on both, RAM consumption and latency. A suitable caching mechanism is provided by the Boost module.
Reduce the nice level of your database
Make sure that your database process has a higher scheduling priority (= gets more CPU time) than your webserver. Think about it. If you cannot deliver a page without doing database queries and your system is under heavy load, it makes more sense to give priority to finish pending requests than to accept new ones.
Include all .htaccess files directly in the Apache config and disallow .htaccess
Drupal comes with a handy .htaccess file in the Documentroot. It conveniently sets up URL rewriting and protects include files from nosy visitors. It also comes with a performance penalty as the webserver has to load and parse at (least one extra file)[http://httpd.apache.org/docs/1.3/howto/htaccess.html#when] per page request. See AllowOverride directive for details.
Having to read a single file might not seem like a big performance impact, but remember, we are talking about systems that are running low on physical RAM in which case the operation system will shrink the disk cache. Files that drop out of that cache add a good amount of latency.
Put a favicon.ico in your Documentroot
Normally, you'd just set your favicon up in the themes setting and let Drupal tell the client where it can be found. Problem with that is that there are still lots of legacy browsers out there, trying to get it from the document root. If such a request is made, clean URLs are enabled and the webserver finds the icon to be missing, it'll boot a new instance of Drupal, just to confirm that, indeed, the file is missing and potentially logging that fact to the database. Having a favicon.ico file in your webroot therefore cuts down on memory consumption and latency.
Mind your sidebars
Blocks usually display dynamic content. Dynamic content pretty much always means database queries. Database queries are costly, so it is a good idea to disable all blocks that are not absolutely necessary. Blocks that are required can be made cheaper by enabling the throttle module (Drupal core module). It will cut down on database queries by rebuilding blocks less often. Of course that means, users may get slightly out of date content.
Temporarily disable all administrative/backend modules
When your webserver clogs down, you are unlikely to do administrative duties anyway, so disable any modules that are currently not needed. This will decrease Drupal's memory footprint and allow you to run more server processes without having to swap.
Disable the database logging module
If the worst comes to the worst, you can disable event logging. You won't cut down on your memory footprint this way, but you'll save one database query per page request. This decreases latency at the cost of statistics.
Switch to a minimalistic theme
Normally, you would not want to display your website with a different theme, but in case your default theme uses a lot of images, you can cut down on latency by using a stripped down theme that does not include any images at all.