Defending Against DDoS Attacks
A distributed denial of service attack is every business’s worst nightmare. One minute, everything is ticking along as normal. The next, your infrastructure is hit by a tsunami of spurious traffic from across the Internet. Legitimate users find themselves locked out, your ability to do business online grinds to a halt, and there’s not a great deal you can do about it – unless you prepare ahead of time.
Nowadays, it is frighteningly easy for attackers to execute a DDoS attack. Botnets comprised of thousands of compromised PCs can be rented cheaply, and software capable of automating attacks can be acquired readily on the underground market. Attacks peaking at tens of gigabits per second have been recorded, and the size of peak attacks grows each year.1 A modest attack can be bought for less than a thousand dollars.2 It’s also quite possible for your site to become collateral damage in an attack against a third party you know nothing about. Witness Twitter, one of the Internet’s most highly trafficked sites, which found itself knocked offline for hours last August due to a politically motivated attack launched against a single user.3
While some evidence shows that massive brute force DDoS attacks are falling out of favor among financially motivated criminal enterprises, there are few signs of a decline in DDoS more generally.4 DDoS attacks are so hard to stop that it’s not unheard of for some companies to surrender to extortion attempts, quietly handing their attackers tens or hundreds of thousands of dollars in protection money in order to make the problem go away.5
Short of paying out, it’s extremely difficult to completely prevent a determined DDoS attack. But there are four general measures organizations can take, both during system design and live operation, to mitigate the risk of genuine users and customers suffering disruption during an attack. Successful defenses involve using all four techniques:
1. Over-provisioning
Many DDoS attacks are brute force in nature, and over-provisioning is a brute force defense. Your opponent simply needs to throw enough traffic at you to overwhelm your capacity. You can reduce his chances of success and limit the impact on your users by provisioning for far more traffic than you would expect to receive during normal operation. You do not necessarily need to provision for a 40Gbps attack – not all attackers have botnet arsenals that large – but you should aim to prepare for traffic many multiples of what you experience in normal operations.
Some people, when designing their networks, have a tendency to provision for their highest anticipated level of genuine traffic. An e-commerce site, for example, might provide enough capacity for a seasonal sales peak. This will rarely be sufficient to fend off a good-sized DDoS attack. If normal business means 60,000 visits per day, expect a DDoS attack to easily send that much traffic your way in ONE minute. That translates to 86 million “visits” in a single 24-hour attack. A site only provisioned for 60,000 visits will quickly fall to its knees.
A good rule of thumb when building out your hardware infrastructure is to provision for ten times normal peak traffic. Work out the most amount of traffic you’ve ever had, multiply it by ten, and deploy sufficient hardware to cope with at least that level of activity.
Similar rules apply to bandwidth, so you must ensure that your contract is flexible enough to permit traffic coming into your systems to “burst” to many times the normal volume. You don’t want your connectivity provider to shut down all traffic to your site in order to prevent collateral damage to its other customers. Work out the largest amount of bandwidth your site has ever consumed under normal circumstances, then check that your contracts would allow a sustained burst of ten times that amount. Keep in mind that handling that much traffic will take a hefty bite out of your checkbook, too.
2. Remote/redundant monitoring
If up-time is important to you, chances are you already have systems in place to monitor the performance and availability of your site. But in-house monitoring systems can be of limited utility if they’re under a DDoS attack as well. If a system designed to alert you when the network experiences problems sits behind the same bottleneck as the site it is monitoring, the alert probably won’t make it to your phone or in-box in a timely fashion.
When you’re under attack, it helps to know that you are under attack – and quickly. A more reliable alternative is to subscribe to a third-party service that monitors your site around the clock from dozens of other places on the Internet, evaluating its responsiveness from a genuine end-user perspective and providing alerts to your phone when problems are found.
3. Dump the logs
Your Web server logs can’t tell the difference between a genuine visitor and a botnet node. Both visits will usually be recorded in the same way. Even if your server is provisioned correctly and is able to recover from a DDoS attack flood, if its logs stack up, you can often add insult to injury if your server fails because the logs became too large. While the log data could possibly be used for forensic purposes after the attack is over, its value is relatively limited. It’s far more important that servers are able to respond to genuine users during the attack.
If you find log files growing large quite quickly, you’re faced with the choice between keeping the data and losing the server, or losing the data and keeping the server. If your Web server is mission critical and large log files are preventing you from recovering, your choice should be clear: dump the logs.
4. Know the people at your providers
While it is technically possible to locally configure network hardware to drop some malicious packets, ideally you’ll want the unwanted traffic throttled as close to the source as possible. This means that coordination with your upstream providers is a must.
Unfortunately, if your opponent has done his reconnaissance properly, he will launch his attack at the most inconvenient time possible. There’s a good chance that the text message alerting you to an incoming DDoS will arrive at 1am on a Saturday morning, when both you and your regular ISP points of contact are off for the weekend.
The normal support numbers you know to call might go to voice-mail, the night-shift staff may not have the expertise or authority to help, and automated ticketing systems may not be as comprehensively attended as they are during business hours. If you can’t find anyone in a position to help you, you’re then faced with the prospect of two or three days of compromised performance or outright downtime.
In these circumstances it’s essential to have the direct telephone numbers of clued-in people at your ISP’s network operations center. If you know how to contact the right person to help shut down the attack, regardless of the hour, you’ll experience far fewer headaches when a DDoS strikes.
It’s a truism that most security vulnerabilities are people problems. Fortunately, that sometimes also applies to the solutions.
DDoS attacks are here to stay – after all, they are cheap to setup and easy to implement. By appropriately deploying plans in these four areas (provisioning, monitoring, log management and escalation) you should be able to hold your own against all but the most determined and aggressive attackers. Can your organization survive a massive cyber attack?
1 ShadowServer, http://www.shadowserver.org/wiki/pmwiki.php/Stats/BotnetCharts
2 The Day Before Zero, “Want to rent an 80-120k DDoS Botnet?,” 28 Aug. 2009
3 The Washington Post, “Russia-Georgia Conflict Blamed for Twitter, Facebook Outages.” 6 August 2009
4 Dark Reading, “Report: DDoS Attacks Still Growing, But At Slower Rate,” 19 Jan 2010
5 The Register, “Unfashionable DDoS attacks still menace websites,” 23 March 2010
