Connect with us

Hi, what are you looking for?



The Big Business of Bad Bots

Bad bots are big news largely because of the FBI investigation into Russia’s involvement in the 2016 presidential election. But bad bots are a bigger problem than automated tweeting: 42.2% of all website traffic comes from bots; and 21.8% of it is down to bad bots.

Bad bots are big news largely because of the FBI investigation into Russia’s involvement in the 2016 presidential election. But bad bots are a bigger problem than automated tweeting: 42.2% of all website traffic comes from bots; and 21.8% of it is down to bad bots.

Distil Networks’ 2018 Bad Bot Report, based on an analysis of hundreds of billions of bad bot requests, shows that bad bot traffic increased by 9.5% in 2017. Bad bots differ from good bots, whose traffic also increased by 8.8% to 20.4%. It means that only — on average — 57.8% of visiting traffic comes from a genuine human being interested in the website content.

Good bots are those that all websites require. They include the search engine page indexing bots from Google and Bing, and they bring humans to the site. Bad bots, however, are secretive and nefarious. They come from outright criminals and commercial competitors; and their purpose is to detract and/or steal from the website.

Distil highlights eight different bad bot functions: price scraping, content scraping, account takeover, account creation, credit card fraud, denial of service, gift card balance checking, and denial of inventory. They fall into three primary categories: competitive, organized criminal, and nuisance. 

Price scraping and content scraping are generally competitor attacks. Price scraping allows competitors to maintain price levels slightly lower to score more highly in search engine rankings. Content scraping is simply the theft of proprietary content to augment another site’s own content.

Account takeover bots are automated attempts at illegal log-ins. They can deliver brute-force attacks cycling through the most popular passwords to see if one of them works, or they can use the process known as credential stuffing

Distil reports a 300% increase in credential stuffing bad bots in the weeks following a new major credential theft. This involves the automatic application of stolen passwords on different websites. “Here,” notes the report, “bot operators make two assumptions. The first is that people reuse their credentials on many websites. The second is that newly stolen credentials are more likely to still be active. This is why businesses should anticipate bad bots running those credentials against their website after every breach.”

Advertisement. Scroll to continue reading.

Account creation bad bots simply generate vast numbers of new accounts — for example, on Twitter — to spam out messages or amplify propaganda.

Credit card fraud bots test out credit card numbers, trying to identify missing information — such as the expiry date and the CVV.

The denial of service bad bot can be either competitive or nuisance. It can be used to reduce the performance of a competitor, or to disrupt the service of a small website either because of a grudge, or simply because it is possible. It can be effected either from a small number of attacking IP addresses, or from a larger number of rotating addresses. Automated defenses often fail because the number of accesses from each IP address is below the warning threshold before it moves to other addresses, while manual whack-a-mole IP blocking simply cannot keep up.

Gift card balance checking bots are used to steal money from gift card accounts that contain a balance.

‘Denial of inventory’ is a relatively new competitor attack prompted by the growth of ecommerce. In this attack, bots place stock items in online shopping baskets, taking them out stock lists. If the item is no longer available, then visiting human buyers will go elsewhere to make the purchase.

Bad bots are a difficult problem. Many website owners are not aware of them, while common defenses have little effect. Geo-blocking, for example, is only somewhat effective. Many sites block all Russian traffic. While this will inevitably include some bad bot traffic, it may also exclude some genuine human traffic. Russia is, however, the most blocked country.

In reality, the greatest source of bad bot traffic is the U.S. (although the operators may be elsewhere). According to Distil, 45.2% of all bad bot traffic originates in the United States (China is second, but way down with just 10.5%). This is because nobody, anywhere in the world, is likely to block all traffic coming from the U.S.

“This year bots took over public conversation, as the FBI continues its investigation into Russia’s involvement in the 2016 U.S. presidential election and new legislation made way for stricter regulations,” said Tiffany Olson Jones, CEO of Distil Networks. “Yet, as awareness grows, bot traffic and sophistication continue to escalate at an alarming rate. Despite bad bot awareness being at an all-time high, this year’s Bad Bot Report illustrates that no industry is immune to automated threats and constant vigilance is required in order to thwart attacks of this kind.”

Related: Advanced Persistent “Bad Bots” are Rampant 

Related: Many Web Attacks Come from United States: Sucuri 

Written By

Kevin Townsend is a Senior Contributor at SecurityWeek. He has been writing about high tech issues since before the birth of Microsoft. For the last 15 years he has specialized in information security; and has had many thousands of articles published in dozens of different magazines – from The Times and the Financial Times to current and long-gone computer magazines.

Click to comment

Daily Briefing Newsletter

Subscribe to the SecurityWeek Email Briefing to stay informed on the latest threats, trends, and technology, along with insightful columns from industry experts.

Join security experts as they discuss ZTNA’s untapped potential to both reduce cyber risk and empower the business.


Join Microsoft and Finite State for a webinar that will introduce a new strategy for securing the software supply chain.


Expert Insights

Related Content


The changing nature of what we still generally call ransomware will continue through 2023, driven by three primary conditions.


As it evolves, web3 will contain and increase all the security issues of web2 – and perhaps add a few more.


A recently disclosed vBulletin vulnerability, which had a zero-day status for roughly two days last week, was exploited in a hacker attack targeting the...


Luxury retailer Neiman Marcus Group informed some customers last week that their online accounts had been breached by hackers.


Zendesk is informing customers about a data breach that started with an SMS phishing campaign targeting the company’s employees.


Satellite TV giant Dish Network confirmed that a recent outage was the result of a cyberattack and admitted that data was stolen.

Artificial Intelligence

The release of OpenAI’s ChatGPT in late 2022 has demonstrated the potential of AI for both good and bad.


A new study by McAfee and the Center for Strategic and International Studies (CSIS) named a staggering figure as the true annual cost of...