Bot protection firm Distil Networks has published its annual report that uncovers trends and statistics on global bot traffic across the Internet.
In 2015, 46 percent of all web traffic originated from bots, with over 18 percent from bad bots, the report said. Those figures are down from the previous year, when bots made up 59 percent of all web traffic in 2014, with roughly 22 percent of that traffic coming from bad bots. Back in 2013, Distil Networks found that 24.22 percent of all web traffic could be traced to bad bots.
For the first time since 2013, humans outnumbered bots in terms of website traffic, with humans accounting for 54.4 percent of all website traffic.
Distil explains that “bad bots” are used by competitors, hackers and fraudsters to conduct malicious activities such as web scraping, brute force attacks, fraud, account hijacking, data theft, unauthorized vulnerability scans, spam, digital ad fraud, and more.
S
“When we dug into the bot activity in 2015, we identified an influx of Advanced Persistent Bots,” said Rami Essaid, co-founder and CEO of Distil Networks. “ABPs can mimic human behavior, load JavaScript and external assets, tamper with cookies, perform browser automation, and spoof IP addresses and user agents.”
“The persistency aspect is that they evade detection with tactics like dynamic IP rotation from huge pools of IP addresses, use Tor networks and peer to peer proxies to obfuscate their origins, and distribute attacks over hundreds of thousands of IP addresses,” Essaid continued. “A whopping 88 percent of 2015 bad bot traffic were APBs. This shows that bot architects have already taken note of traditional bot detection techniques and are finding new sophisticated ways to invade websites and APIs, in an effort to take advantage of critical assets and impact a business’s bottom line.”
Chrome was the browser of choice for bad bot creators with over 26 percent of all user agents utilizing the Google browser. Additionally, 53 percent of bad bots are now able to load external resources like JavaScript—leading to them frequently being falsely attributed as humans in Google analytics and other tools.
Furthermore, the report revealed that 36 percent of bad bots disguise themselves using two or more user agents, while 73 percent of bad bots rotate or distribute their attacks over multiple IP addresses. An impressive 20 percent used more than 100 different IP addresses during their operations, the report said.
While there has been a significant increase in bad bot traffic originating from China, the United States still has biggest bot problem, taking credit for 39 percent of global bot traffic. Six out of the top 20 ISPs with the highest percentage of bad bot traffic originated from China, while the United States and the Netherlands had the most mobile carriers, 5 and 3 respectively, on the top 20 list of bad bot mobile carriers.
The dataset in the report covers 74 billion bot requests that the company observed in 2015, in addition to human traffic. The dataset resides in Distil’s Hadoop Cluster and includes data from hundreds of customers as well as Distil’s global network of 17 data centers, the company said.
“The bad bot landscape continues to evolve rapidly, especially in relation to the sophistication of bot software and the number of bots coming from Chinese service providers,” the report concludes.
Distil Networks credits cheap and free cloud computing resources as a culprit, as a rather unskilled attacker can download open source software and launch malicious bot operations.
“IT security teams must ensure that nefarious actors can’t harvest their data or breach their defenses. And marketing teams seek accurate data on website and conversion metrics,” the company advised.
To learn more about the findings in the report and how your organization can defend against bot threats, register for the upcoming webcast, “Quantifying the Risk and Economic Impact of Bad Bots,” with SecurityWeek and Derek Brink, vice president of research at Aberdeen Group on Tuesday, April 5, 2015 at 1:00 pm ET.