Building a slide deck, pitch, or presentation? Here are the big takeaways:
- Bot traffic rose 10% in the last year, with gambling, airline, financial, healthcare, and ticketing websites most affected.
- Prevent bots from wreaking havoc on your website by monitoring unusual traffic, securing backdoors, and blocking old user agents and browsers.
Traffic from malicious bots, the report finds, has increased by nearly 10 percent in the past year to 21.8% of all internet traffic. In some industries the percentage of traffic coming from bad bots accounts for half of all visitor traffic.
The report defines malicious bots as those that "scrape data from sites without permission in order to reuse it (e.g., pricing, inventory levels) and gain a competitive edge," as well as "nefarious ones [that] undertake criminal activities, such as fraud and outright theft."
Malicious bots can, among other things, steal credentials, cause denial of service, create fake accounts to flood services, scrape prices and content, cause denial of inventory by holding items in shopping carts, and perform mass credit card fraud.
Contrary to popular belief that bad bots are originating from overseas, nearly half of all malicious bot traffic originates from data centers in the United States. The percentage of traffic coming from data centers continues to grow in large part due to cloud hosting, which is an excellent way to disguise bot traffic and hide the identity of the originator.
Who the bots are attacking
Discovering where malicious bots are acting is as easy as finding an attacker's most lucrative targets. The 10 industries that have to deal with the most malicious bot traffic are:
- Gambling: 53% of all traffic on gambling sites is from malicious bots
- Airlines: 43.9% of all airline website traffic comes from malicious bots.
- Financial: Malicious bots account for 24.66% of all traffic.
- Healthcare: Malicious bots make up 24.37%, and good bots account for 57%. Only the remaining 18% is human traffic.
- Tickets: 22.97% of traffic comes from bots controlled by scalpers and other bad actors.
- Ecommerce: 21.45% of traffic originates from bad bots.
- Travel, including airlines: The entirety of the travel industry as to deal with 19.24% of its traffic coming from malicious bots.
- Adult industry: Adult-related websites see 17.57% of their traffic coming from bad bots.
- Insurance: 12.88% of insurance website traffic is bad bot based.
- Real estate: While 12.44% of real estate traffic is from bad bots, 37.21% comes from good ones.
The report defines good bots as those that work to get a site's services out in front of others, like Google's website indexing bot, or bots for sites that aggregate information and data. Good bots can still be bad, though: They can skew ad click data, inflate impression data, and increase page rankings for sites that don't deserve it.
How to fight bots
Distil gives a few suggestions for combating bad bots—anyone who is responsible for a website should consider implementing at least some of these strategies.
SEE: Network security policy (Tech Pro Research)
- Block outdated user agents/browsers: Many bots disguise themselves as user agents, like Firefox, Chrome, IE, and other browsers, but those disguises often end up being outdated versions as bots age. Block versions of browsers and agents that are more than three years old.
- Keep an eye on backdoors: Bots use exposed APIs, mobile apps, and other access points to get in. Make sure you take the time to protect backdoors and don't assume anything is safe.
- Investigate traffic spikes and sources: Seeing a lot of traffic from one source, or sudden spikes from a single location? These are signs of a bot attack and should be looked into.
- Monitor failed login attempts: A lot of bots hammer login pages to try breaking into accounts—keep an eye out for login anomalies.
- Pay attention to data breach news: If a large data breach occurs, there's a high likelihood that stolen data will be tried against your website. Be aware of what may be coming and keep an eye out for related traffic.
- Consider a bot mitigation tool: There is software available to fight bots. If it's within budgetary constraints, consider using one.
- Special report: Cybersecurity in an IoT and mobile world (free PDF) (TechRepublic)
- Humans, cover your mouths: Lip reading bots in the wild (ZDNet)
- How to tell if your website has been visited by a bot: 5 tips (TechRepublic)
- Twitter has a spam bot problem — and it's getting worse (ZDNet)
- Beware of the bots: How they're created and why they matter (TechRepublic)
Brandon Vigliarolo has nothing to disclose. He does not hold investments in the technology companies he covers.
Brandon writes about apps and software for TechRepublic. He's an award-winning feature writer who previously worked as an IT professional and served as an MP in the US Army.