Bot Traffic Impact

If you are running a server and you undoubtedly will be dealing with endless bot traffic to your sites. As a webmaster, yes, you should be concerned about bot traffic.

What is bot traffic?

“An Internet bot, web robot, robot or simply bot, is a software application that runs automated tasks (scripts) over the Internet. Typically, bots perform tasks that are simple and repetitive much faster than a person could. The most extensive use of bots is for web crawling, in which an automated script fetches, analyzes and files information from web servers. More than half of all web traffic is generated by bots.” – wikipedia

Impact on your server

If your site is dynamic, like a WordPress site powered by MySQL/PHP, every visit to your site, from either a human visitor or a machine robot, will generate requests to the database and consume server CPU resources, even you are running an efficient caching system.

Rapid-fired bot traffic could easily overwhelm your server, slow down, and in worse case, crash your server. It pays to keep an eye on your web logs and web traffic.

Impact on environment

All those unnecessary bot traffic consumes precious resource – electricity. Even the hosting companies go “green”, the wasted electricity could be more usefully deployed somewhere else.

The issue

There are so many bots running wild in the background of World Wide (Wild) Web, it is very difficult to rid of all the bots as you ban some and others will pop out tomorrow. There is a “demand” for “intelligence” by way of sneaking onto websites, unless the demand is stopped or those bots are banned and companies could no longer peddle their services, we will continue to see bots coming.

How to determine a good or bad bot?

Bots that help bring real (human) traffic to your sites are good, all others should be banned.