The majority of website page-views are bots. Is that a problem?
These days the vast majority of website page views on my clients' sites are from bots and crawlers. But is that actually a problem?
21 Sep 2026
The Good, the Bots and the Crawlers
Most of the traffic your website gets these days isn't human. Search engines, SEO tools, AI training crawlers, link previewers, and plain old scrapers all show up in your access logs, and most of them behave themselves. Below you'll find articles on telling the two apart: honest crawlers that respect robots.txt and pace themselves, versus disguised bots and outright botnets that hammer a site, steal content, or run card-testing fraud through your checkout.
Some of this is investigative (working out from raw log files what's actually requesting your pages), some of it is defensive (firewall rules, rate-limiting, blocking the traffic that costs you bandwidth without ever converting), and some of it is just useful context for anyone wondering why their analytics don't add up. The goal isn't to block every bot. It's knowing which ones are worth the fight.
