A solo publisher running an information site with roughly 1.5 million pages finally took a real look at the server logs. Monthly visitors registered in the millions, but ad revenue had been stuck in the same place for months. Server costs, meanwhile, just kept climbing.
Ninety-nine percent of the requests in the logs weren't human. Their behavior gave them away: dozens of page requests per second, the same IP hitting the site tens of thousands of times a day. No clicks, no pauses, no return visits. Reading the page was clearly not the point.
Once the blocking began, traffic dropped by nearly 99%. And then something strange happened: nothing got worse.
What Only Became Visible After the Blocking
At first, the access logs looked bleak after the bots were shut out. Requests that had run into the millions each day fell to the tens of thousands. But inside those tens of thousands, a real pattern finally emerged.
Actual readers requested one page at a time. They moved to the next page one or two seconds later. Some worked their way through several articles on the same topic in a row. None of this had been visible underneath the noise the bots generated.
What surfaced was simple: for a full year, the site had been optimizing content for traffic, while what actual readers wanted was something else entirely. Bots favor page count. People go looking for a single piece that digs deep into one subject. The gap between those two directions only became visible once the bots were gone.
Every Bot Had a Different Agenda
The bots visiting the site weren't all after the same thing.
The largest share, by far, were crawlers scraping data to train AI models. They vacuum up an entire site indiscriminately, regardless of any given page's quality or popularity — every article gets the same treatment. This category alone accounted for more than half of all bot traffic.
Then there were SEO analysis tools — software used by marketers trying to map a competitor's site structure, visiting on a regular schedule. These don't read content either; they harvest titles, metadata, and internal link structure. On top of that were data-aggregation services repackaging site information for their own products.
None of them clicked an ad, signed up for anything, or shared an article. It was traffic disconnected from any revenue model whatsoever.
One more thing stood out. AI training crawlers scrape without regard to quality — a carefully researched piece and a thin, padded page get pulled with the same weight. High traffic from these bots isn't a signal that the site is good. It's only a signal that the site is big.
What the Remaining 1% Looked Like
Once the bots were stripped out, the real readers left behind made up just 1% of total traffic. But that 1% behaved completely differently.
They didn't bounce immediately after arriving from search. Average time on page ran more than ten times higher than the blended, bot-inflated figure. Recalculating the newsletter signup rate against real visitors alone produced a number that was actually respectable — one that had rounded down to nearly zero when bots were still padding the denominator.
The content had been working all along. It was just buried under bot traffic.
The real question was which signal had driven a year's worth of content decisions — bot behavior or human behavior. There was no way to know whether adding more pages, tuning keyword density, and publishing more frequently had been decisions grounded in actual reader behavior, or decisions made in service of a bloated traffic total. The denominator had been contaminated the whole time.
Change the Metric, and What You Write Changes Too
After blocking the bots, the publisher went looking for other numbers to trust.
Email open rate was the first one that stood out — whether subscribers actually opened a given newsletter. Bots can't touch that number. It rose when the content was strong and fell when a subject line was weak. It was a completely different signal from site traffic.
Next came the direct-traffic ratio: visitors typing the URL straight into the address bar or arriving via a bookmark. That's a sign someone remembered the site and came back deliberately — not through search or an algorithm, but on their own. The higher that ratio, the stronger the case that the content was actually memorable.
Unsubscribe rate followed close behind. Since bots don't subscribe, this metric reflects pure human reaction. Whenever unsubscribes ticked up, something real had changed.
Most solo content creators asking "how many readers do I actually have" are working from the wrong denominator. Page views, visitor counts, follower totals — these are all numbers that blend bots and humans together. Build a content strategy on top of that blend, and you end up writing for bots more than for people.
What this publisher walked away with after a year wasn't better blocking technology — it was a reset of what to measure. The moment the benchmark shifted from traffic to email open rate, what to write next changed with it. One deeply read piece started to matter more than another dozen pages.
