99% of My Website Traffic Is Bots

In his blog post, the author, who built PatronView, a database of US philanthropists, details his year-long fight against bots. His server responded to 2.5 million requests and delivered 1.28 million pages in one week, but only 5,977 were human views, meaning less than 0.5% of traffic is human. He initially blocked SEO crawlers like SemrushBot via robots.txt and Cloudflare rules, then in April 2026 a huge wave of Chinese bots hit, with 3.6 million requests in a day, leading him to block China, Vietnam, and Singapore. He also notes that Anthropic's Claude-SearchBot scraped 35,000 pages per visitor it sent, so he blocked it. Amazon's Amzn-SearchBot, used for Rufus and Alexa, scraped 117,000 pages a day without sending traffic, so he blocked it too. He replaced JavaScript Detections with a custom challenge for data centers and old browser versions to combat residential botnets, and he shares his specific Cloudflare WAF rules, which include blocking certain countries and known SEO bots, and challenging requests from cloud providers and outdated browsers.