AI Bots Scan TLS Certificate Logs: Any Information Used to Train LLMs

Human Security published the State of AI Traffic report, according to which AI traffic on the internet has grown by 187% over the past year, while traffic from AI agents increased by 7851% and from AI scrapers by 597%. AI traffic is growing eight times faster than human traffic. Sites with fresh content, such as stores and news media, are under the greatest pressure, with bot share approaching 90%. The report is based on data from the Human Defense Platform. Cloudflare CEO Matthew Prince predicted that by 2027 AI bots would exceed human traffic, but this has already happened in 2026. Experts describe the situation as the end of the Open Web: content is instantly absorbed by AI bots and used to train large language models (LLMs), leading to a catastrophic decline in human traffic (e.g., visits to technical media have dropped by more than 50%). Publishers are moving to paywalls, even blocking the Internet Archive. AI bots even scan TLS certificate logs (Certificate Transparency) to discover new domains for crawling: as soon as a user registers a certificate, almost instantly the first visitor arrives, such as OpenAI's OAI-SearchBot. Additionally, information archives of bankrupt companies are being bought up for LLM training, and existing firms are creating "digital twins" of employees based on their correspondence and files. In factories, employees work with video cameras on their heads to record nonverbal behavior. Any user-generated content (UGC) becomes valuable, so live employees are now in high demand.