Cloudflare to Block AI Training and Agent Crawlers by Default on Ad-Supported Sites from September 15

Summary

Cloudflare announced new controls that force AI companies to declare the purpose of their web crawlers — Search, Agent, or Training — and set a September 15, 2026 deadline after which new default policies take effect. On pages that display ads, Training and Agent crawlers will be blocked by default while Search crawlers remain allowed. Crawlers that blend search, agent, and training behavior (mixed-use) will be blocked entirely by default unless the site owner opts in. The changes apply to new c