Cloudflare, Inc. has introduced new classifications, enhanced analytics, and commercial partnerships to help site owners and AI companies adapt to the rise of the agentic internet, where automated AI agents now generate more than half of all web requests.
Matthew Prince, co-founder and CEO of Cloudflare, said: “Now that the majority of traffic on the internet is non-human, we must go further and act faster so that a sustainable ecosystem can emerge. Cloudflare’s new tools and partnerships give website owners increased visibility and commercial opportunities and benefit AI companies that have bots with clear and transparent intent. We hope that our proposed default changes encourage mixed-use crawlers to separate search from agent use and training.”
Key updates
Cloudflare unveils a new default policy that separates AI search, agent use and model training. Beginning September 15, 2026, new Cloudflare customers will automatically allow search crawlers while blocking AI training and agent use on ad-supported pages unless they choose otherwise. Existing free customers who do not update their settings will also adopt the new defaults.
Cloudflare introduces the Attribution Business Insights dashboard to help publishers and AI companies monitor AI bot content consumption and referral traffic.
Expanding into Pay Per Use, Cloudflare enables publishers to earn revenue from content used in AI responses, with partnerships like Ceramic.ai and You.com automating payments and enhancing search efficiency.
