Obenan Briefing · Signal
On July 1, 2026, Cloudflare split AI traffic into Search, Agent, and Training.
A major edge platform stopped treating AI access as one on-or-off switch. AI traffic is now three separate operating decisions: stay discoverable, govern real-time agents, and limit training crawls, with a crawl-to-referral ratio to test whether that traffic sends anything back.
Published July 8, 2026
The one line
AI visibility governance is becoming a question of who may read, for what purpose, and what they send back, not a single block-AI-bots toggle.
- Published
- July 8, 2026
- Format
- Signal briefing
- Sources
- 5 primary, public
- Coverage
- Cloudflare release, July 1, 2026
Public sources only. Observed facts and our interpretation are kept separate.
The 60-second read
Three questions, three answers.
What Cloudflare changed at the edge, why governing AI traffic by behavior matters, and what it still does not decide.
AI traffic became three separate decisions.
On July 1, 2026, Cloudflare replaced its single Block AI bots preset with behavior-level controls that separate AI crawlers into Search, Agent, and Training. Every customer, including the Free tier, can now allow a behavior, block it on all pages, or block it only on pages that display ads.
A site can stay discoverable while being stricter with agents and training.
Because the three behaviors are governed separately, a business no longer has to make one all-or-nothing choice. It can keep Search access that supports discoverability while treating real-time agent traffic and extractive training crawls differently. Cloudflare also set a dated default change: starting September 15, 2026, new domains will block Agent and Training on pages that display ads while leaving Search allowed.
Whether the fact behind the page is true, current, and complete.
Controlling who may crawl does not make the answer an AI gives about a business correct. A crawl-to-referral ratio can show whether a bot returns any visitors, but it does not prove recommendation quality, and it does not fix the facts a business publishes.
What to do with this
Five moves for the people governing AI access.
Split by what a site owner controls and what the platform controls.
- 01
Treat AI access as three decisions, not one (operator-controlled)
Decide separately whether you allow Search discoverability, Agent activity, and Training crawls. A single block-everything setting is now the blunt option, not the only one.
- 02
Keep Search open if discoverability matters (operator-controlled)
For most local and multi-location businesses, being reachable by search-oriented crawlers supports being found. You can be stricter with Agent or Training without giving up discoverability.
- 03
Read crawl-to-referral as pressure, not proof (shared)
The ratio tests whether a bot operator sends visitors back for the content it takes. Use it to question extractive crawlers, not to claim conversion, recommendation, or revenue.
- 04
Edge controls are platform-owned, not merchant truth (platform-controlled)
Cloudflare governs crawler access and classification at the edge. It does not author the hours, address, services, and policy an AI repeats about a business. That truth stays yours.
- 05
Separate access from recommendation when you report (shared)
Keep two columns. Did crawlers reach the site, and separately, is the fact the AI repeats correct. A green access setting is not a correct answer.
The evidence, dated
One release, four moving parts.
One launch wave on July 1, 2026, four public artifacts, shown in the order Cloudflare published them.
Behavior-level AI traffic controls launch for all customers
Cloudflare announced that all customers, including the Free tier, can now manage AI traffic by Search, Agent, and Training, because not all AI traffic serves the same purpose.
Each behavior gets its own allow or block option
The changelog says each behavior can be left unblocked, blocked on all pages, or blocked only on pages that display ads. Starting September 15, 2026, new domains default to blocking Agent and Training on ad pages while Search stays allowed.
Attribution Business Insights adds a value signal
For Bot Management Enterprise customers, a new dashboard shows bot traffic to content pages and site-wide and per-operator crawl-to-referral ratios over 24 hours, 7 days, or 30 days. Cloudflare describes it as visibility for a wider set of stakeholders, not a new control plane.
BotBase exposes per-bot behavior and detection IDs
BotBase shows how Cloudflare classifies each bot by behavior, including Search, Agent, and Training, exposes a detection ID for each, and publishes every tracked bot in Cloudflare Radar's public bots and agents directory.
In one release, AI-traffic policy went from a single switch to a set of behavior-specific decisions with a business-readable value signal attached.
The distinction that matters
A traffic control is not a true answer.
Governing which crawlers may reach a site decides whether an AI can read it. It does not decide whether what the AI then says about the business is accurate, current, and complete.
For a static page the gap is small. For a local business whose hours, availability, services, and policy change, the fact behind the page is where most AI answers go wrong, and no crawl setting fixes that.
Accurate identity
Name, address, and category that match reality across the surfaces an agent reads.
Current hours and availability
Open hours, special hours, and real availability, not a list that is quietly out of date.
Complete policy
Booking, deposit, cancellation, and service terms an agent can act on without guessing.
Services and attributes
What the business actually offers, described the way a customer would ask for it.
Freshness
Facts that are current at the moment of the answer, not last quarter's truth.
Evidence
Reviews and signals that corroborate the claim, so the answer holds up.
Where it breaks today
When the access policy is right and the answer is wrong.
Each of these can happen on a site whose AI traffic controls are set exactly as intended.
A site allows Search and blocks Training, and an AI still repeats hours that were changed three weeks ago.
An operator sees a healthy crawl-to-referral ratio for one bot and assumes it is recommending the business, when the ratio only counts referred visits, not recommendations.
A business blocks Agent traffic to protect content, then finds that a real customer's assistant cannot complete a task it could have handled.
A crawler is allowed and cleanly classified, and an answer engine still cites an address the business corrected months ago.
In each case the access policy was set correctly. The failure was in the truth behind the page, or in reading a traffic metric as a recommendation.
Where the work sits
Three lanes of AI traffic, governed separately.
Cloudflare's release splits AI traffic into three behaviors that can now be allowed or blocked on their own terms, instead of one Block AI bots switch.
The point is not the labels. It is that discoverability, real-time agent activity, and model training are now separate operating decisions instead of one undifferentiated bucket.
Stay discoverable
Crawlers that support being found and indexed. Most businesses keep these allowed so they remain reachable.
Decision: keep discoverability
Govern real-time activity
Autonomous agents acting on a person's behalf in real time. Allow, limit, or block depending on whether agent activity helps or extracts.
Decision: govern agent access
Limit extractive crawls
Crawls that gather content to train models. Often the strictest lane, especially on pages that display ads.
Decision: limit training
AI traffic as three governance decisions, not one switch.
Obenan's view
An edge can decide who reads your site. Only the merchant can make the answer true.
The Cloudflare release is a real step. Treating AI traffic as three decisions is more useful than a single block-AI-bots switch, and a crawl-to-referral ratio gives operators a concrete way to question extractive crawlers.
Obenan works on the layer a traffic control cannot reach: whether the hours, address, services, availability, and policy an AI repeats about a business are accurate, current, and complete. Access is necessary. A correct answer is the goal.
How to read this
Observed, inferred, and watched. Kept separate on purpose.
This is a Signal briefing. We report what the public record shows, then our interpretation, then what we are still watching.
Observed
Cloudflare's July 1, 2026 launch, changelog, and docs describe the Search, Agent, and Training controls, the three per-behavior options, the September 15, 2026 default change for new domains, Attribution Business Insights for Bot Management Enterprise, and BotBase classifications published to Radar. Re-checked July 8, 2026.
Inferred
That AI traffic governance is becoming three separate operating decisions, and that crawl-to-referral is a useful business-pressure metric, is Obenan's reading of the public record, not a Cloudflare claim.
What we are watching
Whether behavior-segmented controls and crawl-to-referral reporting become standard operator practice, and whether any of it begins to touch the accuracy of the facts AI systems repeat about a business.
What this briefing does not claim
The boundaries, stated plainly.
Precision protects the reader and the companies named. These are the limits of the claim.
- 01
This briefing does not claim Cloudflare's controls improve rankings or recommendations in ChatGPT, Gemini, Perplexity, Google AI Mode, or any AI system.
- 02
Attribution Business Insights is available only to Bot Management Enterprise customers, not to all Cloudflare customers.
- 03
Obenan is not affiliated with, partnered with, integrated with, or endorsed by Cloudflare, OpenAI, Microsoft, Google, or any named source. They are public source subjects only.
- 04
The crawl-to-referral ratio measures crawler volume against referred visitors. It is not conversion lift, recommendation quality, citation quality, revenue, or ROI.
- 05
Cloudflare's classifications do not perfectly capture every AI request or every operator intent, and are not a merchant-truth or recommendation signal.
- 06
Obenan is the guide, not the hero: it does not operate edge infrastructure, crawl control, or bot classification, and this Signal reports a Cloudflare change, not an Obenan product launch.
Keep reading
The score, the fix, and the thesis.
If you govern AI access, who fixes the answer?
Cloudflare can decide which crawlers reach your site. Obenan works on the merchant-truth layer a traffic control cannot reach: the facts an AI repeats about your business.
Public research. No partner status is claimed or implied.
Sources
All primary and public. Each can be inspected directly. Five official Cloudflare pages, re-verified on July 8, 2026.
Primary sources
- 1.Cloudflare: Your site, your rules, new AI traffic options for all customersblog.cloudflare.com · July 1, 2026 · checked July 8, 2026
Primary. Launch of the Search, Agent, and Training controls, all-customer availability including the Free tier, and the September 15, 2026 default change for new domains.
- 2.Cloudflare: New options to manage AI traffic (changelog)developers.cloudflare.com · July 1, 2026 · checked July 8, 2026
Primary. Per-behavior allow or block options and the dated default-policy change for new domains.
- 3.Cloudflare: Unmasking the crawls with Attribution Business Insightsblog.cloudflare.com · July 1, 2026 · checked July 8, 2026
Primary. Dashboard purpose, crawl-to-referral ratios, top-bot views, and business-facing reporting scope.
- 4.Cloudflare: Attribution Business Insights (docs)developers.cloudflare.com · July 1, 2026 · checked July 8, 2026
Primary. Bot Management Enterprise availability, the crawl-to-referral ratio definition, and the visibility-not-a-control-plane boundary.
- 5.Cloudflare: More visibility into bot traffic with BotBase and Attribution Business Insights (changelog)developers.cloudflare.com · July 1, 2026 · checked July 8, 2026
Primary. BotBase behavior classifications, per-bot detection IDs, and publication to Cloudflare Radar's public bots directory.
Obenan Briefings. AI Visibility and Discoverability lane. This is a Signal briefing: it reports a public development and Obenan's interpretation, separated on purpose.