Cloudflare AI Search: Give Your Agents a Search Engine for Your Data

What Changed: AI Search Gets Easier

On August 6, 2026, Cloudflare announced a set of developer experience improvements to its AI Search product. The goal is to give your agents their own search engine, where they can easily find data to provide better answers for themselves and their humans. Previously, you had to stitch together components of the Cloudflare primitives (Workers AI, AI Gateway, Vectorize, R2, Browser Run). Now, AI Search can do this automatically, and better.

The update introduces several key features:

• Index a collection of data for your agent: Make structured and unstructured data easily accessible for your agent to build with, from individual files to websites you own. Today, it must be a zone on your Cloudflare account, but more ways to verify ownership are coming soon.
• Skip the sitemap for your websites: Previously, AI Search required that websites have a sitemap to use the website integration. Now you can select the “Discover” parsing option to add a website without a sitemap as a source.
• Get a single public endpoint for searching across a namespace: When you enable public URLs on your namespace, you can get a /search and /mcp endpoint that can search through multiple instances or websites at once without authentication, so you can share easily with your customers.
• Put your own custom domain over public endpoints: You can now add your own domains over your public URLs, so you can brand your /search and /mcp endpoints (e.g., search.example.com/mcp). You can also add Cloudflare Access to create private search instances.
• Add semantic search to your sites built on EmDash with AI Search plugin: If your site runs on EmDash, Cloudflare’s open-source CMS, the AI Search plugin adds semantic search over your content.
• Preview the new pricing model: Pricing is designed to be predictable and scale with you, with embedding and reranking free when you use select models from the Workers AI catalog.

AI Search is powered by Browser Run /crawl in the background, but it goes a step further to identify itself with its own bot identity: Cloudflare-AI-Search. Just like Browser Run, it follows robots.txt, identifies itself with an immutable, public user agent, and will respect whatever bot controls a site has in place.

How It Works: From Index to Public Endpoint

AI Search handles the heavy lifting of crawling, ingestion, embedding, and retrieval. You point it at a site or set of sites, and it takes care of the rest. Creating an instance is a single command, and for a site without a sitemap you add --parse-type discover to find pages by following links (powered by /crawl from Browser Run):

npx wrangler ai-search create cloudflare-community \
  --namespace dev-stack \
  --source https://community.cloudflare.com \
  --type web-crawler \
  --parse-type discover

Once you have instances, you can combine them into a single search across multiple surfaces. There are two ways to do it:

Option A: in a Worker (what Cloudflare did for Cloudflare Dev Stack MCP)

You bind the namespace to a Worker to create a remote MCP server and make one multi-instance call across all instances. This path is useful when you’re folding search into an existing app or MCP server. The binding, in wrangler.jsonc:

{
  "ai_search_namespaces": [
    { "binding": "AI_SEARCH", "namespace": "cloudflare-stack" }
  ]
}

Then a single tool makes one call that fans out across the instances you name:

// One tool, one call that searches every surface in the namespace at once.
context.registerTool(
  'search_dev_stack',
  {
    description: 'Search current docs across the Cloudflare stack.',
    inputSchema: z.object({ query: z.string() }),
  },
  async ({ query }) => {
    const res = await context.env.AI_SEARCH.search({
      query,
      ai_search_options: {
        instance_ids: ['developers-cloudflare-com', 'astro', /* ...every surface */],
        retrieval: { max_num_results: 10 },
        reranking: { enabled: true },
      },
    })
    // res.chunks come back cited and tagged with the instance they came from.
    return { content: [{ type: 'text', text: format(res.chunks) }] }
  }
)

Option B: flip on public endpoints (no code)

If you’d rather not write a Worker at all, enable public URLs on the namespace. You immediately get /search and /mcp endpoints that query every instance, with no auth and nothing to deploy. Reach for the Worker when you’re folding search into an existing app or MCP server, or reach for the public endpoint when you just want a shareable search endpoint in one click.

Practical Use Case: Cloudflare Dev Stack MCP

One of the ways Cloudflare uses AI Search is in the new Cloudflare Dev Stack MCP, which you can try today in the AI Playground. It gives coding agents current, cited docs from across the Cloudflare developer ecosystem, so they build on the latest features and fixes instead of stale training data.

Here’s how they built it:

1. Index each surface: They created one AI Search instance per Cloudflare-owned surface: Docs, Blog, API Docs, Community, Astro, Vite, Vitest, Hono, Replicate, OpenNext. They span different domains, but because Cloudflare owns the website data, AI Search is able to treat them as a single set and ingest them all the same way.
2. Combine the instances into one search: They used Option A (Worker) to create a remote MCP server and made one multi-instance call across all 10 instances. This ships as a tool alongside the Cloudflare tools agents already connect to.
3. Brand it and lock it down: Public endpoints come with a default public URL, but you can put your own custom domain over them to brand the endpoint (e.g., search.example.com/mcp). If the search should be private, add Cloudflare Access in front of the domain. The endpoint now requires a login, so only authorized people (or agents) can query it.

To use the Dev Stack MCP with your agent of choice, drop the MCP URL into your MCP configuration:

{
  "mcpServers": {
    "dev-stack": { "url": "https://stack.mcp.cloudflare.com/mcp" }
  }
}

This replaces the usual fallback (web search then fetching full pages), which is slow, token-heavy, and often lands on the wrong or stale source.

Cloudflare also uses AI Search to power search on its own Blog, Developer Docs, and Cloudflare.com. All of it uses hybrid search, semantic and keyword together in one query, so it handles both open-ended “what does this do” questions and exact lookups of names or keywords. The Blog was recently rebuilt on EmDash, and the EmDash AI Search integration powers that search now. You can add it to your own EmDash site and get the same search over your content out of the box.

Pricing Preview: Predictable Costs with Free Embedding and Reranking

AI Search is currently free while in beta, and billing is not yet enabled; Cloudflare will email you with plenty of notice before it starts. As they move toward general availability, here’s a preview of pricing across ingestion, storage, and queries, plus embedding and reranking (preview prices are subject to change before billing begins):

	Preview usage price	Free monthly allotment (all Workers plans)
Ingestion
Base Ingestion	$0.75 / 1M tokens	5M tokens †
Image processing (add-on)	+$0.50 / 1M tokens	5M tokens †
Storage
Stored data	$2.00 / GB-month	10 GB
Query
Semantic (hybrid and vector search)	$0.75 / 1k queries	2,000 queries ‡
Full-text	$0.10 / 1k queries	2,000 queries ‡
Embedding and Reranking
Ingestion and query	Free with select Workers AI models; third-party billed separately	N/A

† A single pool of 5M ingestion tokens per month, covering any file type currently supported (e.g., text, images). ‡ A single pool of 2,000 queries per month, shared across both query types.

Embedding turns your text into the vectors that search matches on, and reranking reorders results so the most relevant come first. Both run free with AI Search defaults or when using select models from the Workers AI catalog, so the models behind indexing and every search are not a cost you have to worry about. Answer generation and query rewriting are optional steps that run on a model you choose, billed as Workers AI usage, or you can use AI Gateway credits with any model/provider.

Here’s a sample monthly bill on the Workers Paid plan for creating a new AI Search instance for a 20,000-document data source (about 20M tokens of text) plus 1,000 images (assume about 1,000 tokens each), with 30,000 semantic queries a month using the default AI Search embedding and reranking model. Ingestion is chunked with roughly 10% overlap, which shows up as the × 1.1 below:

Line item	Usage	Price	This month
Base ingestion	(20M tokens of text + 1M tokens of images) × 1.1 - 5M free = 18.1M tokens	$0.75 / 1M tokens	$13.58
Image add-on	1M tokens of images × 1.1 = 1.1M tokens	$0.50 / 1M tokens	$0.55
Storage	~1.2 GB (within 10 GB free)	$2 / GB-mo	$0
Queries (semantic)	30,000 - 2,000 free = 28K	$0.75 / 1k	$21.00
Embedding (Workers AI)	Usage included with selected Workers AI model	$0	$0
Reranking (Workers AI)	Usage included with selected Workers AI model	$0	$0
Total			~$35

Images count toward base ingestion and also incur the image add-on cost. Storage assumes about 10 KB per document and 1 MB per image. Indexing is largely a one-time cost, so later months are mostly queries, closer to $21.

Limitations and Trade-offs

While AI Search is powerful, there are a few limitations to keep in mind:

• Website ownership: Currently, websites must be a zone on your Cloudflare account to be indexed. More ways to verify ownership are coming soon, but for now this restricts non-Cloudflare users.
• Pricing subject to change: The preview pricing is not final and may change before billing begins. Cloudflare will notify you before billing starts.
• Bot policy compliance: AI Search identifies itself with Cloudflare-AI-Search and respects robots.txt and bot controls, which is good for content owners but means you must ensure your site allows crawling.
• Beta status: AI Search is still in beta, so features and pricing may evolve.

Takeaway: A Managed Search Engine for Your Agents

AI Search positions itself as a managed service that gives your agents a search engine for your own data, without the complexity of stitching together primitives. For product builders, the key benefits are:

• Lower integration cost: No need to handle crawling, embedding, vector storage, and reranking yourself. One command creates an index, suitable for prototyping or production.
• Native MCP support: The /mcp endpoint lets agents query via the standard protocol, which is friendly to the current coding agent ecosystem.
• Predictable pricing: Embedding and reranking are free with default models, making budget planning simpler, especially for variable usage.

To get started, point AI Search at your site, turn on hybrid search for both semantic and keyword matching, and you have a search engine for your own data, ready for your agents. Spin one up with one command:

npx wrangler ai-search create my-search \
  --namespace my-namespace \
  --source https://my-website.com \
  --type web-crawler \
  --hybrid-search

From there, query it, wire it into an agent over /mcp, or put a custom domain on a public /search endpoint to share it with your users. Check out the AI Search docs for more information.

Sources

• Original source

AI-assisted summary compiled from the sources above, reviewed by a human before publishing.