Agent-Ready Website: How We Took beevr.ai to Level 5 (2026)

An agent-ready website is one that AI agents can find, read and use without scraping your HTML. In practice that means robots.txt rules with Content Signals, Link headers, Markdown versions of your pages, and machine-readable descriptions of real tools such as an MCP server, an A2A agent or a JSON API. On 6 October 2026 we took beevr.ai from Level 1 to Level 5 ("Agent-Native"), the top of Cloudflare's Agent Readiness scale, in one working day. The core change was about 1,200 lines of Next.js across 16 files. This case study covers every check, how we built it, the effort, the Next.js gotcha we hit and what we deliberately skipped. Every endpoint is live, so you can verify each claim with curl. Updated October 2026.

What does "agent-ready" mean, and how is it measured?

AI agents now search, compare and act for users. A buyer's assistant might shortlist software studios, read their pricing and draft an enquiry before a human opens a browser. Scraping HTML is slow, costly in tokens and often wrong. An agent-ready site gives them three things instead: permission (what they may do with your content), readable content (Markdown, not 100 KB of markup) and usable tools (structured endpoints with published descriptions).

Cloudflare launched its Agent Readiness score and the free scanner at isitagentready.com in April 2026. The scanner checks 22 standards in five groups (discoverability, content accessibility, bot access control, API/auth/MCP discovery and commerce) and assigns a level:

You can run the same scan from the command line. It returns JSON with the level, every check and what the next level needs:

curl -s -X POST https://isitagentready.com/api/scan \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://beevr.ai"}'

Where did beevr.ai start, and where is it now?

On the morning of 6 October 2026, beevr.ai was at Level 1. robots.txt, the sitemap and AI crawler rules passed. Content Signals, Link headers, Markdown negotiation and every discovery check failed. Our sister product EcoCheck, a greenhouse-gas inventory platform, was the same.

The scan we ran for this article (6 October 2026, 18:27 UTC) shows both sites at Level 5, Agent-Native, with no higher level left:

• beevr.ai: 13 checks pass, from robots.txt and Link headers to DNS-AID, Markdown negotiation, the MCP Server Card, the A2A Agent Card, Agent Skills and WebMCP. Three auth checks fail by design, and so do the five commerce checks.
• ecocheck.ai: the same, plus OAuth discovery and protected-resource metadata, because EcoCheck has real customer accounts. Commerce checks are marked "not applicable".

What matters is what sits behind the score. Every file we publish describes an endpoint that really works and returns the same services, prices, case studies and articles the website shows. A discovery file that points to nothing fails every agent that tries to use it.

How do agents find out what your site allows and offers?

Agents learn what they may do from robots.txt and Content Signals, and learn what you offer from Link headers, an API catalog, an AI catalog and DNS records. On beevr.ai all of these took under a day combined.

robots.txt with Content Signals. robots.txt already says which crawlers may fetch what. Content Signals add what they may do with the content: search (index and link), ai-input (ground an AI answer) and ai-train (training). Ours says yes to all three under every user-agent group, because we want AI engines to cite us. We generate robots.txt with next-sitemap, so this was a short transformRobotsTxt function that adds the Content-Signal line after each User-agent line. Choose your own values on purpose: many publishers set ai-train=no.

Link headers (RFC 8288). Every HTML page now returns a Link header pointing to the API catalog, the OpenAPI description, llms.txt, the AI catalog and the sitemap, so an agent that fetches any page learns where the structured data lives. In Next.js this is one headers() entry in next.config.js, with Vary: Accept because the same URL can also return Markdown. Check it with curl -sI https://beevr.ai/.

API Catalog (RFC 9727) and AI catalog. /.well-known/api-catalog is a linkset+json document listing our three agent interfaces (JSON API, MCP, A2A) with their descriptions. /.well-known/ai-catalog.json is the ARD (Agentic Resource Discovery) manifest: one list of the MCP server card, A2A agent card, API catalog and skills index, each with example queries.

DNS-AID. DNS for AI Discovery publishes agent endpoints as SVCB records under an _agents namespace. We added _index._agents.beevr.ai and _a2a._agents.beevr.ai, both pointing at beevr.ai on port 443. The scanner only accepts these from a DNSSEC-signed zone, so we also turned on DNSSEC at Cloudflare and added a DS record at the registrar.

How do agents read your pages without scraping HTML?

They ask for Markdown. The same URL returns clean Markdown to agents and normal HTML to browsers, and llms.txt gives them a curated map of the site.

Markdown negotiation. When a request sends Accept: text/markdown, beevr.ai returns the page's main content as Markdown. Browsers never send that header, so people still get the normal page. Our homepage is about 111 KB of HTML and 6 KB of Markdown, and the response carries x-markdown-tokens: 1508 so the agent knows the cost up front:

curl -s -H 'Accept: text/markdown' https://beevr.ai/ | head

Cloudflare can do this conversion at the edge on paid plans. We built it into the app instead. Middleware rewrites Markdown requests to an API route, which renders the page internally, extracts the main content and converts it to Markdown with front matter (title, description, canonical URL). The converter is about 120 lines with no dependencies.

llms.txt. llms.txt is a curated Markdown index of the site, with an "Agent access" section listing the MCP, A2A, API and Markdown endpoints. The scanner does not score it, but AI search engines read it.

How do agents actually use your site?

Agents use your site through tools: endpoints with published descriptions that return real data. This is the part that matters for business, and the part most sites skip. beevr.ai exposes the same read-only data through five interfaces:

• MCP server at https://beevr.ai/mcp. Stateless JSON-RPC over Streamable HTTP with 8 read-only tools in English and Vietnamese, including search_beevr, get_pricing, get_case_study and get_article. The server card at /.well-known/mcp/server-card.json lists the endpoint, tools and authentication.required: false. Add the URL as a custom connector in an MCP-capable assistant and price questions are answered from get_pricing, not model memory. Architecture and security are covered in MCP server development.
• A2A agent at https://beevr.ai/a2a, described by /.well-known/agent-card.json. Another agent sends a text message (message/send) and gets back matching services, case studies, articles or pricing. It is deliberately deterministic, with no LLM behind it, so it cannot invent a price.
• JSON API under /agent-api/, described by /openapi.json.
• Agent Skills at /.well-known/agent-skills/index.json: two SKILL.md files with SHA-256 digests, one for researching BeevR and one for giving a user a grounded first estimate from our published packages, then routing them to a real quote.
• WebMCP. When an in-browser agent is present, the site registers four tools through document.modelContext. In other browsers the component does nothing.

Try the A2A agent:

curl -s https://beevr.ai/a2a -H 'content-type: application/json' -d '{
  "jsonrpc":"2.0","id":1,"method":"message/send",
  "params":{"message":{"role":"user","messageId":"m1",
    "parts":[{"kind":"text","text":"How much does an MVP cost?"}]}}}'

It answers with our three fixed-price packages and links, so a user's assistant can quote real numbers with a source.

How did we build it in Next.js, and what went wrong?

beevr.ai runs on Next.js 12 with built-in i18n (English at the root, Vietnamese under /vi). Three modules do the work. lib/agent/data.js holds services, packages and case studies, with articles coming from the CMS. lib/agent/tools.js defines each tool once for the MCP server, A2A agent and JSON API. lib/agent/manifests.js generates every discovery document from the same data, so a new tool appears in every manifest automatically and the interfaces never disagree. A handful of API routes serve it all.

The gotcha. The obvious way to map /mcp or /.well-known/agent-card.json onto API routes is rewrites() in next.config.js. With i18n enabled, every rewrite returned 404. Next.js prefixed the destination with the locale (/en/api/...), and API routes do not exist under a locale. locale: false and moving rules between beforeFiles and afterFiles did not help. The fix was to route agent paths at the top of middleware.js with an absolute URL, which carries no locale:

const AGENT_ROUTES = [
  [/^\/\.well-known\/agent(?:-card)?\.json$/, () => '/api/wk?doc=agent-card'],
  [/^\/mcp\/?$/, () => '/api/mcp'],
  [/^\/a2a\/?$/, () => '/api/a2a'],
  [/^\/agent-api\/(.+)$/, m => `/api/agent/${m[1]}`],
]
// in middleware(): match, then
return NextResponse.rewrite(new URL(target, request.url))

Two smaller traps. The middleware matcher did not match the bare root / under i18n, so Markdown failed on the homepage until we listed '/' explicitly. And the Markdown route fetches the page internally, so it sends a marker header telling middleware not to rewrite that request again, or it loops.

We generate the well-known documents in a route from live data. Static files in public/ work just as well if your offering rarely changes.

What did we deliberately skip, and why?

We skipped OAuth, agent self-registration and commerce protocols, because none of them fits a site with no user accounts and nothing to buy online. Level 5 does not require them. We added Web Bot Auth a day later (see below).

• OAuth discovery and protected-resource metadata. beevr.ai has no user accounts and no private data. Publishing an authorization server to pass a check would describe an auth flow that protects nothing. EcoCheck is the opposite case: customers have workspaces and emissions data, so https://ecocheck.ai/mcp/account is an OAuth 2.1 resource server. An unauthenticated call returns 401 with a WWW-Authenticate header pointing to its RFC 9728 metadata, which names the authorization server and the read-only ecocheck/read scope. Both OAuth checks pass there because there is something real to protect.
• auth.md agent registration. Our auth.md states what is true: no registration, no credentials, anonymous read-only access, and agents must not submit contact forms for a user. The check expects a flow for agents to register themselves. We will not let agents create accounts, so it fails, as intended.
• Commerce protocols (x402, MPP, UCP, ACP, AP2). These let agents pay or check out. We sell engagements that start with a conversation, not a cart. The scanner flagged beevr.ai as possible commerce because PayPal and Adyen appear in a payments case study, a false positive that does not affect the level.
• Web Bot Auth (added 7 October). This lets a bot prove its identity by signing requests (RFC 9421). We publish an Ed25519 public key at /.well-known/http-message-signatures-directory, and the directory response is itself signed over @authority. It took about an hour. It only pays off once you run outbound agents, so we have not filed Cloudflare's verified-bot form yet; the scanner treats it as informational.

What is the full agent-readiness checklist?

Every item we handled on beevr.ai, with our effort. These are our figures on one Next.js site with an existing CMS, not quotes.

Levels 1 to 3 are a morning's work on most sites. Level 4 and above depend on having something real behind the manifests: days if your data already sits behind a clean API, and a separate project for an authenticated server over customer data like EcoCheck's.

Is agent readiness worth it for your business?

If agents act for your buyers, yes. The cheap levels cost hours and make AI answers about you come from your own pages. Tools are worth building when an agent should answer from live data rather than memory: price, availability, eligibility, order status. For EcoCheck that question is "must my facility report greenhouse-gas emissions?", answered by the public check_ghg_inventory_obligation tool. Don't publish discovery files for endpoints you don't have, and don't add OAuth or commerce protocols just to raise a number. Once agents can act, you also need limits on what they may do; our AI agent governance guide covers that.

FAQ

What is an agent-ready website?

A website that AI agents can discover, read and use through published standards rather than scraping: robots.txt rules with Content Signals, Link headers, Markdown versions of pages, and machine-readable descriptions of real tools such as an MCP server, an A2A agent card or an API catalog.

How do I check my site's agent readiness score?

Enter your URL at isitagentready.com, or POST {"url":"https://your-site"} to https://isitagentready.com/api/scan. The response gives your level from 0 to 5, all 22 checks and what the next level needs.

Do I need an MCP server to be agent-ready?

Not for Level 3, which needs only robots.txt, a sitemap, AI bot rules, Content Signals and Markdown negotiation. Level 4 needs at least one integration: an MCP server card, an A2A agent card, Agent Skills or an API catalog. Build the one that answers a real question from your data.

Does Markdown for Agents hurt SEO?

No. Browsers and search crawlers ask for HTML and still get HTML. Only clients that send Accept: text/markdown get Markdown. Send Vary: Accept so caches keep the versions apart, and include the canonical URL in the Markdown.

Why do Next.js rewrites to /api return 404?

With built-in i18n, Next.js 12 adds the locale to rewrite destinations, so the target becomes /en/api/..., which does not exist. Route these paths in middleware with NextResponse.rewrite(new URL('/api/...', request.url)), and add '/' to the matcher explicitly.

BeevR builds the agent layer for products: MCP servers, A2A agents, public APIs and OAuth for agent access, with a fixed price per phase and full code ownership. You can test ours now at https://beevr.ai/mcp. To make your product agent-ready, see our AI agent development work or tell us what agents should be able to do with your product.