← Blog
Field note

Fixed-Price AI Agent Development: Scope, Evals, Milestones

Thien Nguyen · Nov 11, 2026

Yes, you can build an AI agent on a fixed price, if you fix the right things: the scope (tasks, tools, integrations), a labelled evaluation set, and the pass thresholds the agent must hit on it. You don't fix individual model outputs. Price discovery separately, then build in phases that each end with a measurable eval gate.

Founders and CTOs often hear that AI work "can't be fixed price because LLMs are unpredictable." It is half true. Each output is unpredictable. Performance measured over a fixed test set is not, and you can contract on that. This guide covers what a fixed-price AI agent project should include and exclude, how to write the scope document, how to set acceptance criteria and handle change requests, and how a vendor can keep a fixed price honest when the model itself is uncertain.

Can AI agent development be done on a fixed price?

Yes, for a well-defined agent: a known set of tasks, a known set of tools and systems it may touch, and an agreed way to measure success. Fixed price stops working when the goal is still open research ("see what AI can do with our data") or when the integrations aren't known yet. Do those as a short discovery phase first.

LLMs are non-deterministic, and OpenAI's own evaluation guide says so directly: "Generative AI is variable." That is why OpenAI's evaluation best practices recommend tests built from real tasks, automated scoring checked against human judgement, and continuous re-running as the system changes. A fixed-price contract uses the same tool. The deliverable is not "the agent always answers correctly." It is "the agent reaches these scores on this test set, with these guardrails in place."

What is included in a fixed-price AI agent project?

A good fixed-price scope includes the agent's workflows, its tool integrations, the guardrails and approval steps, the evaluation harness and test set, observability and cost controls, deployment to your cloud, and documentation. Model usage fees, new integrations added mid-project and open-ended prompt tuning after acceptance are usually not included.

Usually includedUsually not included (or priced separately)
Named workflows the agent performs, with inputs and outputsNew workflows or tasks added after sign-off
Integrations with named systems through official APIsIntegrations with systems that have no API, or that the client can't grant access to
Guardrails: allowed actions, approval steps, spending or value limitsChanges to the client's own approval policies mid-build
Evaluation harness, labelled test set, scoring and a pass reportLabelling large new datasets beyond the agreed set
Observability: traces of every model and tool call, audit log, cost dashboardThird-party monitoring licences
Deployment to the client's cloud account, infrastructure as codeCloud, model API and SaaS running fees (paid by the client directly)
Runbooks, handover and a warranty period for defectsOngoing support and new features after warranty (retainer)

Two items cause most disputes. The first is model usage cost. It should be billed to your own account, so you see it directly and the vendor has no reason to pad it. The second is "make it smarter" requests after acceptance. Those are new work, priced as a change request or a retainer.

How do you write a scope document for an AI agent?

Write it as a list of jobs, not features. For each job, state the trigger, the inputs, the tools the agent may call, the actions it may take on its own, the actions that need a human, the output, and how success is scored. Then list the systems, data and access the client provides, and by which date.

Scope document checklist

  1. Jobs. For example: "Match incoming supplier invoices to purchase orders; flag mismatches over 2% for review."
  2. Tools and permissions. Each API the agent can call, read or write, with the scopes it needs. Use official APIs and the narrowest scopes, never a shared admin login.
  3. Autonomy levels. What the agent does alone, what it drafts for approval, and what it must never do. Anthropic's guidance on building effective agents warns that autonomy brings "higher costs, and the potential for compounding errors". That is why approval steps belong in the scope, not in a later phase.
  4. Data. Which records the agent sees, how much of each, retention, and whether regulated data (PHI, card data) is involved.
  5. Evaluation. Size and source of the test set, who labels it, the metrics and the thresholds.
  6. Non-functional targets. Latency, cost per task, availability, languages.
  7. Client dependencies. Sandbox accounts, API keys, sample data and a named reviewer, each with a date.
  8. Out of scope. Written explicitly. This is the line that keeps the price fixed.

Start with the simplest design that can pass the evals. Anthropic's advice is the same: begin with the simplest solution and add complexity only when it is clearly needed. Often a deterministic workflow with one or two model calls beats a free-roaming agent on cost, reliability and price.

How do you set acceptance criteria when LLM output is non-deterministic?

Accept the agent against a frozen, labelled test set with thresholds per metric, run several times to account for variation, plus a zero-tolerance list of unsafe actions. If the agent meets every threshold on the agreed set, the milestone is accepted, even if a single conversation later goes wrong.

MetricExample threshold (set per project)How it's measured
Task success≥ 90% of test cases fully correctAutomated checks plus human review of a sample
Critical errors0 on the red-team set (e.g., an action outside the allowed list)Adversarial cases written in discovery
Escalation qualityUncertain cases routed to a human, not guessedLabelled "should escalate" cases
ConsistencyPass rate holds within an agreed margin across 3 runsRepeated runs of the same set
Latencyp95 under an agreed number of secondsMeasured in the staging environment
Cost per taskUnder an agreed amount at the pinned model's list priceToken and tool-call logs

The numbers above are examples, not industry standards. Set yours in discovery, based on how good the current manual process is. Freeze the test set when the milestone starts. If either side wants to add cases, they go into the next milestone's set, so the target doesn't move mid-build. For regulated work, see LLM testing for regulated industries. For how to monitor the agent after launch, see AI agent evaluation and observability.

Table of example acceptance criteria for an AI agent: task success at least 90 percent fully correct by automated checks plus a human sample; zero critical errors on the red-team set; uncertain cases escalated to a human; pass rate holding across 3 runs; p95 latency under an agreed number of seconds; cost per task under an agreed amount from token and tool-call logs.
Contract on scores over a frozen test set, not single outputs

How should change requests work on a fixed-price AI project?

Any change to jobs, tools, integrations, data sources or thresholds is a change request with a written price and a schedule impact, approved before the work starts. Tuning prompts and logic to reach the thresholds already agreed is the vendor's job and is never a change request.

This split keeps both sides honest. The client can't add a new integration for free by calling it "tuning." The vendor can't bill extra for failing to reach a threshold it agreed to. Two more rules help:

  • Model changes by the provider. Pin the model version. If the provider retires it mid-project, the vendor re-runs the evals on the replacement and absorbs a fixed, agreed amount of re-tuning. Anything beyond that is shared through a change request.
  • Late client dependencies. If sandbox access or labelled data arrives late, the schedule moves by the same amount. The price only changes if the delay forces the team to stop and restart.

What milestones should a fixed-price AI agent project have?

Use four to five milestones, each ending with something you can run and score: discovery and the eval set, a thin end-to-end slice, full workflows with guardrails, production hardening, and go-live with warranty. Pay against milestones, not against hours.

  1. Discovery (fixed, priced separately). Jobs, tools, data review, a feasibility spike on the hardest task, the first labelled test set, thresholds and the fixed quote for the build.
  2. Thin slice. One job working end to end in staging against real (sandbox) systems, with tracing on. Baseline eval scores reported.
  3. Full workflows. All jobs, guardrails and approval steps. Evals at or near threshold.
  4. Hardening. Red-team set, cost and latency targets, audit log, runbooks, security review.
  5. Go-live and warranty. Production deployment in your account, final pass report, defect warranty period.

A working demo every week keeps the milestones honest. Problems surface early, not at the final reveal. Our pilot to production guide covers what changes between milestones 2 and 4.

Five milestones for a fixed-price AI agent: discovery priced separately covering jobs, tools, data review, a feasibility spike on the hardest task, a labelled test set and thresholds, producing a fixed build quote; a thin slice with one job end to end in staging producing baseline eval scores; full workflows with guardrails and approvals reaching evals at or near threshold; hardening with red-team set, cost and latency targets, audit log and security review; go-live in the client's cloud with final pass report and defect warranty.
Pay against milestones you can run and score

How does a vendor keep a fixed price honest with LLM uncertainty?

By buying down the uncertainty before quoting: a paid discovery with a feasibility spike on the hardest task, a baseline eval score, and a price for the build only after both are known. A vendor that quotes a fixed price for an agent without testing the hard part is either padding heavily or guessing.

Ask a vendor how it handles four risks. Feasibility: was the hardest task tried on your data before the quote? Cost to run: is there a hard cap on tokens and tool calls per task, enforced in code? Agentic loops can multiply costs, and our cost-to-run guide shows how. Drift: are model versions pinned and evals re-run on every change? Governance: does the design map to a recognised framework such as the NIST AI Risk Management Framework and its Generative AI Profile (NIST AI 600-1), so your risk team can review it?

When is fixed price a bad idea for an AI agent?

When you can't yet describe the jobs, when the success measure is "we'll know it when we see it," when the core systems have no API access, or when the project is research into whether the problem can be solved at all. Use a time-boxed discovery or a time-and-materials spike first, then fix the price for the build.

It is also a bad idea when an off-the-shelf product already does the job well enough. Buying is often cheaper. See build vs buy an AI agent and our general comparison of fixed price vs time and materials.

How BeevR builds this

We are a senior, founder-led team in Hanoi, and we build AI agents on fixed scope and fixed price per phase, from $10K per phase. Discovery produces the job list, a feasibility spike on the hardest task, a labelled test set and thresholds, and only then a fixed quote for the build. Every milestone ends with an eval report you can re-run yourself, and you see a working demo every Friday. Agents call your systems through official APIs with scoped credentials. Consequential actions go through a human approval step, and every model and tool call lands in an audit log. Model usage runs on your own account. For healthcare clients the model can run in your AWS account under the AWS BAA (Amazon Bedrock is on AWS's HIPAA Eligible Services list), and we sign a BAA as a business associate.

You own the source code, prompts, eval sets and IP from day one, with no per-user fees. We're rated 5.0 on Clutch. See our AI agent development page or the cost ranges in how much an AI agent costs.

Have an agent in mind? Book a 30-minute call. We'll help you write the first version of the scope and tell you honestly whether it should be fixed price yet.