Scoping and estimating an AI engagement

#The 60-second version

An opportunity assessment says an AI idea is worth trying. The next question from the buyer is always the same: what will it cost, how long will it take, and what do we get for the money?

A good AI engagement plan answers it in five moves:

1. Splits the work into phases: discovery, prototype, pilot, production. Each phase answers one question.
2. Ends every phase at a gate with written exit criteria. The gate can say go, iterate once, or stop.
3. Gives ranges, not single numbers. "Between 145,000 and 175,000 EUR" is more useful than "140,000 EUR", because it shows the risk.
4. Counts the run cost, not just the build: platform, people and model usage, every month after go-live.
5. Prices each phase by how certain it is. A two-week discovery can be a fixed price; a production build estimated before the pilot can't.

The result goes into a statement of work: what will be done, what won't, what each phase must prove, who owns each assumption, and how changes are agreed.

#Why it matters to the business

Most disappointment with AI projects starts in the proposal, not in the code.

• A single number hides the risk. Early in a project, estimates can be wrong by a factor of four either way, according to Steve McConnell's widely cited "cone of uncertainty". A single price quoted before anyone has seen the data is a guess.
• The model is the small part. A Google paper found model code is a small fraction of a real machine learning system. In SAP terms, add data, authorizations, testing, training and support. Proposals that price "the AI" undercount the rest.
• Gates protect the budget. If a phase can stop, you only risk the money spent up to that gate. A plan that goes straight from kickoff to build risks everything on day one.

Take the course's running example: an assistant that explains blocked sales orders to credit analysts. The business case assumed 135,000 EUR of one-time cost. In this topic's made-up estimate, the production phase alone comes out at 145,000 EUR at the midpoint and 175,000 EUR at the cautious end, and only stays within 135,000 in about 4 of 10 simulated outcomes. That doesn't kill the idea. The sponsor should fund the first three phases, about 63,000 EUR at the cautious end, and re-decide production after the pilot with a much narrower estimate.

#How SAP does it

SAP gives you a delivery method and cost tools, not an AI estimate. As of November 2026:

• SAP Activate is SAP's implementation methodology. SAP Learning lists the core phases Prepare, Explore, Realize and Deploy, with Discover before and Run after. Realize is built in waves of sprints of two to four weeks. An AI engagement inside an SAP program should line up its gates with these phases. SAP Learning
• Quality gates in SAP Cloud ALM's project and task management let a project hold a checklist review at the end of a phase. A post on SAP Community recommends at least one gate per phase. Use the same mechanism for your AI exit criteria. SAP Community
• SAP BTP commercial models decide how platform cost is paid. SAP Learning describes pay-as-you-go as suited to pilots, with no upfront commitment, next to prepaid CPEA or SAP BTPEA credits and fixed-price subscriptions. The SAP Discovery Center service estimator prices the services a design uses. SAP Learning
• SAP Business AI premium features use AI Units, bought annually and expiring after 12 months if unused, according to SAP's pricing page. The SAP Business AI Feature Estimator in SAP Discovery Center estimates the units a feature needs. SAP pricing

None of these estimates your team's days. That number is the partner's or the FDE's job, and it is the one to challenge.

#The engagement on one page

This is the shape of a scoped AI engagement. The numbers are made up for the blocked-orders assistant; the deep layer computes them.

P50 means half the simulated outcomes cost less. P80 means four in five do. The gap between them is the uncertainty you are pricing.

#Reading a partner's AI proposal

Use this table when a partner or vendor sends a proposal.

#Questions to ask

Your partner or vendor:

• What is the price of the first phase, and what will we know at its end that we don't know now?
• What would make you recommend stopping at each gate?
• What does it cost per month to run, and per blocked order (or invoice, or exception)? How much of that is people?
• Which assumptions do you need us to own, by when, and what happens to price and date if one fails?
• Is the production price firm now, or after the pilot? If firm now, where is the contingency and how big is it?

Your team:

• Which SAP Activate phase is the program in, and which of our gates line up with its quality gates?
• Who from the business is in the estimate, for how many days, and has their manager agreed?

#Common misconceptions

• "A precise estimate is a good estimate." Precision is not accuracy. Early on, an honest estimate is a wide range.
• "Contingency is a fixed percentage on top." A flat 10% or 20% is a guess. Contingency should come from the gap between the likely and the cautious estimate of the real risks.
• "Fixed price moves the risk to the partner." It moves the risk into the price, and into change requests. It works when the scope is clear, which is rarely true before a pilot.
• "The expensive part is the model." In most enterprise AI work, people and platform cost far more than model usage. In this topic's example, model tokens are about 2% of the monthly run cost.
• "Once signed, the estimate is done." Every gate is a chance to re-estimate with better facts. The production estimate should be much narrower after the pilot.

#Key terms

All course terms, A to Z, in the glossary

• Phase: a stretch of work with one question to answer, ending at a gate.
• Exit criteria: the written conditions a phase must meet to pass its gate.
• Three-point estimate: a low, likely and high value for one task.
• P50 / P80: the cost that half, or four in five, of the simulated outcomes stay under.
• Contingency: money held for known risks, sized from the range rather than a flat percentage.
• Run cost: what a live AI solution costs every month: platform, people and model usage.
• Statement of work (SOW): the contract document listing scope, deliverables, phases, exit criteria, assumptions, pricing and change control.
• Fixed price: one agreed price for a defined scope.
• Time and materials (T&M): paying for the days worked at agreed rates, often with a cap.
• Assumption owner: the person who must make an assumption true, by a date.

#Check yourself

#Mental model

An estimate is a range that narrows only when you make decisions, and each phase buys the next narrowing.

McConnell's cone of uncertainty puts early estimates at up to four times too high or too low. His white paper adds the part most teams forget: the cone narrows only as decisions remove variability. Time passing doesn't narrow it. A discovery that confirms data access, a prototype that shows users want the output, and a pilot that measures the real time saved are all decisions. Each one removes a large source of doubt.

So scoping an AI engagement is not "how much will the whole thing cost?" It is three smaller questions:

1. What is the cheapest phase that removes the biggest doubt? That is the next phase.
2. What must it prove to continue? Those are its exit criteria.
3. How certain is its cost? That decides how to price it.

The production phase still gets estimated up front, because the sponsor needs an order of magnitude to decide whether to start at all. It just isn't contracted until the pilot has narrowed it.

#How it works

flowchart LR
  A[Assessed<br/>opportunity] --> D[Discovery]
  D -->|gate| P[Prototype]
  P -->|gate| L[Pilot]
  L -->|gate +<br/>re-estimate| R[Production]
  R --> O[Run]
  D -.stop.-> X[Stop]
  P -.stop.-> X
  L -.stop.-> X

#Phases and what each one removes

The four phases follow the arc the earlier Unit 14 topics taught. Here they get budgets and contracts.

The GOV.UK Service Manual says a government discovery has no set length, "around 4 to 8 weeks" being typical. An AI discovery after an opportunity assessment is usually shorter, because framing and assessment already happened. If the problem is new to everyone, budget the longer discovery.

Exit criteria must be measurable and agreed before the phase starts. "Sponsor happy" isn't one. "Median handling time at most 12 minutes against a control group" is. Reuse the gate criteria from the rapid prototyping topic, and add one commercial criterion per gate: "cost per order within the run-cost estimate".

#Team shape

An AI engagement on SAP is a small cross-functional team, not a team of data scientists. For the blocked-orders assistant:

SAP Learning describes SAP Activate scrum teams of five to nine members. The production phase above fits one such team. The customer experts' days matter: if their manager hasn't agreed them, the gate can't be judged.

#Three-point estimates, widened

For each task, ask the person who will do it for a low, likely and high number of days. One number per task hides the risk; a range shows it.

People give ranges that are too narrow. The GAO Cost Estimating and Assessment Guide describes one correction: treat an expert's minimum and maximum as the 15th and 85th percentiles rather than true extremes. This topic's script uses a simpler version. It stretches each side of the range around the likely value by a factor (1.5 by default), so a 35 / 50 / 90 day task is simulated as 27.5 / 50 / 110.

#Correlation: why phases overrun together

If you simulate each task independently, overruns on one task cancel underruns on another, and the total looks safer than it is. Real projects don't behave that way. When data access is late, every task waits. When the SAP authorization concept is messier than expected, testing and security grow too.

The script models this with one shared phase factor per simulated run. A phase with uncertainty 0.6 can shrink by up to 30% or grow by up to 60% as a whole, with 1.0 most likely. The skew is deliberate: overruns are larger than underruns. Early phases get small values (0.1 for a two-week discovery), and production estimated before the pilot gets a large one (0.6).

These values are course inputs, not standards. Set them from your own past engagements where you have them (see Going further).

#P50, P80 and contingency

Run the simulation thousands of times and sort the outcomes. P50 is the middle; P80 is where four in five outcomes stay under. GAO's guide says a range of costs is more useful to decision-makers than a point estimate, and that without a risk analysis no confidence level can be given.

Three numbers are worth showing a sponsor:

• The "likely only" total: the sum of the likely values. It is almost always below P50, because high-side risks are larger than low-side ones. It is the number a single-point proposal quotes.
• P50: a fair midpoint for planning.
• P80: a cautious budget. Contingency is P80 minus P50, sized from the risks rather than a flat percentage. GAO's guide gives a case where a 10% contingency factor could not be justified.

#Money at risk by gate

Add each phase's P80 in order. The running total is the most you expect to have spent when each gate decides. For the blocked-orders example that is about 16,000 EUR after discovery and 63,000 EUR after the pilot, against about 238,000 EUR for all four phases. That table is often what persuades a sponsor to accept gates.

#Run cost: tokens are the small part

Run cost per month has three parts:

• Model usage: calls per item, times tokens per call, times the price per million tokens, times items per month. Token economics builds this number.
• Platform: runtime, model access, vector store, logging. On SAP BTP, price it with the SAP Discovery Center service estimator.
• People: support, prompt and model updates, evaluation runs, the product owner's time.

The Sculley et al. paper shows the model code is a small fraction of a real machine learning system; the cost follows the same shape. In the sample, model tokens are about 2% of the monthly run cost.

#Pricing the uncertainty

Match the contract to how certain the phase is.

The script uses a course rule: fixed price only where P80 is at most 15% above P50; otherwise capped T&M and a new estimate at the gate. Your organization will have its own threshold.

The US government's 18F team argues the same idea for agile software: their solicitation guide recommends a statement of objectives rather than a statement of work when "you don't know exactly what needs to be done", and labor-hour or time-and-materials contracts over firm fixed price. Most SAP customers sign SOWs, so the practical answer is a SOW per phase, with objectives and exit criteria in it, and the production phase contracted only after the pilot.

#What goes in the statement of work

#Build it yourself: an engagement estimator

You will build a command-line tool that reads an engagement file, checks it for the gaps a buyer would challenge, simulates P50 and P80 per phase, prints the money at risk by gate and the run cost, and writes a statement-of-work outline.

Before you start: complete Set up your computer for this course and Set up for Unit 14. They create your orchestrate-course folder, its .venv and the unit14 folder. This script uses only Python's built-in modules, so there is nothing new to install and no lines to add to requirements.txt.

flowchart LR
  I[init] --> J[(engagement JSON)]
  J --> V[review: gaps]
  V --> E[estimate: P50, P80,<br/>money at risk, run cost]
  E --> S[sow: statement of work outline]

#What you need

• Your course folder from the setup topics.
• About 45 minutes.
• No accounts and no cost. Every number in the sample is made up.

#Step 1: Open your course folder

1. Open VS Code, choose File > Open Folder, and open orchestrate-course.

2. Open a terminal: Terminal > New Terminal.

3. If the prompt doesn't start with (.venv), turn it on:

   • Windows (PowerShell):

     Copy

     .venv\Scripts\Activate.ps1

   • macOS / Linux:

     Copy

     source .venv/bin/activate

4. Check that the unit14 folder exists:

   • Windows (PowerShell):

     Copy

     dir unit14

   • macOS / Linux:

     Copy

     ls unit14

   If you see an error saying the folder doesn't exist, create it with mkdir unit14.

#Step 2: Create the script

1. In VS Code, right-click the unit14 folder, choose New File, name it engagement_estimator.py, paste the code below and save.

"""Engagement estimator: turn an assessed AI opportunity into phases, ranges, run cost and a statement of work.

Commands:
  init      write the made-up blocked-orders engagement (or, with --partner, a made-up partner proposal)
  review    check an engagement file for the gaps a buyer would challenge
  estimate  simulate effort and cost per phase (P50 and P80), money at risk per gate, and run cost
  sow       write a statement-of-work outline in markdown from the same file

Uses only Python's built-in modules. No account, no network. Every number in the sample is made up.
"""
import argparse
import json
import random
import sys
from pathlib import Path

DEFAULT_FILE = Path("unit14/engagement/blocked_orders.json")

SAMPLE = {
    "name": "Blocked-orders assistant",
    "currency": "EUR",
    "baseline": "17 minutes of credit analyst time per blocked sales order (discovery sample of 50 orders)",
    "target": "10 minutes per order, analyst still decides every release",
    "budget_in_business_case": 135000,
    "rates_per_day": {
        "fde": 900,
        "sap_functional": 850,
        "architect": 1000,
        "security": 850,
        "project_lead": 800,
        "change": 700,
        "customer_sme": 450,
    },
    "phases": [
        {
            "id": "discovery",
            "uncertainty": 0.1,
            "name": "Discovery (2 weeks, time-boxed)",
            "exit_criteria": [
                "Baseline measured on at least 50 blocked orders",
                "Read access to block reasons, credit data and customer notes confirmed in writing",
                "Sponsor signs metric, target and the pilot comparison method",
            ],
            "work": [
                {"role": "fde", "task": "Interviews, shadowing, data walk-through", "low": 6, "likely": 8, "high": 10},
                {"role": "sap_functional", "task": "Credit management and block reason review", "low": 3, "likely": 4, "high": 6},
                {"role": "customer_sme", "task": "Credit analysts' time for interviews and timing", "low": 2, "likely": 3, "high": 4},
                {"role": "project_lead", "task": "Plan, sponsor sessions, gate", "low": 2, "likely": 2, "high": 3},
            ],
        },
        {
            "id": "prototype",
            "uncertainty": 0.15,
            "name": "Prototype (1 day + 1 week)",
            "exit_criteria": [
                "Two analysts rate at least 16 of 20 context notes usable (gate 1)",
                "Four of five analysts use the prototype on at least 3 days and keep most notes (gate 2)",
            ],
            "work": [
                {"role": "fde", "task": "Data pull, prompt, simple screen", "low": 5, "likely": 7, "high": 10},
                {"role": "sap_functional", "task": "Check notes against SAP rules", "low": 2, "likely": 3, "high": 4},
                {"role": "customer_sme", "task": "Analysts rating and using the prototype", "low": 2, "likely": 3, "high": 5},
            ],
        },
        {
            "id": "pilot",
            "uncertainty": 0.25,
            "name": "Pilot (2 weeks with evaluation)",
            "exit_criteria": [
                "Evaluation set of 60 orders scores at least 0.85 with no unsafe proposal",
                "Median handling time at most 12 minutes against a control group",
                "Cost per order within the run-cost estimate",
            ],
            "work": [
                {"role": "fde", "task": "Harden, log, deploy to the pilot account", "low": 10, "likely": 14, "high": 20},
                {"role": "fde", "task": "Evaluation set and harness", "low": 3, "likely": 5, "high": 8},
                {"role": "security", "task": "Data protection and authorization review for real data", "low": 2, "likely": 4, "high": 8},
                {"role": "customer_sme", "task": "Pilot users and control group", "low": 5, "likely": 8, "high": 12},
                {"role": "project_lead", "task": "Pilot running, report, gate", "low": 3, "likely": 4, "high": 5},
            ],
        },
        {
            "id": "production",
            "uncertainty": 0.6,
            "name": "Production build and rollout",
            "exit_criteria": [
                "All credit teams live, with the release action approved by an analyst in SAP",
                "Four weeks of hypercare with no open severity 1 or 2 incident",
                "Benefits tracking handed to the process owner",
            ],
            "work": [
                {"role": "fde", "task": "Build: all block reasons, error handling, monitoring", "low": 35, "likely": 50, "high": 90},
                {"role": "architect", "task": "Design authority, identity and integration design", "low": 5, "likely": 8, "high": 15},
                {"role": "sap_functional", "task": "Authorization roles, test cases, process changes", "low": 8, "likely": 12, "high": 25},
                {"role": "security", "task": "Security review, penetration test follow-up", "low": 5, "likely": 8, "high": 15},
                {"role": "fde", "task": "Integration, regression and user acceptance testing", "low": 8, "likely": 12, "high": 25},
                {"role": "change", "task": "Training and change for all credit teams", "low": 8, "likely": 12, "high": 20},
                {"role": "fde", "task": "Hypercare", "low": 5, "likely": 8, "high": 15},
                {"role": "project_lead", "task": "Plan, steering, cutover", "low": 10, "likely": 14, "high": 20},
            ],
        },
    ],
    "run_cost": {
        "items_per_month": 6000,
        "item": "blocked order",
        "model_calls_per_item": 3,
        "input_tokens_per_item": 3700,
        "output_tokens_per_item": 300,
        "price_per_million_input": 4.0,
        "price_per_million_output": 16.0,
        "monthly": {
            "Platform (runtime, model access, vector store)": 2500,
            "Support and model updates (people)": 3000,
            "Evaluation and monitoring": 500,
        },
    },
    "assumptions": [
        {"text": "Read access to credit management data through released APIs is granted within 5 days", "owner": "Customer IT", "if_wrong": "Discovery extends; prototype starts late"},
        {"text": "Five credit analysts are available half a day a week during prototype and pilot", "owner": "Credit manager", "if_wrong": "Gate 2 cannot be judged; pilot slips"},
        {"text": "The pilot runs in the customer's own BTP account, not a trial account", "owner": "Customer IT", "if_wrong": "No real data in the pilot; the gate decides nothing"},
        {"text": "Model prices and token counts stay within 25% of the estimate", "owner": "FDE", "if_wrong": "Run cost changes; re-check at the pilot gate"},
    ],
    "risks": [
        {"text": "Analysts accept proposals without reading them (rubber-stamping)", "response": "Measure time per proposal and spot-check in the pilot"},
        {"text": "A standard SAP feature covers credit blocks during the project", "response": "Check the release at every gate; stop or shrink scope if it does"},
    ],
    "out_of_scope": [
        "Releasing orders automatically without an analyst",
        "Delivery and billing blocks (a later phase if the pilot succeeds)",
        "Changes to credit limits or credit rules in SAP",
    ],
    "pricing_rules": {
        "fixed_price_if_spread_at_most": 0.15,
        "note": "Course rule: fixed price only where P80 is at most 15% above P50",
    },
}

# A made-up partner proposal with the gaps buyers most often miss.
PARTNER = {
    "name": "Partner proposal: AI credit assistant (made up)",
    "currency": "EUR",
    "baseline": "",
    "target": "Up to 70% faster order release",
    "rates_per_day": {"consultant": 1100, "developer": 950},
    "phases": [
        {
            "id": "discovery",
            "name": "Kick-off and requirements",
            "exit_criteria": [],
            "work": [{"role": "consultant", "task": "Requirements workshops", "low": 10, "likely": 10, "high": 10}],
        },
        {
            "id": "production",
            "name": "Build and go-live (fixed price)",
            "exit_criteria": ["Go-live"],
            "work": [
                {"role": "developer", "task": "Build AI assistant", "low": 80, "likely": 80, "high": 80},
                {"role": "consultant", "task": "Testing and training", "low": 20, "likely": 20, "high": 20},
            ],
        },
    ],
    "assumptions": [{"text": "Customer provides all data", "owner": ""}],
}


def load(path):
    p = Path(path)
    if not p.exists():
        sys.exit(f"file not found: {p}. Run: python unit14/engagement_estimator.py init")
    try:
        return json.loads(p.read_text(encoding="utf-8"))
    except json.JSONDecodeError as e:
        sys.exit(f"ERROR {p} is not valid JSON: line {e.lineno}, column {e.colno}: {e.msg}")


def money(x):
    return f"{x:,.0f}"


def review(eng):
    """Return (problems, warnings): the questions a buyer would ask of this proposal."""
    problems, warnings = [], []
    rates = eng.get("rates_per_day", {})
    phase_ids = [ph.get("id") for ph in eng.get("phases", [])]
    for needed in ("discovery", "pilot"):
        if needed not in phase_ids:
            problems.append(f"no {needed} phase: the estimate jumps straight to a build")
    for ph in eng.get("phases", []):
        pid = ph.get("id", "?")
        u = ph.get("uncertainty", 0)
        if not isinstance(u, (int, float)) or not 0 <= u <= 1:
            problems.append(f"{pid}: uncertainty must be a number from 0 to 1")
        if not ph.get("exit_criteria"):
            problems.append(f"{pid}: no exit criteria, so the gate cannot say stop")
        if not ph.get("work"):
            problems.append(f"{pid}: no work items")
        for w in ph.get("work", []):
            label = f"{pid}: '{w.get('task', '?')}'"
            if w.get("role") not in rates:
                problems.append(f"{label} uses role '{w.get('role')}' with no day rate")
            try:
                low, likely, high = float(w["low"]), float(w["likely"]), float(w["high"])
            except (KeyError, TypeError, ValueError):
                problems.append(f"{label} needs numbers for low, likely and high")
                continue
            if not low <= likely <= high:
                problems.append(f"{label} needs low <= likely <= high")
            elif low == high:
                warnings.append(f"{label} is a single number, not a range")
        if pid == "pilot" and not any("evaluat" in w.get("task", "").lower() for w in ph.get("work", [])):
            problems.append("pilot: no evaluation work, so the gate has no evidence")
    run = eng.get("run_cost", {})
    if not run:
        problems.append("no run cost: the proposal prices the build only")
    else:
        people = [k for k in run.get("monthly", {}) if "people" in k.lower() or "support" in k.lower()]
        if not people:
            warnings.append("run cost has no support or people line")
    assumptions = eng.get("assumptions", [])
    if not assumptions:
        problems.append("no assumptions listed")
    for a in assumptions:
        if not a.get("owner"):
            problems.append(f"assumption without an owner: '{a.get('text', '?')}'")
    if not eng.get("out_of_scope"):
        warnings.append("nothing marked out of scope")
    if not eng.get("baseline") or not eng.get("target"):
        problems.append("no baseline or target: nothing to measure the pilot against")
    return problems, warnings


def simulate(eng, runs, seed, widen):
    """Monte Carlo over triangular ranges. Returns per-phase sorted cost and day samples."""
    rng = random.Random(seed)
    rates = eng["rates_per_day"]
    results = {}
    for ph in eng["phases"]:
        costs, days = [], []
        u = float(ph.get("uncertainty", 0))
        for _ in range(runs):
            # One shared factor per phase: when a phase goes badly, all its tasks tend to go badly together.
            # It can fall by u/2 or grow by u (overruns are larger than underruns).
            factor = rng.triangular(1 - u / 2, 1 + u, 1) if u else 1.0
            c = d = 0.0
            for w in ph["work"]:
                low, likely, high = float(w["low"]), float(w["likely"]), float(w["high"])
                # Experts' ranges tend to be too narrow: stretch each side around the likely value.
                low_w = max(0.0, likely - (likely - low) * widen)
                high_w = likely + (high - likely) * widen
                x = rng.triangular(low_w, high_w, likely) * factor
                d += x
                c += x * rates[w["role"]]
            costs.append(c)
            days.append(d)
        costs.sort()
        days.sort()
        results[ph["id"]] = {"cost": costs, "days": days}
    return results


def pct(sorted_values, q):
    i = min(len(sorted_values) - 1, int(q * len(sorted_values)))
    return sorted_values[i]


def single_point(ph, rates):
    return sum(float(w["likely"]) * rates[w["role"]] for w in ph["work"])


def run_cost(eng):
    r = eng["run_cost"]
    tokens = (r["input_tokens_per_item"] * r["price_per_million_input"]
              + r["output_tokens_per_item"] * r["price_per_million_output"]) / 1_000_000
    tokens_month = tokens * r["items_per_month"]
    fixed = sum(r["monthly"].values())
    total = tokens_month + fixed
    return tokens, tokens_month, fixed, total


def pricing_advice(p50, p80, rule):
    spread = (p80 - p50) / p50 if p50 else 0
    if spread <= rule:
        return spread, "fixed price"
    return spread, "capped time and materials; re-estimate at the gate"


def cmd_init(args):
    path = Path(args.file)
    if path.exists() and not args.force:
        sys.exit(f"{path} exists. Use --force to overwrite it.")
    path.parent.mkdir(parents=True, exist_ok=True)
    data = PARTNER if args.partner else SAMPLE
    path.write_text(json.dumps(data, indent=2), encoding="utf-8")
    print(f"Wrote {path}")
    print(f"All numbers are made up. Next: python unit14/engagement_estimator.py review --file {path.as_posix()}")


def cmd_review(args):
    eng = load(args.file)
    problems, warnings = review(eng)
    for p in problems:
        print(f"PROBLEM {p}")
    for w in warnings:
        print(f"WARNING {w}")
    if not problems and not warnings:
        print(f"OK {eng.get('name', args.file)}: every phase has exit criteria and ranges; run cost and assumptions are present")
    return 1 if problems else 0


def cmd_estimate(args):
    eng = load(args.file)
    problems, _ = review(eng)
    if problems:
        for p in problems:
            print(f"PROBLEM {p}")
        sys.exit("Fix the problems first (run: review).")
    cur = eng.get("currency", "")
    rule = eng.get("pricing_rules", {}).get("fixed_price_if_spread_at_most", 0.15)
    res = simulate(eng, args.runs, args.seed, args.widen)
    print(f"{eng['name']}: {args.runs:,} simulated runs, seed {args.seed}, ranges widened x{args.widen}")
    print(f"Amounts in {cur}. Days are person-days across all roles.\n")
    print(f"{'phase':<12}{'likely only':>13}{'P50':>11}{'P80':>11}{'days P50':>10}{'days P80':>10}  pricing")
    cum80 = 0.0
    at_risk = []
    total_p50 = total_p80 = 0.0
    for ph in eng["phases"]:
        r = res[ph["id"]]
        p50, p80 = pct(r["cost"], 0.5), pct(r["cost"], 0.8)
        d50, d80 = pct(r["days"], 0.5), pct(r["days"], 0.8)
        spread, advice = pricing_advice(p50, p80, rule)
        print(f"{ph['id']:<12}{money(single_point(ph, eng['rates_per_day'])):>13}{money(p50):>11}{money(p80):>11}"
              f"{d50:>10.0f}{d80:>10.0f}  {advice} (+{spread:.0%})")
        cum80 += p80
        at_risk.append((ph["id"], cum80))
        total_p50 += p50
        total_p80 += p80
    print(f"\n{'sum of phases':<12}{'':>13}{money(total_p50):>11}{money(total_p80):>11}  (P80s added up; a cautious ceiling)")
    budget = eng.get("budget_in_business_case")
    prod = res.get("production")
    if budget and prod:
        share = sum(1 for c in prod["cost"] if c <= budget) / len(prod["cost"])
        print(f"Business case assumed {money(budget)} for the build; the production phase stays within it in {share:.0%} of runs.")
    print("\nMoney spent (P80) by the time each gate is decided:")
    for pid, amount in at_risk:
        print(f"  after {pid:<11}{money(amount):>10}")
    tokens, tokens_month, fixed, total = run_cost(eng)
    r = eng["run_cost"]
    print(f"\nRun cost at {r['items_per_month']:,} {r['item']}s a month:")
    print(f"  {'Model tokens':<46}{money(tokens_month):>8}  ({tokens:.4f} per {r['item']}, {r['model_calls_per_item']} calls)")
    for k, v in r["monthly"].items():
        print(f"  {k:<46}{money(v):>8}")
    print(f"  {'Total per month':<46}{money(total):>8}  tokens are {tokens_month / total:.1%} of it; {total / r['items_per_month']:.2f} per {r['item']}")
    print("\nEstimates are ranges from made-up inputs. They show the method, not a real project.")


def cmd_sow(args):
    eng = load(args.file)
    problems, _ = review(eng)
    if problems:
        sys.exit("Fix the problems first (run: review).")
    cur = eng.get("currency", "")
    rule = eng.get("pricing_rules", {}).get("fixed_price_if_spread_at_most", 0.15)
    res = simulate(eng, args.runs, args.seed, args.widen)
    lines = [f"# Statement of work (outline): {eng['name']}", "",
             "Draft from `engagement_estimator.py`. All numbers are made up and show the method only.", "",
             "## Objective", "", f"- Baseline: {eng['baseline']}", f"- Target: {eng['target']}", "",
             "## Phases, exit criteria and pricing", ""]
    for ph in eng["phases"]:
        r = res[ph["id"]]
        p50, p80 = pct(r["cost"], 0.5), pct(r["cost"], 0.8)
        _, advice = pricing_advice(p50, p80, rule)
        lines += [f"### {ph['name']}", "",
                  f"Estimate: {money(p50)} to {money(p80)} {cur} (P50 to P80). Pricing: {advice}.", "",
                  "Exit criteria (decided at the gate: go, iterate once, or stop):", ""]
        lines += [f"- {c}" for c in ph["exit_criteria"]]
        lines += [""]
        if ph["id"] == "production":
            lines += ["This phase is estimated now but contracted only after the pilot gate, with a new estimate.", ""]
    tokens, tokens_month, fixed, total = run_cost(eng)
    lines += ["## Run cost after go-live (customer budget)", "",
              f"About {money(total)} {cur} a month, of which model tokens {money(tokens_month)}.", "",
              "## Out of scope", ""]
    lines += [f"- {x}" for x in eng.get("out_of_scope", [])]
    lines += ["", "## Assumptions", "", "| Assumption | Owner | If wrong |", "| --- | --- | --- |"]
    lines += [f"| {a['text']} | {a['owner']} | {a.get('if_wrong', '')} |" for a in eng["assumptions"]]
    lines += ["", "## Risks", "", "| Risk | Response |", "| --- | --- |"]
    lines += [f"| {x['text']} | {x['response']} |" for x in eng.get("risks", [])]
    lines += ["", "## Change control", "",
              "Any change to scope, exit criteria or assumptions is agreed in writing before work on it starts.",
              "", "## Decision asked for", "", "(Write the decision you need from the sponsor here.)", ""]
    out = Path(args.out)
    out.parent.mkdir(parents=True, exist_ok=True)
    out.write_text("\n".join(lines), encoding="utf-8")
    print(f"Wrote {out}")


def main():
    p = argparse.ArgumentParser(description="Estimate an AI engagement in phases, with ranges and gates")
    common = argparse.ArgumentParser(add_help=False)
    common.add_argument("--file", default=str(DEFAULT_FILE), help="engagement JSON file")
    sub = p.add_subparsers(dest="cmd", required=True)
    s = sub.add_parser("init", parents=[common], help="write the made-up sample engagement")
    s.add_argument("--force", action="store_true", help="overwrite an existing file")
    s.add_argument("--partner", action="store_true", help="write the made-up partner proposal instead")
    sub.add_parser("review", parents=[common], help="check the engagement for gaps")
    for name in ("estimate", "sow"):
        s = sub.add_parser(name, parents=[common])
        s.add_argument("--runs", type=int, default=20000, help="simulated runs (default 20000)")
        s.add_argument("--seed", type=int, default=42, help="random seed (default 42)")
        s.add_argument("--widen", type=float, default=1.5, help="stretch each range around likely (default 1.5)")
        if name == "sow":
            s.add_argument("--out", default="unit14/engagement/sow.md", help="where to write the outline")
    args = p.parse_args()
    handlers = {"init": cmd_init, "review": cmd_review, "estimate": cmd_estimate, "sow": cmd_sow}
    code = handlers[args.cmd](args)
    sys.exit(code or 0)


if __name__ == "__main__":
    main()

#Step 3: Write the sample engagement

1. Run:

   Copy

   python unit14/engagement_estimator.py init

2. You should see:

   Copy

   Wrote unit14/engagement/blocked_orders.json
   All numbers are made up. Next: python unit14/engagement_estimator.py review --file unit14/engagement/blocked_orders.json

3. Open unit14/engagement/blocked_orders.json in VS Code. Find:

   • phases: four phases, each with uncertainty, exit_criteria and work. Every work item has a role, a task and low, likely and high days.
   • rates_per_day: one day rate per role. These are course values, not market rates.
   • run_cost: volume, tokens per order, made-up token prices and three monthly lines.
   • assumptions, risks and out_of_scope: the parts of the SOW that aren't numbers.

   The token counts (3 calls, about 3,700 input and 300 output tokens per order) match the agent design in Token economics. The three monthly lines match the business case.

#Step 4: Review it like a buyer

1. Run:

   Copy

   python unit14/engagement_estimator.py review

2. You should see:

   Copy

   OK Blocked-orders assistant: every phase has exit criteria and ranges; run cost and assumptions are present

3. Now write the made-up partner proposal and review it:

   Copy

   python unit14/engagement_estimator.py init --partner --file unit14/engagement/partner.json
   python unit14/engagement_estimator.py review --file unit14/engagement/partner.json

4. You should see:

   Copy

   PROBLEM no pilot phase: the estimate jumps straight to a build
   PROBLEM discovery: no exit criteria, so the gate cannot say stop
   PROBLEM no run cost: the proposal prices the build only
   PROBLEM assumption without an owner: 'Customer provides all data'
   PROBLEM no baseline or target: nothing to measure the pilot against
   WARNING discovery: 'Requirements workshops' is a single number, not a range
   WARNING production: 'Build AI assistant' is a single number, not a range
   WARNING production: 'Testing and training' is a single number, not a range
   WARNING nothing marked out of scope

   Each line is a question for the partner, from the foundational layer's table. The command ends with exit code 1 when there is a PROBLEM, so you could run it in a pipeline. estimate refuses to run on a file with problems.

#Step 5: Estimate the engagement

1. Run:

   Copy

   python unit14/engagement_estimator.py estimate

2. You should see exactly this (the seed is fixed at 42):

   Copy

   Blocked-orders assistant: 20,000 simulated runs, seed 42, ranges widened x1.5
   Amounts in EUR. Days are person-days across all roles.
   
   phase         likely only        P50        P80  days P50  days P80  pricing
   discovery          13,550     14,586     15,910        18        20  fixed price (+9%)
   prototype          10,200     11,074     12,592        14        16  fixed price (+14%)
   pilot              27,300     30,730     34,710        39        44  fixed price (+13%)
   production        107,600    144,538    174,763       166       201  capped time and materials; re-estimate at the gate (+21%)
   
   sum of phases                 200,928    237,975  (P80s added up; a cautious ceiling)
   Business case assumed 135,000 for the build; the production phase stays within it in 38% of runs.
   
   Money spent (P80) by the time each gate is decided:
     after discovery      15,910
     after prototype      28,502
     after pilot          63,212
     after production    237,975
   
   Run cost at 6,000 blocked orders a month:
     Model tokens                                       118  (0.0196 per blocked order, 3 calls)
     Platform (runtime, model access, vector store)   2,500
     Support and model updates (people)               3,000
     Evaluation and monitoring                          500
     Total per month                                  6,118  tokens are 1.9% of it; 1.02 per blocked order
   
   Estimates are ranges from made-up inputs. They show the method, not a real project.

3. Read it like a sponsor:
   • "Likely only" is below P50 in every phase. For production, the sum of likely values is 107,600 EUR; the P50 is 144,538. That gap is what a single-point proposal leaves out.
   • The early phases are tight. Discovery, prototype and pilot have P80 within 9 to 14% of P50, so the course rule says fixed price.
   • Production is not. It is 21% wide, so the rule says capped T&M and a new estimate after the pilot.
   • The business case number is optimistic. The production phase stays within the business case's 135,000 EUR in only 38% of runs. Even the business case's 20% contingency (162,000 EUR) is below this P80 of 174,763.
   • Money at risk by the pilot gate is about 63,000 EUR, roughly a quarter of the full total.
   • Tokens are 1.9% of the run cost. People and platform are the rest.

#Step 6: Re-estimate after the pilot

A pilot narrows the production estimate. Suppose the pilot showed the data is clean and the build is close to the prototype. You update two numbers.

1. In unit14/engagement/blocked_orders.json, find the production phase. Change "uncertainty": 0.6 to "uncertainty": 0.25.

2. In the same phase, find the first work item, "Build: all block reasons, error handling, monitoring", and change its "high": 90 to "high": 65. Save the file.

3. Run estimate again:

   Copy

   python unit14/engagement_estimator.py estimate

4. The production line and the business case line should now read:

   Copy

   production        107,600    127,550    141,724       147       163  fixed price (+11%)
   ...
   Business case assumed 135,000 for the build; the production phase stays within it in 67% of runs.

   The range shrank because two decisions removed doubt, not because time passed. Production can now be offered at a fixed price. Put the two numbers back (0.6 and 90) before the next step, or run init --force to rewrite the sample.

#Step 7: Write the statement-of-work outline

1. Run:

   Copy

   python unit14/engagement_estimator.py sow

2. You should see:

   Copy

   Wrote unit14/engagement/sow.md

3. Open unit14/engagement/sow.md and choose Open Preview to the Side. It has the objective, each phase with its range, pricing and exit criteria, the run cost, out of scope, assumptions, risks, change control and an empty Decision asked for line. It is an outline for your SOW, not a contract; a real SOW also goes through your legal and procurement teams.

#Step 8: Save your work

git add unit14/engagement_estimator.py unit14/engagement/
git commit -m "Unit 14: engagement estimator and SOW outline for the blocked-orders assistant"

#How the code works

#Estimate your own engagement

1. Copy the sample and give it a new name, for example unit14/engagement/my_engagement.json.
2. Change the phases, tasks and ranges. Ask the people who will do each task for their low, likely and high days. If you wrote a report in the data readiness exercise, turn each of its top fixes into a work item.
3. Set uncertainty per phase: small for time-boxed phases, larger for anything estimated before a pilot.
4. Run review --file unit14/engagement/my_engagement.json until there are no PROBLEM lines, then estimate and sow with the same --file.

#If something goes wrong

#The SAP way

SAP doesn't publish an estimating method for custom AI work. It gives you a delivery frame, a place to hold gates, and calculators for its own services. Names and scope below are as of November 2026.

#Fit the phases into SAP Activate

SAP Learning describes SAP Activate with core phases Prepare, Explore, Realize and Deploy, plus Discover and Run. An AI engagement on a running SAP program should map onto that frame rather than invent its own:

If the AI work runs outside an SAP program, keep the four phases as they are.

#Hold the gates where the program holds its gates

A post on SAP Community about SAP Cloud ALM project and task management recommends at least one quality gate at the end of each phase, reviewed with a checklist. Put the AI exit criteria into the same checklist, so the program's steering committee decides on them, not a side meeting. SAP Community

#Price the platform with SAP's own estimators

• SAP BTP services (runtime, model access, vector database, logging): SAP Learning's solution architect course shows the SAP Discovery Center service estimator. You name the estimate, choose a region and commercial model, add services and enter their sizing metrics, then export it as PDF or CSV. SAP Learning
• The commercial model per phase. The same course says pay-as-you-go suits small apps, proofs of concept and pilots, with no upfront investment; CPEA or SAP BTPEA prepay credits with an annual commitment; and subscriptions fix the price of named services. One workable pattern is to pilot in a pay-as-you-go account and move to the customer's consumption contract for production. Confirm what the customer's contract already includes before assuming either.
• SAP Business AI features (premium features embedded in SAP applications): SAP's pricing page says these use AI Units, bought annually, which expire after 12 months if unused. The SAP Business AI Feature Estimator in SAP Discovery Center estimates the AI Units a feature needs. If the opportunity assessment chose a standard feature, this replaces most of the build estimate. SAP pricing · SAP Learning

#What SAP doesn't do for you

SAP's tools price SAP services. They don't estimate your team's days, the customer's experts' time, the authorization and test work, or the run-cost people. Those are most of the money, and they come from your three-point estimates.

#Build vs. SAP

Here "build" means estimating with your own model, and "SAP" means SAP's estimators and templates.

#Architecture view

Scoping is where architecture choices first turn into money. Each one below moves the estimate.

Where it sits. The blocked-orders assistant reads credit and order data from S/4HANA through released APIs, runs on SAP BTP, and writes nothing to SAP without an analyst's approval. Each phase uses a different slice of that landscape.

flowchart LR
  subgraph S4[S/4HANA]
    O[Sales orders,<br/>credit data]
  end
  subgraph Pilot[Pilot: pay-as-you-go BTP account]
    PA[Assistant,<br/>pilot users only]
  end
  subgraph Prod[Production: customer BTP contract]
    A[Assistant] --> M[Monitoring,<br/>evaluation]
  end
  X[Sample extract,<br/>prototype] -.prototype.-> PA
  O -->|released APIs, read| PA
  O -->|released APIs, read| A
  A -->|release after<br/>analyst approval| O

Non-functional requirements that move the estimate.

Integration and identity. Read access goes through released S/4HANA APIs under a technical user scoped to the needed data, or under the analyst's own identity where authorizations must follow the user (see agent permissions and SAP authorizations). The choice changes the estimate: principal propagation needs identity setup and more security review days. Any release action goes back to SAP only after the analyst approves, as in human approval in AI workflows.

The decision to record (short ADR form from the design-doc topic):

• Context: the pilot needs real orders and five analysts for two weeks; production will run on the customer's consumption contract; the customer's procurement wants a single SOW.
• Options: A pilot in a BTP trial account; B pilot in a separate pay-as-you-go account owned by the customer; C pilot directly in the production contract's subaccount.
• Decision: B. A BTP trial is a non-commercial, time-limited offering whose data is deleted at the end, as rapid prototyping covered. C makes the pilot look like a commitment before the gate.
• Consequences: one SOW with a fixed-price pilot and a production phase priced only as a capped range; the platform cost line moves to the customer's contract after the pilot gate; a migration step from the pilot account to production is added to the production estimate.

What a design review will ask.

1. Which estimate did you give the sponsor: likely, P50 or P80? Why?
2. Where does real customer data first appear, and in whose account?
3. Which architecture choice drives the high end of the production range?
4. What is the cost per order to run, and what happens to it at month-end peaks?
5. Which identity does the assistant use to read SAP, and was its security review estimated?

#Production concerns

• Security and SAP authorizations. Estimate the security review and the authorization design as tasks, not as overhead. A new app that reads credit data needs roles; a functional consultant and a security reviewer spend real days on them. If the assistant ever proposes a change in SAP, the approval step is in scope from the first SOW.
• Evaluation. The pilot's exit criteria rely on an evaluation set and a control group. Building them is work: in the sample, 3 to 8 days. A pilot without evaluation work in its estimate can't pass its gate honestly; the script's review flags it.
• Cost. Report run cost per transaction, by part. Recheck model prices and token counts at every gate, because both change. The sample assumes they stay within 25% and names the FDE as owner of that assumption.
• Operations. Production includes hypercare and a named support model. The run-cost people line covers prompt and model updates and the regular evaluation runs from observability.
• Clean core. Estimate side-by-side work on released APIs. If someone proposes modifying SAP code to save days, put the extra upgrade test effort over the solution's life into the comparison.
• Commercial hygiene. Keep assumptions, risks and out-of-scope items in the same file as the numbers, so the SOW and the estimate can't drift apart.

#Pitfalls

• Quoting the likely total. The sum of likely values is below P50 almost every time. It becomes the budget, and every outcome after that looks like an overrun.
• Flat contingency. Adding 10% or 20% to a single number hides where the risk is. Size contingency from P80 minus P50, task by task.
• Simulating tasks as independent. Independent draws cancel out and make the total look safe. Model the shared factor that makes tasks overrun together.
• Fixing the production price before the pilot. The partner either pads it or recovers it through change requests. Price it as a range; contract it after the gate.
• Leaving out the customer's people. Analysts' and functional experts' days are real cost and real constraints. If they aren't free, the pilot can't run.
• Pricing only tokens. Tokens are a small share of run cost. Platform and people are most of it.
• Exit criteria written after the phase. They will match what happened. Agree them in the SOW.
• A proof of concept with no question. If nobody can say what the prototype must prove, it is a demo with a budget.

#Exercise

Estimate the next phase of the blocked-orders assistant: extending it from credit blocks to delivery blocks, which the sample lists as out of scope. Then review a partner's version of the same work.

1. Copy the sample:

   • Windows (PowerShell):

     Copy

     Copy-Item unit14/engagement/blocked_orders.json unit14/engagement/delivery_blocks.json

   • macOS / Linux:

     Copy

     cp unit14/engagement/blocked_orders.json unit14/engagement/delivery_blocks.json

2. In delivery_blocks.json, change name to Delivery-blocks extension and set budget_in_business_case to a number you choose for the extension.

3. Rewrite the discovery phase's exit criteria for delivery blocks (for example, a baseline on 50 delivery-blocked orders), and adjust its work ranges. Keep the phase: review reports a problem if there is no discovery.

4. Add one production work item for the data work delivery blocks need, for example {"role": "sap_functional", "task": "Delivery block reasons and shipping data review", "low": 4, "likely": 6, "high": 12}.

5. Set each phase's uncertainty. Explain your choice for production in one sentence in a new "uncertainty_note" field on that phase.

6. Move Delivery and billing blocks out of out_of_scope, and add one new out-of-scope line.

7. Run review, then estimate and sow, each with --file unit14/engagement/delivery_blocks.json. Use --out unit14/engagement/delivery_blocks_sow.md for sow.

8. Run review --file unit14/engagement/partner.json again. Under Decision asked for in delivery_blocks_sow.md, replace the placeholder with the decision you want from the sponsor, and add five questions you would send the partner, one per PROBLEM line.

9. Commit:

   Copy

   git add unit14/engagement/
   git commit -m "Unit 14: delivery-blocks engagement estimate and partner questions"

Done when review --file unit14/engagement/delivery_blocks.json prints OK, estimate prints a P50 and P80 for every phase and a pricing line for each, and delivery_blocks_sow.md names a decision and holds five partner questions that each trace to a PROBLEM line. Keep the file: the AI roadmap topic sequences engagements like this one into a roadmap.

#Going further

For readers who want the reasoning behind ranged estimates and calibration.

The script's ranges come from people's judgment, and judgment is biased. Three sources go deeper:

• Bent Flyvbjerg, "From Nobel Prize to Project Management: Getting Risks Right". Explains inaccurate forecasts by optimism bias and strategic misrepresentation, and proposes reference class forecasting: base the forecast on the actual outcomes of comparable past projects instead of the plan of this one. For an FDE team, the reference class is your own past AI engagements: record estimate versus actual per phase, and use the overrun distribution to set each phase's uncertainty. arXiv
• GAO Cost Estimating and Assessment Guide, Step 9. A public, detailed method for risk and uncertainty analysis: collecting risk data from experts, correcting overly narrow ranges (treating stated extremes as the 15th and 85th percentiles), and why contingency percentages need a justification. The script's --widen factor is a simplified stand-in for that correction. GAO
• Steve McConnell, "The Cone of Uncertainty". The original argument that early estimates carry about 4x variability either way, and that only decisions narrow it. Read it for the distinction between the cone (uncertainty you reduce) and the "cloud" (uncertainty a poorly controlled project keeps). Construx

Two refinements to try on the script: replace the single phase factor with factors per risk driver (data access, authorization complexity, user availability) that affect several tasks each; and calibrate widen by comparing your past experts' ranges with what actually happened. Both turn the course rule into your organization's own.

#Check yourself