Adoption and change management for AI

#The 60-second version

An AI assistant only creates value when people use it in their daily work. That sounds like any software rollout, but there is one big difference. The assistant is sometimes wrong. So the goal is not "everyone clicks accept". The goal is the right reliance: people accept it when it is right and catch it when it is wrong.

That gives two ways to fail:

• Non-use. People keep their old spreadsheet, skip the assistant on hard cases, or never open it. The business case never arrives.
• Over-trust. People accept every suggestion in seconds. The numbers look great until a wrong suggestion goes through: a blocked order released for a customer who should have stayed blocked.

Adoption and change management for AI is the work that steers between the two. You map who is affected, recruit champions, put help inside the tool and show people how good the assistant really is, misses included. Then you measure four things every week: reach (who uses it), coverage (on how much of the work), reliance (do they accept the right suggestions and catch the wrong ones) and the business KPI.

The running example is the blocked sales orders assistant from rapid prototyping: cut handling from about 17 to 10 minutes per order, with the analyst still deciding every release.

#Why it matters to the business

The business case assumes adoption. A pilot with five keen analysts can show 10 minutes per order. If half of 24 analysts use it on a third of their orders, the saving is a fraction of the plan.

Over-trust turns a time saving into a risk. Parasuraman and Manzey (2010) describe automation bias: people relying on a decision aid so much that they miss its errors or follow its wrong advice. It affects experts as well as novices. Their review found that training or instructions alone don't prevent it. In order-to-cash, that means a released order that should have stayed blocked, and a bad debt nobody checked.

People are confident but the tools are not yet reliable. In a WalkMe survey of 2,037 US workers who use AI at work (June 2026), 90% said they feel confident using AI, but only 24.6% said it works on the first try. Workers most wanted guidance built into the tools they already use.

Adoption is where an FDE engagement is won or lost. If the credit team's lead never bought in, the pilot dies quietly after the FDE leaves.

A concrete picture. Six weeks after go-live, the blocked orders dashboard shows three credit teams:

The team with the best handling time is the one the controller should worry about most. Only the reliance numbers show it.

#How SAP does it

SAP offers methods and tools for the people side of a rollout. None is specific to a custom AI assistant, so combine them with your own measurement. As of October 2026:

• Organizational change management (OCM) in SAP Activate. SAP Learning describes an OCM approach aligned with the six SAP Activate phases. Stakeholder and change impact analysis start in Prepare; in Run, the team tracks actual user adoption and plans activities to raise it. SAP calls OCM "an attitude", not a checklist.
• WalkMe, an SAP company. SAP completed the acquisition of WalkMe, a digital adoption platform, on 12 September 2024. SAP said WalkMe's AI capabilities would extend Joule with context-aware help across workflows. SAP's product pages list WalkMe offerings for specific product lines. The WalkMe for SAP SuccessFactors HCM page describes in-app guidance by role and language, and analytics that measure adoption across HR processes and AI capabilities.
• In-app help on SAP BTP. SAP's design guidelines describe the Web Assistant, an add-on framework that puts help topics, guided tours and what's-new notes into the application screen. The guidelines point to SAP Enable Now documentation for its technical requirements.
• Adoption campaigns for Joule. SAP Learning offers a learning journey for adoption leads, "Driving SAP Joule for Consultants Adoption". It covers planning a campaign, refining it with usage data and showing impact to leadership.
• Process outcomes. SAP Signavio Process Insights applies process mining to system event logs and reports cycle times, throughput and error rates. That checks whether the process got faster, not just whether people clicked.

None of these tells you whether users accept the right suggestions. That needs your own log and an audit sample.

#An adoption plan on one page

A simple way to plan is the ADKAR model from Prosci: awareness, desire, knowledge, ability and reinforcement. Each must be in place for a person to change how they work.

Three moves matter more for AI than for ordinary software:

1. Show how good it really is. Share the evaluation results, misses included, before go-live. Microsoft Research's guidelines for human-AI interaction start with making clear what the system can do, and how well.
2. Roll out in waves. Start the next wave only when earlier teams show the right reliance, not just high usage.
3. Audit a sample of decisions. A reviewer checks a random share of cases afterwards. It is the only way to know whether people catch wrong suggestions.

#Questions to ask

Your team:

• Who is affected, and how does each group's day change? Who loses work they were measured on?
• Who is the champion in each team, and how much time do they have?
• What happens to the time saved? If nobody can answer, people will assume the worst.

The process owner:

• Which wrong suggestion would hurt most if it went through?
• Who will audit a sample of decisions, and how many per week?
• What numbers would make you pause the next rollout wave?

IT, HR and employee representatives:

• Can we log usage, and at team or individual level?
• Who may see the adoption dashboard? Will it ever feed performance reviews?

The vendor or partner:

• What usage and feedback data does the product give us? Can we see accept and override decisions, or only logins?
• How do you tell users when the model or its behavior changes?

#Common misconceptions

• "If it's good, people will use it." Without a reason, help and a champion, a good tool sits unused.
• "More usage means more value." High acceptance with very short checks can mean rubber-stamping.
• "Training will stop people over-trusting it." Research found training alone doesn't prevent automation bias. Add evidence on screen, audits and shared misses.
• "Change management starts at go-live." In SAP Activate, OCM starts in the first phases.
• "The adoption dashboard is a way to rank people." Use it to fix the tool and process. If people think it ranks them, they game it.

#Key terms

All course terms, A to Z, in the glossary

• Adoption: people using a new tool in real work, well enough for the business result to appear.
• Organizational change management (OCM): planned work to help people move to a new way of working.
• Reach: the share of intended users who used the tool in a period.
• Coverage: the share of eligible work items where the tool was used.
• Appropriate reliance: accepting the AI when it is right and overriding it when it is wrong.
• Automation bias: relying on a decision aid so much that its errors are missed or followed.
• Over-reliance / under-reliance: accepting wrong suggestions / overriding right ones.
• Audit sample: a random share of decisions checked after the fact by a reviewer.
• Champion: a respected user in a team who helps colleagues and passes feedback on.
• Digital adoption platform: software that shows guidance inside other applications and measures their use. WalkMe is one.
• Rollout wave: a group of teams that goes live together.

#Check yourself

#Mental model

Think of every suggestion the assistant makes as landing in one of four boxes. The assistant was right or wrong. The user accepted or overrode it.

Adoption work moves cases into the two good boxes. Most usage dashboards can't see these boxes at all. They count logins and clicks, which tell you nothing about the right column versus the left. To see the boxes you need two things you build yourself: a decision log (what was suggested, what the user did, how long they looked) and an audit sample (a reviewer later marks whether the suggestion was right).

Put a funnel in front of the boxes and you have the whole picture:

1. Reach: how many of the intended users used it this period.
2. Coverage: on what share of the eligible work.
3. Reliance: over-reliance and under-reliance from the audit sample, plus review time as an early warning.
4. KPI: the business number the project was framed around, here minutes per blocked order.

Each stage has a different cause and a different fix. Low reach is usually access, awareness or a missing reason. Low coverage means the assistant doesn't help on some kinds of cases. Over-reliance needs evidence on screen and audits. Under-reliance needs proof that the assistant is right more often than people think.

#How it works

#Instrument the decision, not the login

Log one event per suggestion, at the moment the user decides. Keep it small:

Also record the volume of eligible work per team and week, from SAP itself. Coverage is events divided by that volume.

flowchart LR
  A[Assistant suggests] --> U{Analyst}
  U -->|accept / override| L[(Decision log)]
  S[(SAP: blocked orders per week)] --> R[Weekly report]
  L --> R
  L --> Q[Random audit sample]
  Q -->|right or wrong| L
  R --> T[Team huddle and fixes]

#Why the audit sample matters

Accuracy on the evaluation set (Unit 8, and the evaluation harness topic) tells you how often the assistant is wrong. It doesn't tell you whether users catch those errors. Only an audit of real decisions does that. Two practical points:

• Sample randomly, across teams, so busy teams aren't skipped.
• Wrong suggestions are rare, so you need many audited cases to judge over-reliance. With 88% accuracy and a 25% audit share, a team handling 80 orders a week yields about 2 or 3 audited wrong suggestions a week. Pool several weeks before you trust the number, and say "not enough data" rather than guess.

#Design the screen for calibrated trust

Microsoft Research's 18 guidelines for human-AI interaction (CHI 2019) are a good checklist for the assistant's screen. The ones that matter most for adoption:

Showing evidence matters most against automation bias. It turns "accept?" into "does this evidence support the suggestion?".

#Run the rollout in waves

flowchart LR
  P[Pilot team] --> W1[Wave 1: 2 teams]
  W1 --> G1{Reliance OK?}
  G1 -->|yes| W2[Wave 2]
  G1 -->|no| F[Fix screen, coach, audit more]
  F --> W1
  W2 --> G2{Reliance OK?}
  G2 -->|yes| W3[Wave 3]

Each wave has a champion per team, a short kickoff with the team lead, help inside the tool and a weekly 15-minute huddle in the first month. At each gate you read the four numbers for the earlier waves.

#Read the numbers as a diagnosis

#Build it yourself: a weekly adoption report

You will build a command-line tool that reads three small files (users, weekly volume of blocked orders, and the decision log) and prints a report for each team: reach, coverage, acceptance, review time, over- and under-reliance, handling minutes and flags with a suggested action. It also writes the report as a Markdown file you can paste into the weekly review. Teams with fewer than five people are not reported, to protect individuals.

Before you start: complete Set up your computer for this course and Set up for Unit 14. They create your orchestrate-course folder, its .venv and the unit14 folder. This script uses only Python's built-in modules, so there is nothing new to install, and it doesn't call a model.

flowchart LR
  S[sample] --> U[(users.csv)]
  S --> V[(volume.csv)]
  S --> E[(events.csv)]
  U --> R[report]
  V --> R
  E --> R
  R --> M[adoption_report.md]

#What you need

• Your course folder from the setup topics.
• About 35 minutes.
• No accounts and no cost. Every number in the sample is made up; the 17-minute baseline and 10-minute target match the rapid prototyping sample.

#Step 1: Open your course folder

1. Open VS Code, choose File > Open Folder, and open orchestrate-course.

2. Open a terminal: Terminal > New Terminal.

3. If the prompt doesn't start with (.venv), turn it on:

   • Windows (PowerShell):

     Copy

     .venv\Scripts\Activate.ps1

   • macOS / Linux:

     Copy

     source .venv/bin/activate

4. Check that the unit14 folder exists:

   • Windows (PowerShell):

     Copy

     dir unit14

   • macOS / Linux:

     Copy

     ls unit14

   If you see an error saying the folder doesn't exist, create it with mkdir unit14.

#Step 2: Create the script

1. In VS Code, right-click the unit14 folder, choose New File, name it adoption_report.py, paste the code below and save.

"""Adoption report for an AI assistant: reach, coverage, reliance and the KPI.

Commands:
  sample   write made-up usage logs for the blocked sales orders assistant
  report   read the logs, print a report per team and write adoption_report.md

Uses only Python's built-in modules. No account, no network, no model call.
"""

import argparse
import csv
import random
import statistics
import sys
from pathlib import Path

DEFAULT_DIR = Path("unit14") / "adoption"

# Thresholds. Agree on your own with the process owner before go-live.
THRESHOLDS = {
    "reach_min": 0.60,          # share of licensed users active in the last 2 weeks
    "coverage_min": 0.50,       # share of blocked orders where the assistant was used
    "over_reliance_max": 0.30,  # share of WRONG suggestions that users accepted
    "under_reliance_max": 0.30, # share of RIGHT suggestions that users overrode
    "fast_accept_seconds": 15,  # median review time below this looks like rubber-stamping
    "min_group": 5,             # never report a group smaller than this (privacy)
    "min_audited_wrong": 5,     # too few audited wrong suggestions to judge over-reliance
}

BASELINE_MINUTES = 17.0  # minutes per blocked order before the assistant (from the frame)
TARGET_MINUTES = 10.0    # the target agreed with the process owner

ACTIONS = {
    "LOW REACH": "Ask the team lead why people don't open it; check access, roles and training dates.",
    "LOW COVERAGE": "People use it for some orders only. Shadow two analysts: which orders do they skip, and why?",
    "OVER-TRUST": "Users accept wrong suggestions. Show the evidence next to each suggestion, "
                  "share real misses in the team huddle, and audit more of this team's decisions.",
    "UNDER-TRUST": "Users override right suggestions. Share the audit results with the team "
                   "and ask the champion to walk through cases where the assistant was right.",
    "SLOW KPI": "Handling time is above target. Look at where the minutes go before changing the model.",
}

USERS_FIELDS = ["user_id", "team", "wave"]
VOLUME_FIELDS = ["team", "week", "blocked_orders"]
EVENT_FIELDS = ["event_id", "week", "user_id", "sales_order", "suggestion",
                "decision", "review_seconds", "handle_minutes", "audited", "suggestion_correct"]


# ---------- sample data ----------

# Each team behaves differently so every flag shows up in the sample.
# use = chance an analyst uses the assistant on an order; p_accept_right / p_accept_wrong =
# chance they accept a right / wrong suggestion; review = typical seconds spent checking.
TEAM_PROFILES = {
    "Credit EMEA":     {"users": 8, "wave": 1, "active": 7, "use": 0.85,
                        "p_accept_right": 0.90, "p_accept_wrong": 0.15, "review": 75, "minutes": 9.5},
    "Credit Americas": {"users": 7, "wave": 1, "active": 7, "use": 0.90,
                        "p_accept_right": 0.99, "p_accept_wrong": 0.85, "review": 7, "minutes": 6.0},
    "Credit APAC":     {"users": 6, "wave": 2, "active": 3, "use": 0.35,
                        "p_accept_right": 0.55, "p_accept_wrong": 0.10, "review": 140, "minutes": 15.5},
    "Credit Shared":   {"users": 3, "wave": 2, "active": 3, "use": 0.80,
                        "p_accept_right": 0.90, "p_accept_wrong": 0.20, "review": 60, "minutes": 10.0},
}
WEEKS = 6
ORDERS_PER_USER_WEEK = 12
SUGGESTION_ACCURACY = 0.88
AUDIT_SHARE = 0.25


def write_csv(path, fields, rows):
    with path.open("w", newline="", encoding="utf-8") as f:
        w = csv.DictWriter(f, fieldnames=fields)
        w.writeheader()
        w.writerows(rows)


def cmd_sample(args):
    folder = Path(args.dir)
    folder.mkdir(parents=True, exist_ok=True)
    targets = [folder / n for n in ("users.csv", "volume.csv", "events.csv")]
    if any(p.exists() for p in targets) and not args.force:
        print(f"Files already exist in {folder}. Add --force to overwrite them.")
        return 1
    rng = random.Random(42)  # fixed seed: everyone gets the same sample
    users, volume, events = [], [], []
    order_no = 4000000
    for team, prof in TEAM_PROFILES.items():
        ids = [f"{team.split()[1][:3].upper()}{i + 1:02d}" for i in range(prof["users"])]
        users += [{"user_id": u, "team": team, "wave": prof["wave"]} for u in ids]
        first_week = 1 if prof["wave"] == 1 else 3
        for week in range(1, WEEKS + 1):
            volume.append({"team": team, "week": week,
                           "blocked_orders": prof["users"] * ORDERS_PER_USER_WEEK})
            if week < first_week:
                continue
            for u in ids[: prof["active"]]:
                for _ in range(ORDERS_PER_USER_WEEK):
                    order_no += 1
                    if rng.random() > prof["use"]:
                        continue  # analyst handled this order without the assistant
                    correct = rng.random() < SUGGESTION_ACCURACY
                    p = prof["p_accept_right"] if correct else prof["p_accept_wrong"]
                    decision = "accepted" if rng.random() < p else "overridden"
                    review = max(2, int(rng.gauss(prof["review"], prof["review"] * 0.3)))
                    minutes = max(2.0, rng.gauss(prof["minutes"], 2.0))
                    audited = rng.random() < AUDIT_SHARE
                    events.append({
                        "event_id": len(events) + 1, "week": week, "user_id": u,
                        "sales_order": order_no,
                        "suggestion": rng.choice(["release", "keep blocked", "ask sales"]),
                        "decision": decision, "review_seconds": review,
                        "handle_minutes": f"{minutes:.1f}",
                        "audited": "yes" if audited else "no",
                        "suggestion_correct": ("yes" if correct else "no") if audited else "",
                    })
    write_csv(targets[0], USERS_FIELDS, users)
    write_csv(targets[1], VOLUME_FIELDS, volume)
    write_csv(targets[2], EVENT_FIELDS, events)
    print(f"Wrote {len(users)} users, {len(volume)} volume rows and {len(events)} events to {folder}")
    print("Every number is made up. Next: python unit14/adoption_report.py report")
    return 0


# ---------- loading ----------

def read_csv(path, fields):
    if not path.exists():
        raise SystemExit(f"ERROR  {path} not found. Run the sample command first, or fix --dir.")
    with path.open(newline="", encoding="utf-8") as f:
        reader = csv.DictReader(f)
        missing = [c for c in fields if c not in (reader.fieldnames or [])]
        if missing:
            raise SystemExit(f"ERROR  {path.name} is missing column(s): {', '.join(missing)}")
        return list(reader)


def load(folder):
    users = read_csv(folder / "users.csv", USERS_FIELDS)
    volume = read_csv(folder / "volume.csv", VOLUME_FIELDS)
    events = read_csv(folder / "events.csv", EVENT_FIELDS)
    team_of = {u["user_id"]: u["team"] for u in users}
    for e in events:
        if e["user_id"] not in team_of:
            raise SystemExit(f"ERROR  event {e['event_id']}: user {e['user_id']} is not in users.csv")
        if e["decision"] not in ("accepted", "overridden"):
            raise SystemExit(f"ERROR  event {e['event_id']}: decision must be accepted or overridden")
        e["team"] = team_of[e["user_id"]]
        e["week"] = int(e["week"])
    return users, volume, events


# ---------- metrics ----------

def share(part, whole):
    return part / whole if whole else None


def team_metrics(team, users, volume, events, last_weeks):
    t = THRESHOLDS
    members = [u for u in users if u["team"] == team]
    m = {"team": team, "users": len(members), "flags": []}
    if len(members) < t["min_group"]:
        m["suppressed"] = True
        return m
    recent = [e for e in events if e["team"] == team and e["week"] in last_weeks]
    orders = sum(int(v["blocked_orders"]) for v in volume
                 if v["team"] == team and int(v["week"]) in last_weeks)
    m["reach"] = share(len({e["user_id"] for e in recent}), len(members))
    m["coverage"] = share(len(recent), orders)
    m["acceptance"] = share(sum(e["decision"] == "accepted" for e in recent), len(recent))
    accepted_secs = [int(e["review_seconds"]) for e in recent if e["decision"] == "accepted"]
    m["review_median"] = statistics.median(accepted_secs) if accepted_secs else None
    m["minutes_median"] = (statistics.median(float(e["handle_minutes"]) for e in recent)
                           if recent else None)
    # Reliance needs enough audited cases, so it uses every week since go-live.
    audited = [e for e in events if e["team"] == team and e["audited"] == "yes"]
    right = [e for e in audited if e["suggestion_correct"] == "yes"]
    wrong = [e for e in audited if e["suggestion_correct"] == "no"]
    m["audited"], m["wrong"] = len(audited), len(wrong)
    m["over"] = (share(sum(e["decision"] == "accepted" for e in wrong), len(wrong))
                 if len(wrong) >= t["min_audited_wrong"] else None)
    m["under"] = share(sum(e["decision"] == "overridden" for e in right), len(right))
    good = (sum(e["decision"] == "accepted" for e in right)
            + sum(e["decision"] == "overridden" for e in wrong))
    m["appropriate"] = share(good, len(audited))

    if m["reach"] is not None and m["reach"] < t["reach_min"]:
        m["flags"].append("LOW REACH")
    if m["coverage"] is not None and m["coverage"] < t["coverage_min"]:
        m["flags"].append("LOW COVERAGE")
    fast = m["review_median"] is not None and m["review_median"] < t["fast_accept_seconds"]
    if (m["over"] is not None and m["over"] > t["over_reliance_max"]) or (
            fast and m["over"] is None):
        m["flags"].append("OVER-TRUST")
    if m["under"] is not None and m["under"] > t["under_reliance_max"]:
        m["flags"].append("UNDER-TRUST")
    if m["minutes_median"] is not None and m["minutes_median"] > TARGET_MINUTES:
        m["flags"].append("SLOW KPI")
    return m


def pct(x):
    return "n/a" if x is None else f"{x * 100:.0f}%"


def num(x, unit=""):
    return "n/a" if x is None else f"{x:.0f}{unit}" if unit == " s" else f"{x:.1f}{unit}"


# ---------- report ----------

def cmd_report(args):
    folder = Path(args.dir)
    THRESHOLDS["min_group"] = args.min_group
    users, volume, events = load(folder)
    if not events:
        print("No events yet: nobody has used the assistant. That is a valid result; check access first.")
        return 0
    last = max(e["week"] for e in events)
    last_weeks = {last - 1, last}
    teams = sorted({u["team"] for u in users})
    results = [team_metrics(t, users, volume, events, last_weeks) for t in teams]

    lines = [f"# Adoption report, weeks {last - 1}-{last}", "",
             f"Baseline {BASELINE_MINUTES:.0f} min per blocked order, target {TARGET_MINUTES:.0f} min. "
             f"Groups under {THRESHOLDS['min_group']} people are not reported. "
             "*Reliance columns use every audited decision since go-live.", "",
             "| Team | Users | Reach | Coverage | Accepted | Review (median) | Over-reliance* "
             "| Under-reliance* | Appropriate* | Minutes (median) | Flags |",
             "| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |"]
    print(f"Adoption report, weeks {last - 1}-{last}  (baseline {BASELINE_MINUTES:.0f} min, "
          f"target {TARGET_MINUTES:.0f} min)\n")
    for m in results:
        if m.get("suppressed"):
            print(f"{m['team']:<16} {m['users']} users: not reported (fewer than "
                  f"{THRESHOLDS['min_group']})")
            lines.append(f"| {m['team']} | {m['users']} | not reported: group too small "
                         "| | | | | | | | |")
            continue
        over = pct(m["over"]) if m["over"] is not None else f"n/a ({m['wrong']} wrong audited)"
        flags = ", ".join(m["flags"]) or "OK"
        print(f"{m['team']:<16} reach {pct(m['reach']):>4}  coverage {pct(m['coverage']):>4}  "
              f"accepted {pct(m['acceptance']):>4}  review {num(m['review_median'], ' s'):>5}  "
              f"over {over:>4}  under {pct(m['under']):>4}  "
              f"minutes {num(m['minutes_median']):>4}  -> {flags}")
        lines.append(f"| {m['team']} | {m['users']} | {pct(m['reach'])} | {pct(m['coverage'])} "
                     f"| {pct(m['acceptance'])} | {num(m['review_median'], ' s')} | {over} "
                     f"| {pct(m['under'])} | {pct(m['appropriate'])} "
                     f"| {num(m['minutes_median'])} | {flags} |")

    lines += ["", "## Actions", ""]
    print("\nActions:")
    any_flag = False
    for m in results:
        for f in m.get("flags", []):
            any_flag = True
            print(f"  {m['team']}: {f}. {ACTIONS[f]}")
            lines.append(f"- **{m['team']}, {f}.** {ACTIONS[f]}")
    if not any_flag:
        print("  None. Keep the weekly review and the audit sample running.")
        lines.append("- None. Keep the weekly review and the audit sample running.")

    weekly = []
    for w in sorted({e["week"] for e in events}):
        active = len({e["user_id"] for e in events if e["week"] == w})
        weekly.append(f"week {w}: {active}/{len(users)} users active")
    print("\nTrend: " + "; ".join(weekly))
    lines += ["", "## Trend", "", *[f"- {w}" for w in weekly], ""]

    out = folder / "adoption_report.md"
    out.write_text("\n".join(lines), encoding="utf-8")
    print(f"\nWrote {out}")
    return 0


def main(argv=None):
    p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    sub = p.add_subparsers(dest="command", required=True)
    s = sub.add_parser("sample", help="write made-up usage logs")
    s.add_argument("--dir", default=str(DEFAULT_DIR))
    s.add_argument("--force", action="store_true", help="overwrite existing files")
    r = sub.add_parser("report", help="print the report and write adoption_report.md")
    r.add_argument("--dir", default=str(DEFAULT_DIR))
    r.add_argument("--min-group", type=int, default=THRESHOLDS["min_group"],
                   help="smallest group size to report (default 5)")
    args = p.parse_args(argv)
    return cmd_sample(args) if args.command == "sample" else cmd_report(args)


if __name__ == "__main__":
    sys.exit(main())

#Step 3: Write the sample logs

1. Run the sample command. It is the same on Windows and macOS/Linux:

   Copy

   python unit14/adoption_report.py sample

2. You should see:

   Copy

   Wrote 24 users, 24 volume rows and 1047 events to unit14/adoption
   Every number is made up. Next: python unit14/adoption_report.py report

3. Open unit14/adoption/events.csv in VS Code. Each row is one decision on one blocked order. Most rows have an empty suggestion_correct: only about a quarter were audited.

The sample has four teams that behave differently on purpose. EMEA checks and decides. Americas accepts nearly everything in seconds. APAC went live two weeks later, and half the team doesn't use it. "Credit Shared" has only three people.

#Step 4: Run the report

1. Run:

   Copy

   python unit14/adoption_report.py report

2. You should see this (the long lines may wrap in your terminal):

   Copy

   Adoption report, weeks 5-6  (baseline 17 min, target 10 min)
   
   Credit APAC      reach  50%  coverage  17%  accepted  56%  review 140 s  over n/a (0 wrong audited)  under  70%  minutes 16.2  -> LOW REACH, LOW COVERAGE, UNDER-TRUST, SLOW KPI
   Credit Americas  reach 100%  coverage  92%  accepted  99%  review   6 s  over  91%  under   0%  minutes  6.0  -> OVER-TRUST
   Credit EMEA      reach  88%  coverage  73%  accepted  85%  review  71 s  over  17%  under  11%  minutes  9.4  -> OK
   Credit Shared    3 users: not reported (fewer than 5)
   
   Actions:
     Credit APAC: LOW REACH. Ask the team lead why people don't open it; check access, roles and training dates.
     Credit APAC: LOW COVERAGE. People use it for some orders only. Shadow two analysts: which orders do they skip, and why?
     Credit APAC: UNDER-TRUST. Users override right suggestions. Share the audit results with the team and ask the champion to walk through cases where the assistant was right.
     Credit APAC: SLOW KPI. Handling time is above target. Look at where the minutes go before changing the model.
     Credit Americas: OVER-TRUST. Users accept wrong suggestions. Show the evidence next to each suggestion, share real misses in the team huddle, and audit more of this team's decisions.
   
   Trend: week 1: 14/24 users active; week 2: 14/24 users active; week 3: 20/24 users active; week 4: 20/24 users active; week 5: 20/24 users active; week 6: 20/24 users active
   
   Wrote unit14/adoption/adoption_report.md

3. Read it as a diagnosis:

   • Americas has the best handling time, 6 minutes. It also accepted 91% of the suggestions the audit found wrong. That is the rubber-stamping team from the foundational layer.
   • APAC shows over n/a (0 wrong audited). There aren't enough audited wrong suggestions to judge over-reliance. That is a valid result: say "not enough data" and audit more, don't guess.
   • Credit Shared isn't reported because it has fewer than five people. With three people, a team number is almost a personal number.

4. Open unit14/adoption/adoption_report.md. VS Code shows a preview with Open Preview to the Side (the icon with a magnifier at the top right of the editor).

#Step 5: Change a threshold and see the effect

The thresholds sit at the top of the script, in THRESHOLDS. Agree on them with the process owner before go-live, not after you see the numbers.

1. In adoption_report.py, change "over_reliance_max": 0.30 to 0.95 and save.
2. Run python unit14/adoption_report.py report again.
3. Credit Americas now shows OK. Nothing changed in the data: the team still accepted 91% of the wrong suggestions. A loose threshold simply hid the problem.
4. Change the value back to 0.30 and save. Run the report once more and check that Credit Americas shows OVER-TRUST again.

#Step 6: Try a smaller privacy limit

1. Run:

   Copy

   python unit14/adoption_report.py report --min-group 3

2. Credit Shared now appears with its own numbers. In a real rollout, agree the minimum group size with HR and employee representatives, and don't lower it to see a small team.

#How the code works

Reach and coverage use the last two weeks, because they change quickly. Reliance pools every audited case since go-live, because wrong suggestions are rare and two weeks hold too few.

#Use your own logs

1. Make a new folder, for example unit14/my_adoption.

2. Export three CSV files with the same column names as the sample: users.csv, volume.csv and events.csv. Use pseudonymous user IDs.

3. Run:

   Copy

   python unit14/adoption_report.py report --dir unit14/my_adoption

#If something goes wrong

#The SAP way

SAP covers the method, in-app help, adoption analytics for SAP's own applications, and process measurement. The decision log and the reliance numbers for your own assistant stay yours. Status as of October 2026.

#Method: OCM in SAP Activate

SAP Learning's lesson on organizational change management maps the work to the six SAP Activate phases:

SAP's Roadmap Viewer holds the detailed OCM deliverables and accelerators for each phase; some accelerators need registration. The lesson also stresses that OCM should be agile, matching the sprints of the project.

#In-app help: Web Assistant and SAP Enable Now

SAP's BTP design guidelines describe the Web Assistant, an add-on framework that adds a help panel with help topics, guided tours and what's-new notes to the application screen. The guidelines point to SAP Enable Now documentation for its technical requirements. Use it for "how to use it" help. It doesn't record whether a user accepted an AI suggestion.

#Adoption analytics: WalkMe

SAP completed its acquisition of WalkMe on 12 September 2024. SAP's page for WalkMe for SAP SuccessFactors HCM describes guidance inside the application, tailored by role, location and language, and behavioral analytics that show where people get stuck. It says the product measures adoption across HR processes and AI capabilities. WalkMe is offered for specific SAP product lines; check the current offer and licensing with your SAP account team. For a custom side-by-side app, confirm with SAP or WalkMe what data you would get before relying on it for reliance metrics.

#Joule rollouts

If the assistant is a Joule capability rather than your own app, SAP Learning's "Driving SAP Joule for Consultants Adoption" journey shows SAP's own campaign pattern for one Joule product: plan, launch, use usage data to refine, and show impact to leadership. The Joule and Joule Studio topic covers what Joule is.

#Process outcomes: SAP Signavio Process Insights

To check that the process improved, not just that people clicked, SAP Signavio Process Insights uses process mining on event logs and reports cycle times, throughput and error rates. For blocked orders, a before/after comparison of the time from block to release is a good cross-check on your handling-minutes KPI.

#Build vs. SAP

#Production concerns

Employee privacy and co-determination. A decision log is data about employees' work. Use pseudonymous IDs, report only groups above a minimum size, restrict who can see the dashboard, and keep the log only as long as needed. In many countries, systems that can monitor employee performance need agreement with a works council or other employee representatives. Involve HR and legal before go-live, not after.

SAP authorizations. The log holds sales order numbers, and the audit view shows customer credit data. Give the auditor and the report reader only the access their SAP roles already allow; the agent permissions topic covers this in detail.

Evaluation. The audit sample is a production evaluation set. Keep the reviewer independent of the analyst being audited. Feed confirmed misses back into your evaluation harness so the next model version is tested on them.

Changes to the model. A new model or prompt can change accuracy overnight and reset users' calibration. Announce changes in the tool before they take effect, and watch the reliance numbers closely for two weeks after.

Cost. Adoption work is mostly people time: champions, huddles, audits and help content. Budget it in the business case. An audit share of a quarter is a lot in month one; reduce it once reliance is stable, but never to zero.

Clean core. Log decisions in the side-by-side app, not by modifying SAP objects. Read volumes through released APIs.

#Pitfalls

• Celebrating acceptance rate. The fastest, most accepting team can be the riskiest. Always read acceptance together with over-reliance.
• Judging over-reliance on too few cases. Two wrong suggestions accepted out of two is not 100% over-reliance. Pool weeks and print "not enough data".
• Reporting individuals. A dashboard that names people becomes a performance tool and gets gamed. Report teams above a minimum size.
• Setting thresholds after seeing the data. You'll set them to make the numbers look fine. Agree them with the process owner first.
• Training once and stopping. Reinforcement is the part that makes a change stick. Keep huddles and shared catches going through the first months.
• Hiding the misses. If users only hear success stories, they over-trust. Share real wrong suggestions and how someone caught them.
• Moving to the next wave on schedule. Rolling out a rubber-stamping pattern to more teams multiplies the risk.

#Exercise

Add a wave gate to the report: a final line saying whether the next rollout wave may start.

1. Copy unit14/adoption_report.py to unit14/adoption_gate.py so the original stays as published.
2. In the copy, find the end of cmd_report, just before the line that writes adoption_report.md.
3. Add code that collects every flag for teams in wave 1 (read the wave column from users.csv; teams that weren't reported don't count).
4. If any wave 1 team has OVER-TRUST or UNDER-TRUST, print Next wave: HOLD followed by the teams and flags. Otherwise print Next wave: GO.
5. Add the same line to the Markdown report under a new ## Wave gate heading.
6. Run python unit14/adoption_gate.py report on the sample.
7. Save your work with Git: git add unit14 and git commit -m "Add adoption wave gate".

Done when: the sample report ends with Next wave: HOLD naming Credit Americas and OVER-TRUST, and the same line appears in adoption_report.md. Keep the script: the capstone in Unit 14 uses it to report adoption for the exception management platform.

#Check yourself