Traceability, Not Accuracy
Accuracy is a vendor's claim about the past. Traceability is what you can check now: the evidence, the failure modes, and a five-minute pass for clinicians.
Respocare Connect AI Team — Respocare (PTY) Ltd · Reg. 2018/411829/07 · Practice No. 9990900010775614. Licensed healthcare practice operating since 2018.
· 12 min read
The right question about clinical AI isn't "is it accurate?" — it's "can you trace it?"
An evidence review and a working protocol for clinicians using AI to document and summarise care.
The argument in five lines
- 81% of clinicians now use AI, overwhelmingly to document and summarise — not to decide.
- "Accuracy" is the wrong test: it is unmeasurable at the point of care and generalises poorly across specialties.
- The answerable test is traceability — can every clinical statement be traced to something that actually happened.
- The known failure modes are fluent, not obvious; each has a specific catch, mapped below.
- The control is unchanged by any model: the AI drafts, the record verifies, the clinician decides.
Start with the wrong question
Ask whether a clinical AI is "accurate" and you have asked a question no one can answer for you. Accuracy is a property of an output measured against a ground truth you often do not have in front of you, across a distribution of cases no vendor has tested on your patients. It is real, it matters, and it is the wrong question to bring into the consulting room.
The right question is narrower, harder to dodge, and answerable in seconds: can every clinical statement this system produces be traced back to something that actually happened? Call it traceability. Unlike accuracy, you can check it. And it is the property that separates a draft you can safely sign from one you cannot.
This is a review of what the evidence shows AI does well in documentation and summarisation, the failure modes that survive every model upgrade, and a protocol built around traceability that takes about five minutes and catches the errors that cause harm.
What the evidence actually shows — read it honestly
The benefits are real, measured, and uneven. A clinician deciding whether to adopt deserves the effect sizes and the caveats in the same view, not the press release.
| Finding | Scale | What it showed | The caveat that matters |
|---|---|---|---|
| Professional AI use | ~1,700 physicians (AMA, 2026) | 81% use AI; top uses are summarising research (39%) and documentation (28–30%); assistive diagnosis lowest (17%) | Adoption is not endorsement of quality — it tracks administrative pain, not proven benefit |
| Documentation time | 46 clinicians (JAMA Netw Open, 2025) | 20.4% less time in notes per appointment; 30.0% less after-hours work | Notes were 20.6% longer; clinician accuracy ratings diverged sharply |
| Burnout | 263 clinicians, six systems (JAMA Netw Open, 2025) | Burnout fell 51.9% → 38.8% after 30 days | Category research on the technology, not any one product; self-reported |
| Time (RCT) | 238 physicians, two tools (NEJM AI, 2025) | One tool −9.5%, the other no significant change | Same task, same trial — benefit did not generalise across tools |
| Time (cohort) | Users vs balanced controls (JAMA Netw Open, 2025) | Less time in notes and in the record overall | No significant reduction in after-hours work for this group |
| Summarisation quality | 10 physicians (Nature Medicine, 2024) | Best model summaries judged equivalent (45%) or superior (36%) to experts | Same study's safety analysis found fabrication and potential-harm errors |
| Reasoning support | 92 clinicians, RCT (Nature Medicine, 2025) | GPT-4 + resources scored +6.5pp on management reasoning (95% CI 2.7–10.2) | Simulated cases; better performers spent longer, not less |
What the evidence does not show
No study here demonstrates improved patient outcomes — the endpoints are time, burnout, reasoning scores and note-quality proxies, not morbidity or mortality. None shows the benefit generalises across specialties. Several show it does not. And more than one shows AI can add review time even as it removes writing time. The honest summary: strong signal on administrative relief, no signal yet on whether patients are better off. Treat anything stronger as marketing.
The failure modes are fluent, not obvious
This is the part that does not improve with the next model, because these are properties of how the systems work — not bugs to be patched. Each is dangerous precisely because it reads well. The value of naming them is that each has a specific catch.
| Failure mode | What it looks like | Why fluency hides it | The catch |
|---|---|---|---|
| Fabrication | A plausible finding, medication or history that was never in the source | Confident, well-formed prose reads as true on a quick pass | Trace it — no source, no signature |
| Omission | An allergy, an abnormal result or a stopped drug silently absent | Nothing on the page signals a gap; the note looks complete | Read the source for what is missing, not just the draft for what is wrong |
| Negation / polarity | "No chest pain" rendered as "chest pain"; a positive dropped | One word inverts the meaning; the sentence still parses cleanly | Check every clinically material negation against what was said |
| Numeric / unit error | 15 mg as 50 mg; a value, date or side transposed | Numbers look authoritative and are rarely re-read | Verify doses, values, dates and laterality digit by digit |
| Temporal / attribution drift | A discontinued drug carried forward; the wrong clinician's plan attributed | The record's timeline is flattened into fluent narrative | Confirm currency — is this true now, and whose decision was it |
| Dilution (note bloat) | A longer, tidier note that buries the one line that matters | Length reads as thoroughness | Ask what a covering clinician would miss at 2 a.m. |
| Automation bias (the human failure) | Signing a fluent draft with less scrutiny than a colleague's note | The better the prose, the weaker the review | Give the AI draft more scrutiny than a human one, not less |
The pattern across the table is the same: the errors are invisible on the draft and visible against the source. That is the whole case for traceability.
Why traceability is the control
The clinician remains the author of record. Whatever wrote the first draft, the signature — and the accountability — is human, and no model changes that. Traceability is simply the operational form of that responsibility: a statement you cannot trace is a statement you cannot stand behind.
It also reframes what to demand of a system. Accuracy is a claim a vendor makes about the past. Traceability is a property you verify in the present — and it is designable. A system built to ground each statement in the patient's own record, to show its source, and to flag or decline when it cannot, is one you can supervise. A system that produces fluent text with no thread back to the evidence is one you cannot, however impressive its benchmarks. Where a system is unsure, the safe behaviour is to surface the uncertainty rather than resolve it. Refusal, in a clinical tool, is not a limitation. It is the highest expression of safety.
The Traceability Pass — a five-minute protocol
Treat every AI draft the way you would treat a capable junior who does not yet know your patient: useful, worth reading, never signed unread. The pass is three questions applied to every clinically material statement, with a high-harm field check underneath.
For each material statement, ask:
| # | Question | You are testing for |
|---|---|---|
| 1 | Sourced — can I trace this to what actually happened | Fabrication, attribution drift |
| 2 | Faithful — is the detail exactly right | Negation, numeric, unit and laterality errors |
| 3 | Complete — is anything clinically relevant missing | Omission, dilution |
High-harm field check — never skip these:
| Field | Verify |
|---|---|
| Allergies | Present, complete, correctly specified |
| Medications | Names, doses, frequencies; nothing discontinued carried forward |
| Numbers | Vitals, labs, dates, paediatric and gestational figures |
| Negations | No positive dropped, no negative inverted |
| Laterality / site | Left/right and anatomical location |
| Plan and follow-up | Accurate, safe, and what was actually decided |
The rule that ends it: if a statement cannot be traced to what actually happened, it does not belong in the record under your name.
Match the tool to the task
Not every use carries the same risk. A simple ladder tells you where to start and how much to verify.
| Rung | Task | Consequence if wrong | Grounding required | Safe to start here |
|---|---|---|---|---|
| 1 | Document what just happened | Low — you were in the room | You are the ground truth | Yes — begin here |
| 2 | Summarise the existing record | Moderate — omission risk | Source must be checkable | With the pass above |
| 3 | Retrieve relevant evidence | Higher — wrong evidence misleads | Citations to real sources | Only if sources are shown |
| 4 | Reason through the case | Highest — shapes decisions | Full transparency of logic | Last, and never unsupervised |
Consequence rises at every rung, and with it the grounding you should demand. Start at the bottom, where you can supervise the output effortlessly, and let trust extend upward deliberately — never faster than your ability to verify.
Governance beyond the individual
Individual discipline is necessary but not sufficient. At the level of a practice or institution, a defensible deployment attends to patient consent and data protection appropriate to the jurisdiction (POPIA in South Africa, and its equivalents elsewhere); a clear line around where AI output may and may not be used unsupervised; an audit trail; and an escalation path when the tool produces something wrong. This is not exotic. It is the ordinary governance any clinical tool warrants, applied to one that happens to write fluently.
The bottom line
The machines are getting better at writing notes. That was never the hard part. The hard part — the part that keeps a patient safe — is knowing which sentence to trust and being able to prove it.
Accuracy is the vendor's claim about the past. Traceability is yours to verify in the present. The AI drafts, the record verifies, the clinician decides — and you sign only what you can trace.
References
- American Medical Association. 2026 Physician Survey on Augmented Intelligence. March 2026 (n = 1,692; fielded 15 January – 2 February 2026).
- Van Veen D, Van Uden C, Blankemeier L, et al. Adapted large language models can outperform medical experts in clinical text summarization. Nature Medicine. 2024;30:1134–1142. doi:10.1038/s41591-024-02855-5
- Duggan MJ, Gervase J, Schoenbaum A, et al. Clinician Experiences With Ambient Scribe Technology to Assist With Documentation Burden and Efficiency. JAMA Network Open. 2025;8(2):e2460637. doi:10.1001/jamanetworkopen.2024.60637
- Olson KD, Meeker D, Troup M, et al. Use of Ambient AI Scribes to Reduce Administrative Burden and Professional Burnout. JAMA Network Open. 2025;8(10):e2534976. doi:10.1001/jamanetworkopen.2025.34976
- Goh E, Gallo RJ, Strong E, et al. GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial. Nature Medicine. 2025;31:1233–1238. doi:10.1038/s41591-024-03456-y
- Lukac PJ, Turner W, Vangala S, et al. Ambient AI Scribes in Clinical Practice: A Randomized Trial. NEJM AI. 2025;2(12). doi:10.1056/AIoa2501000
- Pearlman K, et al. Use of an AI Scribe and Electronic Health Record Efficiency. JAMA Network Open. 2025;8(10):e2537000. doi:10.1001/jamanetworkopen.2025.37000
Frequently asked questions
Is clinical AI accurate enough to use for documentation?
Accuracy is difficult to answer at the point of care: it is measured against a ground truth you often do not have in front of you, and it generalises poorly across specialties and tools. The answerable question is traceability — whether every clinical statement can be traced back to something that actually happened. A statement that cannot be traced should not be signed.
What are the failure modes of AI clinical documentation?
Fabrication, omission, negation or polarity errors, numeric and unit errors, temporal or attribution drift, dilution through note bloat, and automation bias in the reviewing clinician. Each is dangerous because it reads fluently: the errors are invisible on the draft and visible against the source.
Does AI documentation save clinicians time?
The evidence shows real but uneven administrative relief. One study of 46 clinicians found 20.4% less time in notes per appointment and 30.0% less after-hours work, though notes were 20.6% longer. A randomised trial of 238 physicians found one tool reduced time in notes by 9.5% while a second showed no significant change. No study to date demonstrates improved patient outcomes.
What is the Traceability Pass?
A five-minute protocol applied to every clinically material statement in an AI draft: is it sourced, is it faithful in detail, and is it complete. It is followed by a high-harm field check of allergies, medications, numbers, negations, laterality and the plan. The rule that ends it: if a statement cannot be traced to what actually happened, it does not belong in the record under your name.
Who is accountable for an AI-generated clinical note?
The clinician who signs it. Whatever produced the first draft, the signature and the accountability are human, and no model changes that. Traceability is the operational form of that responsibility: a statement you cannot trace is a statement you cannot stand behind.
Built by Respocare (PTY) Ltd — a licensed South African healthcare practice operating since 2018.