How many AI transcription errors survive social workers' checks into case records?

Research question

In local authorities using generative-AI transcription and summarisation in social work, what share of final, signed case records contain errors (invented content, omissions, changed meaning) that are absent from the recorded conversation, which kinds survive the practitioner's check, and what happens when people who draw on care read and try to correct them?

Why it matters

Social care records can inform safeguarding decisions, assessments and legal processes, yet the people they describe have not been asked about AI-generated records in the studies found. Current evidence on AI-generated records in social care is interview-based: practitioners report hallucinations and omissions, but no study found has counted how many survive into the final record. A recent audit of clinical AI scribes found that the measured error rate was substantial and depended heavily on how errors were counted, so social care cannot borrow medicine's numbers.

The debate lists this among what we still do not know: How frequently do uncorrected AI hallucinations enter public-sector case files?

Evidence that motivates it

1. Qualitative evidence from UK social care shows that generative-AI transcription and summarisation can introduce hallucinations, omissions and changes in wording into official records, shifting checking and accountability work onto social workers. (LD05-HQ-C16 v1.0)
   Qualitative evidence from UK social care shows AI transcription can introduce hallucinations, omissions and wording changes and shifts checking work onto social workers; the claim explicitly does not license a prevalence figure or a citizen-outcome effect, which is exactly what this study would measure.
   Cites: Bruff & Groves. “Scribe and prejudice?” Ada Lovelace…
2. For AI-mediated administrative processes, effective contestability depends partly on process provenance—knowing whether a system selected the case, influenced routing, or generated part of the evidentiary record. (LD05-HQ-C17 v1.0)
   Contestability depends partly on knowing whether a system generated part of the evidentiary record; the study's citizen-review component tests whether people can spot and correct AI-generated content in their own records.
   Cites: DWP; Home Office; Ada Lovelace; Robodebt synthesis

Claims are shown as they read on the day this pack was built; claim pages keep every earlier version.

A possible design

Partner with two or three local authorities already using AI transcription. With informed consent from practitioners and the people they visit, sample recorded meetings (for example 150 to 300 across adult and children's services, excluding settings where recording would be unsafe). For each meeting, compare three texts: the recording (human-transcribed as the reference), the AI draft, and the final signed record. Two trained coders, blind to the source of each text where possible, classify discrepancies by type (invented statement, omission, attribution error, changed tone or meaning) and by consequence (could it affect a risk assessment, eligibility or a legal process). Report the share of drafts and of final records with at least one consequential error, using a pre-registered error definition and showing how the rate moves under stricter and looser definitions. A smaller follow-up offers a subsample of people who draw on care the chance to review their record with an advocate, recording what they find and what happens when they ask for corrections.

Data needed

Local-authority partnership and data-sharing agreement; consented recordings; access to AI drafts and final records; coder time; advocacy support for the citizen-review component; high-level ethics and safeguarding approval.

Feasibility

A PhD project. Medium: human transcription and double coding of a few hundred meetings, plus advocate time; feasible within a funded studentship with a small project grant. Access and ethics are the main obstacles; recording vulnerable people requires strong safeguards and some meetings must be excluded, which biases the sample toward less sensitive cases and should be reported. Councils may be reluctant to publish error rates.

Has this been done?

Partly done already: related work covers some of this question (first-pass screen of 12 October 2026, checked again on 13 October 2026).

Social care work so far is qualitative or scoping: Ada Lovelace Institute interviews (no technical evaluation, no interviews with people who draw on care), a Department for Education scoping report with no accuracy figures, and a vendor-commissioned Alan Turing Institute evaluation of Magic Notes in three councils that reports practitioners' accounts of errors but no accuracy measurement. The Ada report mentions vendor evaluations for each deploying council and internal council quality checks, none published with methods or accuracy figures. An analogous audit design exists in healthcare, where three AI scribes were audited on 142 consultations, apparently on AI output rather than records after practitioner checking. No study was found that measures errors in final signed social care records or what happens when record subjects try to correct them.

Literature check. No study was found that compares recordings, AI drafts and final signed social care records, or that follows people who draw on care trying to correct them. The closest new evidence is a Beam-commissioned Alan Turing Institute evaluation of Magic Notes in three English councils (91 staff, April to September 2024), reported by Community Care as qualitative: practitioners described misspellings, repetitions and assumptions, with no quantitative accuracy figures and no comparison against recordings or final records. Healthcare audits of AI scribes remain the only quantitative analogue, and the Ada Lovelace Institute report confirms that council and vendor evaluations are unpublished. Checked by Claude (SocioLens Project Lead) with AI research agents, by web search; not a person and not a domain expert.

Where the search could not reach. Screened by web search on the date shown; vendor-commissioned evaluations and unpublished council pilots are unlikely to be indexed, and Google Scholar, SSRN and paywalled journals were not systematically searched. The clinical scribe literature was sampled, not reviewed. Absence of a hit is not proof that the work does not exist. Literature check of 2026-10-13: The Turing evaluation report itself was not located; it was read only through Community Care. The Social Work Education and Journal of Evidence-Based Social Work articles, and the Mayo Clinic Proceedings: Digital Health simulated-scribe study, could not be fetched. The German-language query found only practice guides. Swedish-language and council committee papers (for example the Havering and Redcar & Cleveland scrutiny presentations) were not read. The final part of the Ada report page was not read. Absence of a hit is not proof that the work does not exist.

Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.

Nearest work found

• Bruff & Groves, "Scribe and prejudice? Exploring the use of AI transcription tools in social care," Ada Lovelace Institute (11 Feb 2026)
  39 interviews, including 29 social workers in 17 English and Scottish local authorities. Reports practitioner accounts of hallucinations and of carrying responsibility for AI records; states it did no technical evaluation and did not interview people who draw on care. About the first three-quarters of the long web page was read.
• Rothera (Research in Practice), "Artificial Intelligence (AI) in case recording - Reducing unnecessary social worker workload through the use of AI in case recording," supplementary report for the National Workload Action Group, Department for Education (Sept 2025)
  Scoping exercise (desk review, 18 stakeholder interviews, two practitioner focus groups). Notes that AI output can be inaccurate or lack nuance and must be checked, which takes time, but reports no measured accuracy or outcomes for families, and children and families were not directly engaged.
• Fox, Markham, Lail & Karotsieris, "One note in three - a verified census of three deployed AI scribes, and the instrument that counted it," arXiv:2608.31017 (31 Aug 2026)
  Audits three commercial clinical AI scribes on 142 consultations (recorded UK primary-care and US ambulatory encounters plus authored scenarios; 565 notes); 31.3% of notes carry a verified failure, and the rate depends strongly on the review instruction and standard applied. Healthcare; it appears to audit AI drafts only, with no practitioner checking stage (an inference from the abstract, which notes that a clinician signs every note) and no social care.
• Procter & Zhydkov (Alan Turing Institute), evaluation of Magic Notes commissioned by Beam, as reported by Community Care, "AI tool improves direct work in adult social care despite accuracy concerns, practitioners report" (date of article not confirmed; trial April to September 2024) Some details not confirmed
  Article read: 91 staff in Swindon, Barnet and Camden councils; usage analysis, staff ratings (mean 4.26 of 5), 11 interviews and Swindon timing data. Accuracy evidence is qualitative only (name misspellings, repetitions, assumptions; editing sometimes time-consuming); no comparison of AI output with recordings or final records. The underlying report itself was not found or read. This is the vendor-commissioned third-party evaluation the Ada report mentions, most likely (our inference).
• "A kind of magic? Artificial intelligence-assisted recording: developing principles for an AI-informed social work curriculum," Social Work Education (2026), doi:10.1080/02615479.2026.2669634 Judged from an abstract, summary or search listing only Some details not confirmed
  Seen in search results only (publisher page could not be fetched; authors not confirmed). Title indicates curriculum principles for AI-assisted recording, not an accuracy audit.
• "Unintended Ramifications of AI-Assisted Documentation: Navigating Pragmatic & Ethical Clinical Social Work Workload Challenges," Journal of Evidence-Based Social Work (Taylor & Francis, 2025), doi:10.1080/26408066.2025.2571439 Judged from an abstract, summary or search listing only Some details not confirmed
  Seen in search results only (not opened; authors and journal name inferred from the DOI prefix and not confirmed). Title indicates a discussion of workload and ethical issues of AI documentation in clinical social work, not an error audit of signed records.

References

Sources behind the motivating claims:

• Bruff & Groves. “Scribe and prejudice?” Ada Lovelace Institute, 11 February 2026 (SRC-LD05-BRUFF-GROVES-ADA-2026, original)
• DWP; Home Office; Ada Lovelace; Robodebt synthesis (SRC-R04-HQ-CONTESTABILITY-PROVENANCE-SYNTHESIS, no checked link to the original yet)

BibTeX

The first entry cites this opportunity; then its nearest work; then the sources above. Nearest work and sources are recorded as reference strings, so each goes in whole as the title; add authors and year from the original before you cite it.

@misc{sociolens_ro_ld05_04_v2,
  author = {{SocioLens project}},
  title = {How many AI transcription errors survive social workers' checks into case records?},
  year = {2026},
  date = {2026-10-13},
  howpublished = {\url{https://sociolens-web.pages.dev/opportunities/ro-ld05-04/v2/}},
  note = {Research opportunity RO-LD05-04, version v2}
}

@misc{ro_ld05_04_nearest_1,
  title = {Bruff \& Groves, "Scribe and prejudice? Exploring the use of AI transcription tools in social care," Ada Lovelace Institute (11 Feb 2026)},
  url = {https://www.adalovelaceinstitute.org/report/scribe-and-prejudice/},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld05_04_nearest_2,
  title = {Rothera (Research in Practice), "Artificial Intelligence (AI) in case recording - Reducing unnecessary social worker workload through the use of AI in case recording," supplementary report for the National Workload Action Group, Department for Education (Sept 2025)},
  url = {https://assets.publishing.service.gov.uk/media/68da6c86c487360cc70c9e95/Artificial_intelligence_in_case_recording_-_national_workload_action_group_supplementary_report.pdf},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld05_04_nearest_3,
  title = {Fox, Markham, Lail \& Karotsieris, "One note in three - a verified census of three deployed AI scribes, and the instrument that counted it," arXiv:2608.31017 (31 Aug 2026)},
  url = {https://arxiv.org/abs/2608.31017},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld05_04_nearest_4,
  title = {Procter \& Zhydkov (Alan Turing Institute), evaluation of Magic Notes commissioned by Beam, as reported by Community Care, "AI tool improves direct work in adult social care despite accuracy concerns, practitioners report" (date of article not confirmed; trial April to September 2024)},
  url = {https://www.communitycare.co.uk/?p=215389},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Some details not confirmed}
}

@misc{ro_ld05_04_nearest_5,
  title = {"A kind of magic? Artificial intelligence-assisted recording: developing principles for an AI-informed social work curriculum," Social Work Education (2026), doi:10.1080/02615479.2026.2669634},
  url = {https://doi.org/10.1080/02615479.2026.2669634},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only; Some details not confirmed}
}

@misc{ro_ld05_04_nearest_6,
  title = {"Unintended Ramifications of AI-Assisted Documentation: Navigating Pragmatic \& Ethical Clinical Social Work Workload Challenges," Journal of Evidence-Based Social Work (Taylor \& Francis, 2025), doi:10.1080/26408066.2025.2571439},
  url = {https://doi.org/10.1080/26408066.2025.2571439},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only; Some details not confirmed}
}

@misc{src_ld05_bruff_groves_ada_2026,
  title = {Bruff \& Groves. “Scribe and prejudice?” Ada Lovelace Institute, 11 February 2026},
  url = {https://www.adalovelaceinstitute.org/report/scribe-and-prejudice/},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_r04_hq_contestability_provenance_synthesis,
  title = {DWP; Home Office; Ada Lovelace; Robodebt synthesis},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}