Claim LD05-HQ-C04 (v1.1)

Research opportunities this claim motivates

What this claim leaves open for research. Each is a hypothesis about a gap, not a finding.

• Outside corrections, how often do officials depart from algorithmic recommendations?Allegheny supervisors screened out 21.0% of mandatory screen-ins, a different kind of departure (a different margin and denominator) from Poland's, showing why a common coding scheme is needed.

Provenance and status

Claim type

Fact

Publication status

Eligible for publication, with a qualifier

Verification state

Checked against the primary source. Who checked this claim is not recorded for it; claim checks so far were AI-led, and no person has checked claims against their sources.

Does not license

Overall human accuracy superiority; generalisation to child welfare elsewhere

Limitations

One county; the corrected analysis is a preprint; its authors say every specific number of the original analysis changed when they re-ran it, without reporting the override share again; the study period and population of the 21.0% share are not recorded here

Source (as cited)

Cheng et al., CHI 2022; Stapleton et al., corrected analysis arXiv:2204.13872

Source type

Peer-reviewed conference paper, Preprint (recorded as “Peer-reviewed conference paper + correction preprint”)

Institutional position

Independent academic

Maturity of the evidence

Rapidly maturing evidence

Recorded date

21 October 2026

Currentness / correction

This is the current version. Correction or concern on its sources: Cheng et al., “How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions,” CHI 2022: corrected by Stapleton et al., “Extended Analysis of ‘How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions’,” arXiv:2204.13872.

Sources this version cites

• Cheng et al., “How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions,” CHI 2022 (Peer-reviewed conference paper)
  Read the original Correction or concern: Cheng et al., “How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions,” CHI 2022: corrected by Stapleton et al., “Extended Analysis of ‘How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions’,” arXiv:2204.13872.
• Stapleton et al., “Extended Analysis of ‘How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions’,” arXiv:2204.13872 (Preprint)
  Read the original Correction or concern: Stapleton et al., “Extended Analysis of ‘How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions’,” arXiv:2204.13872: corrects Cheng et al., “How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions,” CHI 2022.

Version

This is the current version, v1.1. Earlier versions stay available, and their records never change.

Versions: v1.0, v1.1 · current

What changed in this version

This version supersedes version v1.0 (LD05-HQ-C04--v1.0), which stays available with its record unchanged.

Changed fields

2 PUBLIC-FACING CLAIM; 14 MAIN RESULT; 18 LIMITATIONS; 25 RECORDED DATE

What changed

The corrected disparity figures are no longer held back. The main result now gives them from the authors’ corrected analysis: for discretionary referrals in 2016–18, the gap between the screen-in rates of Black and white children was 17.9 percentage points for AFST-only (counterfactual) decisions (67.9% against 50.0%) and 7.1 points for decisions by workers using the AFST (50.2% against 43.1%). The public-facing claim now leads with the disparity result that the correction supports, and gives the 21.0% override share second, as a figure of the original analysis whose numbers the authors later said all changed. The limitations add that the corrected analysis is a preprint, that its authors did not report the override share again, and that the study period and population of the 21.0% share are not recorded here. The direction of the claim, its sources, scope and verification state are unchanged.

Why

Version 1.0 held back the exact 17.9→7.1 figure (marked HOLD in the R04 LD-05 Canonical Claim Ledger v1.0), pending direct extraction from the corrected analysis. On 21 October 2026 the full text of Stapleton et al., arXiv:2204.13872v1 (29 April 2022), was read line by line. Appendix A, "Detailed Comparisons of Analyses 1 and 2", first point, Analysis 2 column, reports that AFST-only decisions would have screened in 67.9% of Black and 50.0% of white children in discretionary referrals from 2016-18, "a racial disparity of 17.9%", and that workers using the AFST screened in 50.2% and 43.1%, "a disparity of 7.1%". The paper’s own "%" is a difference between two rates, so this version calls it percentage points. The check read a text extraction of the PDF and looked at its figure pages. The paper’s Figure 1(b) plots the same screen-in rates; it labels the AFST-only gap 17.9% but the worker+AFST gap 7.0%, 0.1 point below the 7.1% of Appendix A. This version follows the appendix text and its rates, whose difference is 7.1 points. Section 2 of the same paper says that "all of the specific numbers we reported in Analysis 1 changed in Analysis 2"; the HQ batch notes take the 21.0% override share from the original CHI 2022 paper, which reports Analysis 1, and the corrected analysis does not report that share again. The CHI 2022 paper itself was not read for this revision (the ACM Digital Library blocks the project’s tools), so the attribution of the 21.0% share to the original analysis rests on the HQ batch notes. The check and this revision were done by AI agents reading the paper, not by a person.