Reviewing 341 academic AI audit studies and non-academic audits, researchers found that only a subset led to accountability outcomes, that many prominent academic audits were collaborations with large technology companies examining their own systems, and that journalistic audits resulted in the most impactful outcomes.

SocioLens project,

Claim LD06-SL-C28 (v1.0)

Provenance and status

Claim type: Estimate and Interpretation
Publication status: Eligible for publication, with a qualifier
Verification state: Checked against the primary source. Who checked this claim is not recorded for it; claim checks so far were AI-led, and no person has checked claims against their sources.
Does not license: A rate of audit impact
Limitations: Pre-2024 audits, mostly not of generative AI
Source (as cited): Birhane, Steed, Ojewale, Vecchione & Raji, “AI auditing: The Broken Bus on the Road to AI Accountability,” IEEE SaTML 2024, arXiv:2401.14462
Source type: Peer-reviewed conference paper (recorded as “Peer-reviewed conference paper”)
Institutional position: Independent academic
Maturity of the evidence: Emerging evidence
Recorded date: 15 October 2026
Currentness / correction: This is the current version. No correction or concern is recorded on its sources.

Sources this version cites

Birhane, Steed, Ojewale, Vecchione & Raji, “AI auditing: The Broken Bus on the Road to AI Accountability,” IEEE SaTML 2024, arXiv:2401.14462 (Peer-reviewed conference paper)
Read the original

Version

This page shows version v1.0. Its record never changes, even when newer evidence arrives; the site may add a note to the page or withhold a figure not cleared for public use.

This pinned version is also the current version.

Versions: v1.0 · current

Raw record: every recorded field, with its coded values

All 25 canonical Claim fields, verbatim, with the coded values derived from them. The plain-language labels above are read from these.

Claim type (coded): ESTIMATE + INTERPRETATION (as printed: “ESTIMATE / INTERPRETATION”)
Publication status (coded): ELIGIBLE (QUALIFIED) (as printed: “ELIGIBLE WITH QUALIFIER”)
Verification state (coded): CONTENT_VERIFIED_PRIMARY
Source type (coded): PEER_REVIEWED_CONFERENCE_PAPER
Maturity (coded): EMERGING

#	Canonical field	Canonical value (verbatim)	Normalized (Claim Value Vocabulary v1.0.2)
1	CLAIM ID	LD06-SL-C28	—
2	PUBLIC-FACING CLAIM	Reviewing 341 academic AI audit studies and non-academic audits, researchers found that only a subset led to accountability outcomes, that many prominent academic audits were collaborations with large technology companies examining their own systems, and that journalistic audits resulted in the most impactful outcomes.	—
3	CLAIM TYPE	ESTIMATE / INTERPRETATION	base_types: ESTIMATE, INTERPRETATION
4	RESEARCH STRAND	C — Evaluation in governance, audits and red-teaming	—
5	MATURITY	EMERGING	states: EMERGING
6	SOURCE	Birhane, Steed, Ojewale, Vecchione & Raji, “AI auditing: The Broken Bus on the Road to AI Accountability,” IEEE SaTML 2024, arXiv:2401.14462	source_ids: SRC-SL-BIRHANE-ET-AL-AI-AUDITING-SATML-2024
7	SOURCE TYPE	Peer-reviewed conference paper	classes: PEER_REVIEWED_CONFERENCE_PAPER
8	INSTITUTIONAL POSITION	Independent academic	—
9	SCOPE	AI audit studies and their outcomes	—
10	DESIGN	Taxonomy and survey of audit studies	—
11	SAMPLE / DATASET	341 academic audit studies plus non-academic audits	—
12	MODEL / TOOL / VERSION	Not applicable	—
13	COMPARATOR	Academic versus journalistic and civil-society audits	—
14	MAIN RESULT	Only a subset of audits translate into accountability; company-collaborative audits of companies’ own systems common; critics warn of audit washing	—
15	INFERENTIAL TARGET	Whether audits hold AI developers to account	—
16	DOES NOT LICENSE	A rate of audit impact	—
17	VERIFICATION STATE	CONTENT VERIFIED	state: CONTENT_VERIFIED_PRIMARY · verified_by: UNKNOWN
18	LIMITATIONS	Pre-2024 audits, mostly not of generative AI	—
19	CONTRADICTORY / SUPPORTING SOURCES	Consistent with LD06-SL-C26	—
20	GEOGRAPHY / LANGUAGE	Not checked / English	—
21	DISTRIBUTIONAL LEDGER	Not analysed in this claim	—
22	R01 / FIELD-MAP LINKS	LD-06, D4b	—
23	WHAT WOULD CHANGE THIS CLAIM?	Outcome tracking of audits under mandated regimes	—
24	PUBLICATION STATUS	ELIGIBLE WITH QUALIFIER	state: ELIGIBLE · qualifiers: QUALIFIED
25	RECORDED DATE	2026-10-15	—

Field 22 references resolved against Field Map v0.2: LD-06 (LIVING_DEBATE); D4b (SUB_AREA)

The sources as recorded, with their record IDs and coded source types:

Pinned Sources (field 6, version-specific)

“Birhane, Steed, Ojewale, Vecchione & Raji, “AI auditing: The Broken Bus on the Road to AI Accountability,” IEEE SaTML 2024, arXiv:2401.14462” → Birhane et al. (2024), “AI auditing: The Broken Bus on…” (SRC-SL-BIRHANE-ET-AL-AI-AUDITING-SATML-2024--v1.0) — Birhane, Steed, Ojewale, Vecchione & Raji, “AI auditing: The Broken Bus on the Road to AI Accountability,” IEEE SaTML 2024, arXiv:2401.14462; class PEER_REVIEWED_CONFERENCE_PAPER (explicit public provenance pin)

Cited by Living Debate versions

The evaluation apparatus: Can benchmarks, leaderboards and audits carry the weight placed on them? (v0.1) LD-06

Cite this page

SocioLens project. (2026, October 15). Reviewing 341 academic AI audit studies and non-academic audits, researchers found that only a subset led to accountability outcomes, that many prominent academic audits were collaborations with large technology companies examining their own systems, and that journalistic audits resulted in the most impactful outcomes [Claim LD06-SL-C28, version v1.0]. SocioLens. https://sociolens-web.pages.dev/claims/ld06-sl-c28/v1.0/

@misc{sociolens_ld06_sl_c28_v1_0,
  author = {{SocioLens project}},
  title = {Reviewing 341 academic AI audit studies and non-academic audits, researchers found that only a subset led to accountability outcomes, that many prominent academic audits were collaborations with large technology companies examining their own systems, and that journalistic audits resulted in the most impactful outcomes.},
  year = {2026},
  date = {2026-10-15},
  howpublished = {\url{https://sociolens-web.pages.dev/claims/ld06-sl-c28/v1.0/}},
  note = {Claim LD06-SL-C28, version v1.0}
}

The link points to this version, whose record never changes. For a finding, cite the original study as well: SocioLens summarizes research and does not replace it.

Research-of-record provenance

Materialized by sociolens-materializer 0.6.0 at 2026-09-28T20:00:00.000Z; normalization contract SocioLens — Claim Value Vocabulary v1.0.2 — Pre-Spike Materialization Contract (v1.0.2). Content seal SHA-256 154444800909f2886e24b109f440ca9b58ebf3fc92b47eccc215c259cc8e38de.

Repository-authored inputs (not Drive exports)

SocioLens claim — LD06-SL-C28 v1.0 (v1.0) — written in the SocioLens repository by Project Lead (Claude, AI) on 2026-10-15; no Drive identity exists. Authority: AGENTS.md "Authority" (the Project Lead decides research content within the research-integrity rules) and the Owner's choice in the SocioLens project chat on 15 October 2026 to build the LD-06 debate; a new Claim, first version, written from the project's LD-06 source notes, checked against full texts or official abstracts. Block “ENTRY 28 — LD06-SL-C28” SHA-256 659fb0bc637c9069300d1a0b7d03c8d22d286257072fc7cd6db1ac6905a494d6; fixture fixtures/authored/claims/LD06-SL-C28--v1.0.

Builder decisions (not Drive data)

VERSION = v1.0 — basis: Claim version taken from the repository-authored claim fixture version (SocioLens claim — LD06-SL-C28 v1.0); the first version of a new Claim of the LD-06 debate
CURRENT_POINTER = true — basis: repository-authored claim "SocioLens claim — LD06-SL-C28 v1.0" (Project Lead (Claude, AI), 2026-10-15); basis: AGENTS.md "Authority" (the Project Lead decides research content within the research-integrity rules) and the Owner's choice in the SocioLens project chat on 15 October 2026 to build the LD-06 debate; a new Claim, first version, written from the project's LD-06 source notes, checked against full texts or official abstracts
FIELD_6_SOURCE_REFERENCE = SRC-SL-BIRHANE-ET-AL-AI-AUDITING-SATML-2024--v1.0 — basis: field 6 of ENTRY 28 — LD06-SL-C28 cites "Birhane, Steed, Ojewale, Vecchione & Raji, “AI auditing: The Broken Bus on the Road to AI Accountability,” IEEE SaTML 2024, arXiv:2401.14462"; the only field-6 segment of this ClaimVersion's repository-authored ledger block.
SOURCE_PROVENANCE_PIN = SRC-SL-BIRHANE-ET-AL-AI-AUDITING-SATML-2024--v1.0 — basis: explicit version-pinned public provenance of this ClaimVersion (Amendment v1.2 §6): the pinned field-6 Source record "Birhane, Steed, Ojewale, Vecchione & Raji, “AI auditing: The Broken Bus on the Road to AI Accountability,” IEEE SaTML 2024, arXiv:2401.14462".