Preview: not announced. Search engines asked not to index.

Proposal pack · RO-LD10-03 v2

Are AI search answers less faithful to their cited sources outside US English?

To keep this as a PDF or print it, use your browser's Print command (Ctrl+P, or Cmd+P on a Mac) and choose "Save as PDF". Links print with their addresses. This pack gathers what the opportunity page already holds; nothing in it is new.

Read this first. This is a research hypothesis drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-10 debate on 12 October 2026, not a finding. Checked by Claude (SocioLens Project Lead) with AI research agents, by web search; not a person and not a domain expert; an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Every check so far was done by AI; no person or domain expert has checked it yet. The literature screen below was done by AI web search and can miss relevant work. Version v2, permanent address https://sociolens-web.pages.dev/opportunities/ro-ld10-03/v2/.

Geographic or language coverage Master's thesis Low to medium cost Partly addressed

Research question

For matched public-interest queries (news, health, elections) issued in several languages and country markets, how often are claims in Google AI Overviews and AI Mode unsupported by the pages they cite, judged claim by claim with native-speaker validation, and how do these rates differ from a US English baseline run through the same pipeline?

Why it matters

Citation fidelity is one of the few directly measurable qualities of AI intermediation, and it matters because citations raise trust even when they are wrong. The only large claim-level audit of AI Overviews is US-localized, and multilingual audits of chatbots check sourcing by journalist judgement rather than claim by claim. If fidelity is worse in smaller languages, the users with the fewest alternative sources may get the least reliable intermediation.

The debate lists this among what we still do not know: stronger non-English empirical coverage

Evidence that motivates it

  1. A 2026 Google AI Overviews audit found 13.7% overall activation, 64.7% on question-form queries, nearly 30% of cited domains absent from the co-displayed first results page, and 11.0% of atomic claims unsupported by cited pages. (LD10-HQ-C15 v1.0)
    Reports 11.0% of atomic claims unsupported by cited pages in a 2026 AI Overviews audit; a cross-market replication is named as what would change it, and it does not license claims about user beliefs.
    Cites: Xu, Iqbal & Montgomery 2026, arXiv:2605.14021
  2. A 2025 EBU/BBC-led audit across 18 countries and 14 languages found significant issues in 45% of AI-assistant news answers, serious sourcing problems in 31% and major accuracy issues in 20%. (LD10-HQ-C17 v1.0)
    A multilingual audit of AI assistants (18 countries, 14 languages) found serious sourcing problems in 31% of news answers, but covered chatbots rather than answers built into search and does not give a platform-wide error rate.
    Cites: EBU/BBC, News Integrity in AI Assistants, Oct 2025
  3. In a preregistered U.S. experiment, citations increased trust in AI-search answers even when incorrect or hallucinated, while uncertainty highlighting reduced trust and sharing. (LD10-HQ-C20 v1.0)
    Citations increased trust in AI-search answers even when incorrect, which is why unsupported citations matter; the claim does not establish real-world reliance levels.
    Cites: Li & Aral 2025, arXiv:2504.06435

Claims are shown as they read on the day this pack was built; claim pages keep every earlier version.

A possible design

Build a matched query set (about 500 queries per language) from trending searches and fixed public-interest topics, translated and localized by native speakers, in four to six languages spanning high- and low-resource tiers (for example English, German, Spanish, Polish, Hindi, Swahili). Collect AI Overviews and AI Mode answers with their cited pages from in-country vantage points (residential proxies or volunteers) over four to eight weeks. Extract atomic claims and judge support against cited pages with an LLM pipeline validated by native-speaker annotators on a stratified sample in every language, reporting per-language agreement. The primary outcome is the unsupported-claim rate. Activation rates, the share of local-language and local-country sources (compared with the recent health audit by Zha and Chang rather than presented as new) and overlap with the co-displayed results are secondary. Compare them using the same pipeline on a US English arm as the benchmark.

Data needed

In-country collection points; native-speaker annotators (paid) for translation and validation; page archiving for cited sources; an LLM or natural-language-inference model for claim checking.

Feasibility

Master's thesis. Low to medium: proxies and model calls a few hundred to a few thousand US dollars; annotator pay is the main cost (roughly US$2,000 to 6,000 for two to three languages). A two-language version suits a master's thesis. Automated collection may conflict with platform terms; check the legal position and prefer volunteer or research-access routes. Claim-checking models are weaker in low-resource languages, so human validation per language is essential, not optional.

Has this been done?

Partly done already: related work covers some of this question (first-pass screen of 12 October 2026, checked again on 13 October 2026).

Claim-level checks of AI answers in search against their cited pages exist only for English queries from US locations (Xu et al.; Huang et al.; a journalistic Oumi analysis for the New York Times), and only Xu et al. report any human validation. Cross-country and cross-language audits now document where AI Overviews draw their sources (Zha and Chang for health in 12 country settings and 4 languages; Li et al. for Baidu and Google in English and Chinese; Zhen et al. for Chinese platforms; Weinbrand for Australian political search) but none checks whether claims are supported by the cited pages, and none covers AI Mode. A human-validated, claim-level fidelity comparison of AI answers in search across languages and markets was not found.

Literature check. The local-language and local-country sourcing part of the question is now partly answered: Zha and Chang audit Google and Baidu AI Overviews for health queries in 12 country settings and 4 languages and find that native-language queries raise locally sourced citations 3.5 to 13.5 times, Li et al. compare Baidu and Google overview source exposure across English and Chinese, and Zhen et al. map citations in Chinese generative search. None of these checks whether claims are supported by the cited pages (Zha and Chang name claim-level checking as future work), and the only grounding checks found (Xu et al.; Huang et al.; the Oumi analysis for the New York Times) are English and US-located. The opportunity should be narrowed to claim-level, human-validated fidelity across languages and markets, with source localization as a secondary measure. Checked by Claude (SocioLens Project Lead) with AI research agents, by web search; not a person and not a domain expert.

Where the search could not reach. Screened by web search in English on the date shown; searches in other languages were not run, so audits published only in those languages (for example by national regulators or newsrooms) may have been missed. Google Scholar and paywalled journals were not searched. Absence of a hit is not proof that the work does not exist. Literature check of 2026-10-13: ScienceDirect, SSRN and the New York Times could not be opened; the Oumi analysis was read through a trade-press summary. German and Chinese searches were run, but audits published only in other languages (for example by national regulators) may have been missed. The web-search budget for this session ran out during later checks. Absence of a hit is not proof that the work does not exist.

Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.

Nearest work found

References

Sources behind the motivating claims:

BibTeX

The first entry cites this opportunity; then its nearest work; then the sources above. Nearest work and sources are recorded as reference strings, so each goes in whole as the title; add authors and year from the original before you cite it.

@misc{sociolens_ro_ld10_03_v2,
  author = {{SocioLens project}},
  title = {Are AI search answers less faithful to their cited sources outside US English?},
  year = {2026},
  date = {2026-10-13},
  howpublished = {\url{https://sociolens-web.pages.dev/opportunities/ro-ld10-03/v2/}},
  note = {Research opportunity RO-LD10-03, version v2}
}

@misc{ro_ld10_03_nearest_1,
  title = {Xu, Iqbal \& Montgomery, "Measuring Google AI Overviews: Activation, Source Quality, Claim Fidelity, and Publisher Impact," arXiv:2605.14021 (13 May 2026)},
  url = {https://arxiv.org/abs/2605.14021},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld10_03_nearest_2,
  title = {Aral, Li \& Zuo, "The Rise of AI Search: Implications for Information Markets and Human Judgement at Scale," arXiv:2602.13415 (13 Feb 2026)},
  url = {https://arxiv.org/abs/2602.13415},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld10_03_nearest_3,
  title = {Nguyen, Noorily, Flathers, Notsu, Ospina-Pinillos, Nguyen, Clark, Keane, Thompson \& Torous, "Sources of Truth: A Multi-Platform, Multilingual Audit of Citations in AI Mental Health Information Queries," arXiv:2609.00319 (31 Aug 2026)},
  url = {https://arxiv.org/abs/2609.00319},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld10_03_nearest_4,
  title = {European Broadcasting Union \& BBC, "News Integrity in AI Assistants: An International PSM Study" (EBU, 21 Oct 2025)},
  url = {https://www.ebu.ch/research/open/report/news-integrity-in-ai-assistants},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld10_03_nearest_5,
  title = {Zha \& Chang, "Who Anchors AI Overviews in Health? Baidu, Google, and the Geography of Authority," arXiv:2609.06798 (6 Sep 2026)},
  url = {https://arxiv.org/abs/2609.06798},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld10_03_nearest_6,
  title = {Li, Guan, Cai, Liu \& Pierri, "Auditing Source Exposure in Baidu and Google AI Search," arXiv:2609.24407 (21 Sep 2026)},
  url = {https://arxiv.org/abs/2609.24407},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld10_03_nearest_7,
  title = {Zhen, Liu, Zhang \& Niu, "What Do Chinese-Language Generative Search Engines Cite and Surface? A Large-Scale Empirical Study," arXiv:2607.15771 (17 Jul 2026)},
  url = {https://arxiv.org/abs/2607.15771},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld10_03_nearest_8,
  title = {Huang, Goyal, Saha \& Chandrasekharan, "Answer Bubbles: Information Exposure in AI-Mediated Search," arXiv:2603.16138 (v2 28 Aug 2026)},
  url = {https://arxiv.org/abs/2603.16138},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld10_03_nearest_9,
  title = {Oumi analysis commissioned by The New York Times, reported in "Google AI Overviews: etwa 91 Prozent der Antworten sind richtig," SEO Südwest (9 Apr 2026), citing a New York Times article of 7 Apr 2026},
  url = {https://www.seo-suedwest.de/10726-google-ai-overviews-etwa-91-prozent-der-antworten-sind-richtig.html},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld10_03_nearest_10,
  title = {Suzgun, Shen, Bianchi, Spangher, Icard, Ho, Jurafsky \& Zou, "Evaluating Commercial AI Chatbots as News Intermediaries," arXiv:2605.22785 (21 May 2026)},
  url = {https://arxiv.org/abs/2605.22785},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld10_03_nearest_11,
  title = {Weinbrand, "A search changer: auditing Google's AI overviews interface in political and news search," ScienceDirect pii S0736585326000535 (2026)},
  url = {https://www.admscentre.org.au/google-ai-search-summaries-vary-across-political-and-news-searches-study-finds/},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only; Some details not confirmed}
}

@misc{src_ld10_xu_iqbal_montgomery_2026,
  title = {Xu, Iqbal \& Montgomery 2026, arXiv:2605.14021},
  url = {https://arxiv.org/abs/2605.14021},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_ld10_ebu_bbc_news_integrity_2025,
  title = {EBU/BBC, News Integrity in AI Assistants, Oct 2025},
  url = {https://www.ebu.ch/research/open/report/news-integrity-in-ai-assistants},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_ld10_li_aral_2025,
  title = {Li \& Aral 2025, arXiv:2504.06435},
  url = {https://arxiv.org/abs/2504.06435},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

Back to the opportunity · All research opportunities