Preview: not announced. Search engines asked not to index.

Proposal pack · RO-LD03-05 v1

Do generative-AI deployments in government improve services, not just staff time?

To keep this as a PDF or print it, use your browser's Print command (Ctrl+P, or Cmd+P on a Mac) and choose "Save as PDF". Links print with their addresses. This pack gathers what the opportunity page already holds; nothing in it is new.

Read this first. This is a research hypothesis drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-03 debate on 15 October 2026, not a finding. Checked by an independent AI reviewer with fresh context (not a person), 15 October 2026 (review of PR #77): verdicts, query counts and 13 abstract-read labels checked; label slips corrected. Every check so far was done by AI; no person or domain expert has checked it yet. The literature screen below was done by AI web search and can miss relevant work. Version v1, permanent address https://sociolens-web.pages.dev/opportunities/ro-ld03-05/v1/.

Wrong or missing validation target Funded team project Needs a budget (medium or higher) Partly addressed

Research question

When a government agency deploys generative AI to caseworkers, adjudicators or reviewers, do service outcomes recorded in administrative data (processing time, error and appeal rates, outcomes for citizens) improve against a credible comparison group, beyond staff time savings?

Why it matters

Public managers in the EU already use generative AI, often without approval or policy, and in US state agencies what drove adoption of earlier chatbots differed from what made implementation succeed. The UK and Australian government evaluations of assistant trials could not be read here, and the comparison-group evaluations the screen found rely on staff self-reports. Time saved by staff is not the public value an agency exists to deliver, so the measure that would justify or stop wider deployment is missing.

The debate lists this among what we still do not know: Do public-sector deployments improve service outcomes? The UK and Australian government trials could not be read for this version.

Evidence that motivates it

  1. The authors of a survey of 576 public managers in seven EU member states describe public servants using generative AI to draft reports, refine policy proposals and generate content, often without explicit approval or institutional policies, which they call 'shadow use'; the abstract does not say how much of this the survey itself measured. (LD03-SL-C10 v1.0)
    Public managers in seven EU member states use generative AI to draft reports and policy proposals, often without explicit approval or institutional policy.
    Cites: Tangi et al. (2025), “A Silent Partner: The Shadow…”
  2. In 22 US state agencies using chatbots, the factors that drove adoption (ease of use, leadership, external shocks such as COVID-19) overlapped only partly with those that shaped implementation (building and maintaining the knowledge base, skills, resources, cross-agency coordination, external shocks and citizens' expectations). (LD03-SL-C22 v1.0)
    In 22 US state agencies using chatbots, adoption and implementation success had different drivers; outcomes for services were not the measure.
    Cites: Chen et al. (2024), “The Adoption and Implementation of…”

Claims are shown as they read on the day this pack was built; claim pages keep every earlier version.

A possible design

Randomized or staggered rollout across offices or caseloads in one agency, randomizing cases within reviewer and case-type strata where possible. Outcomes come from administrative records: time to decision, error rates found in audit samples, appeal and reversal rates, and, where measurable, outcomes for the people served. Pre-register the outcomes and the audit sampling, and report staff time alongside service outcomes so the two can be compared.

Data needed

An agency partner with case-level administrative records and an audit process; legal basis for research use of the records; enough offices or cases for a comparison group.

Feasibility

Funded team project. Medium; the deployment is the agency's own, but audit sampling, data access agreements and an independent evaluation team cost money and time. A registered randomized trial of AI in building-permit review in Chilean municipalities, due to end in December 2026, may report the core design soon, so check it before starting. Appeals arrive months after decisions, so follow-up must be long enough to see them.

Has this been done?

Partly done already: related work covers some of this question (first-pass screen of 15 October 2026; not yet checked in depth).

Controlled evaluations exist, but they use historical or benchmark cases, constructed tasks or staff self-reports. No evaluation was found of a live generative-AI deployment in a government agency that uses a randomized or quasi-experimental comparison and measures service outcomes from administrative records. A registered randomized trial with processing time as its outcome is under way in Chilean municipalities, with no results yet; the gap may close soon.

Where the search could not reach. Screened by web search (8 queries) and arXiv abstract pages on the date shown; government sites (gov.uk, Australian government evaluation pages) and the Stanford PDF could not be opened from this environment. No native-language searches were run for Korea, Singapore, Estonia or China, and public-administration journals were not searched directly. Absence of a hit is not proof that the work does not exist.

Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.

Nearest work found

References

Sources behind the motivating claims:

BibTeX

The first entry cites this opportunity; then its nearest work; then the sources above. Nearest work and sources are recorded as reference strings, so each goes in whole as the title; add authors and year from the original before you cite it.

@misc{sociolens_ro_ld03_05_v1,
  author = {{SocioLens project}},
  title = {Do generative-AI deployments in government improve services, not just staff time?},
  year = {2026},
  date = {2026-10-15},
  howpublished = {\url{https://sociolens-web.pages.dev/opportunities/ro-ld03-05/v1/}},
  note = {Research opportunity RO-LD03-05, version v1}
}

@misc{ro_ld03_05_nearest_1,
  title = {Magesh, Martin, Surani, Perez, Rodolfa \& Ho, "Evaluating Generative AI in Benefits Administration: A Demonstration Project," Stanford RegLab and Yale Tobin Center working paper (2026)},
  url = {https://dho.stanford.edu/wp-content/uploads/CDLE.pdf},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only; Some details not confirmed}
}

@misc{ro_ld03_05_nearest_2,
  title = {Gosciak, Giannella, Guo, Chen \& Koenecke, "LLMs in social services: How does chatbot accuracy affect human accuracy?" arXiv:2603.11213 (2026)},
  url = {https://arxiv.org/abs/2603.11213},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld03_05_nearest_3,
  title = {Fitzpatrick, Kelly, Carey, Walsh \& Nugent, "Assessing Generative AI value in a public sector context: evidence from a field experiment," arXiv:2502.09479 (2025)},
  url = {https://arxiv.org/abs/2502.09479},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld03_05_nearest_4,
  title = {UK Department for Work and Pensions (DWP), "An Evaluation of DWP's Microsoft 365 Copilot Trial" (January 2026)},
  url = {https://www.gov.uk/government/publications/an-evaluation-of-dwps-microsoft-copilot-365-trial},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only; Some details not confirmed}
}

@misc{ro_ld03_05_nearest_5,
  title = {AEA RCT Registry, "Experimental Evaluation of the Use of Artificial Intelligence in Building Permit Review," trial AEARCTR-0017575 (Chilean municipalities)},
  url = {https://www.socialscienceregistry.org/trials/17575},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{ro_ld03_05_nearest_6,
  title = {Commonwealth of Pennsylvania, ChatGPT Enterprise pilot report (2025)},
  url = {https://www.pa.gov/governor/newsroom/2025-press-releases/-shapiro-administration-leads-the-way-in-ethical-use-of-ai},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated; Judged from an abstract, summary or search listing only}
}

@misc{src_sl_tangi_rodriguez_muller_combetto_epart_2025,
  title = {Tangi, Rodriguez Müller \& Combetto, “A Silent Partner: The Shadow Presence of Generative Artificial Intelligence in Public Administrations,” Electronic Participation (ePart 2025), LNCS},
  url = {https://doi.org/10.1007/978-3-032-02515-9_5},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_sl_chen_gasco_hernandez_esteve_arpa_2024,
  title = {Chen, Gascó-Hernandez \& Esteve, “The Adoption and Implementation of Artificial Intelligence Chatbots in Public Organizations: Evidence from U.S. State Governments,” American Review of Public Administration 54(3), 2024},
  url = {https://doi.org/10.1177/02750740231200522},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

Back to the opportunity · All research opportunities