Preview: not announced. Search engines asked not to index.

Proposal pack · RO-LD07-03 v2

How much AI-assisted or agent responding reaches probability panels?

To keep this as a PDF or print it, use your browser's Print command (Ctrl+P, or Cmd+P on a Mac) and choose "Save as PDF". Links print with their addresses. This pack gathers what the opportunity page already holds; nothing in it is new.

Read this first. This is a research hypothesis drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-07 debate on 12 October 2026, not a finding. Checked by Claude (SocioLens Project Lead) with AI research agents, by web search; not a person and not a domain expert; an independent AI reviewer with fresh context (not a person), 12 October 2026: every nearest-work citation checked against its source page; corrections applied. Every check so far was done by AI; no person or domain expert has checked it yet. The literature screen below was done by AI web search and can miss relevant work. Version v2, permanent address https://sociolens-web.pages.dev/opportunities/ro-ld07-03/v2/.

Not yet measured One PhD chapter or paper Low to medium cost No study found

Research question

In probability-based panels, what share of completed interviews show AI assistance or autonomous-agent completion when measured with embedded multi-method detection validated against consented chatbot-history linkage, and how does that compare with opt-in platforms measured the same way?

Why it matters

Probability panels are the reference standard that much of social science relies on, and providers say they are less exposed. That is currently an assumption rather than a measurement. A measured rate, even a low one, would give the dispute about AI contamination of survey research its first evidence from this tier of panels.

The debate lists this among what we still do not know: How common are fully autonomous agents in probability-based or high-assurance survey panels?

Evidence that motivates it

  1. Autonomous LLM agents can complete online surveys while passing conventional attention and logic checks at very high rates under tested conditions. (LD07-HQ-C09 v1.0)
    Agents can pass conventional quality checks, so standard panel QA cannot be assumed to catch them.
    Cites: Westwood (2025), “The potential existential threat of…”
  2. AI-mediated responding is measurable in current online research, but self-reported assistance, overt outsourcing, AI-test failure and autonomous-agent detection are different estimands and cannot be pooled into one prevalence rate. (LD07-HQ-C10 v1.0)
    Existing prevalence estimates measure different pathways (assistance, outsourcing, AI-test failure, agents) and cannot be pooled into one rate; none of them comes from a probability panel.
    Cites: Westwood & Frederick (2026), “Reply to Van der Stigchel…”; Asher et al. (2026), “Chatbots Are Undermining…”; Zhang et al. (2025), “Generative AI Meets Open-Ended…”; Veselovsky et al. (2025), “Prevalence and prevention of…”; Chen et al. (2026), “Estimating the threat of AI-agent…”; Gordon et al. (2026), “AI Agent Prevalence and Data…”; Xu & Malkin, “A Penny for Your Prompts: Experiments…”; Lin (2026), “Artificial-Intelligence-Mediated…”

Claims are shown as they read on the day this pack was built; claim pages keep every earlier version.

A possible design

Partner with a probability panel that accepts researcher-proposed modules. Embed a short detection module: paste and keystroke paradata, honeypot items, and AI-capability test items, with a consented sub-sample linking chatbot histories (following Sehgal et al.) to validate the detectors. Field the identical module on two opt-in platforms. Report pathway-specific rates (assistance, outsourcing, agent completion) rather than one contamination rate.

Data needed

A module in a probability panel (several panels accept academic proposals at low or no cost); parallel opt-in samples; consent and ethics approval for history linkage.

Feasibility

One PhD chapter or paper. Low to medium: panel access is often free for academic proposals; opt-in samples cost a few hundred to a few thousand US dollars. Depends on a panel partner; the ethics approval for history linkage is the main hurdle.

Has this been done?

The search found no study that does the core design (first-pass screen of 12 October 2026, checked again on 13 October 2026).

No measurement in a probability panel was found. Sehgal et al. (7 October 2026) linked donated chatbot histories to over 700,000 submissions on a curated opt-in platform and found about 1% AI-assisted, highly concentrated among a few workers, with half of assisted submissions outside what the platform detector covers; that method could validate detectors in a probability panel. Opt-in studies now estimate AI use and agent presence across many platforms (Gordon et al.; Chen, Urminsky, Zhang et al.; Xu and Malkin 2026) and validated agent-detection items exist (Affonso 2026), and NORC has built an AmeriSpeak-trained detector, but none reports a prevalence rate in a probability panel measured with validated, pathway-specific detection.

Literature check. No measurement of AI-assisted or agent responding in a probability-based panel was found. New work is all on opt-in platforms (Xu and Malkin 2026: under 10% on Prolific to over 80% on MTurk; Affonso 2026: cognitive-trap detection of agents on Prolific, MTurk and CloudResearch; Zhang, Xu and Alvero 2025: 34% self-reported use on an online platform) or on detector building (Wang et al. 2026; NORC's AmeriSpeak detector reports misclassification, not prevalence). The design gap therefore stands; verdict stays OPEN because nothing measures the core quantity in the probability-panel tier, though the opt-in comparison arm and detector components are now well developed. Checked by Claude (SocioLens Project Lead) with AI research agents, by web search; not a person and not a domain expert.

Where the search could not reach. Screened by web search on the date shown; Google Scholar, SSRN and paywalled journal pages were only partly reachable. Absence of a hit is not proof that the work does not exist. Literature check of 2026-10-13: Probability-panel provider reports (LISS, GESIS Panel, Understanding America Study, Ipsos KnowledgePanel, Pew ATP methodology notes) were not systematically searched beyond the queries listed; a provider-internal rate could exist in grey literature or conference talks (for example AAPOR 2026 sessions) not indexed by web search. Scholarly APIs were not used. Absence of a hit is not proof that the work does not exist. A NORC talk at AAPOR (Lerner, 15 May 2026) on detecting AI-generated survey responses was not read; which panel it uses and whether it reports a prevalence rate is unknown.

Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.

Nearest work found

References

Sources behind the motivating claims:

BibTeX

The first entry cites this opportunity; then its nearest work; then the sources above. Nearest work and sources are recorded as reference strings, so each goes in whole as the title; add authors and year from the original before you cite it.

@misc{sociolens_ro_ld07_03_v2,
  author = {{SocioLens project}},
  title = {How much AI-assisted or agent responding reaches probability panels?},
  year = {2026},
  date = {2026-10-13},
  howpublished = {\url{https://sociolens-web.pages.dev/opportunities/ro-ld07-03/v2/}},
  note = {Research opportunity RO-LD07-03, version v2}
}

@misc{ro_ld07_03_nearest_1,
  title = {Sehgal, Tonneau, Folk, Ungar \& Guntuku, "AI-Assisted Submissions in Online Research Are Rare and Highly Concentrated but Routinely Approved," arXiv:2610.09279 (7 Oct 2026)},
  url = {https://arxiv.org/abs/2610.09279},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld07_03_nearest_2,
  title = {Gordon, Rothschild, Affonso, Sulik, Hauser, Pepin \& Jones, "AI Agent Prevalence and Data Quality Across Multiple Online Sample Providers," PsyArXiv (Apr 2026)},
  url = {https://osf.io/preprints/psyarxiv/pvdjr_v2},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld07_03_nearest_3,
  title = {Chen, Urminsky, Zhang et al., "Estimating the threat of AI-agent responding across online survey platforms," PsyArXiv (Mar 2026)},
  url = {https://doi.org/10.31234/osf.io/xcg26},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld07_03_nearest_4,
  title = {NORC, "Fraudulent respondents and bots in nonprobability surveys: A literature review" (Apr 2026)},
  url = {https://www.norc.org/content/dam/norc-org/pdf2026/cpss-research-brief-fraud-lit-review.pdf},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld07_03_nearest_5,
  title = {Sepulvado, "Detecting AI Responses in Survey Data: NORC's Next Leap for Data Quality," NORC (Dec 2025)},
  url = {https://www.norc.org/research/library/detecting-ai-responses-survey-data-norcs-next-leap-data-quality.html},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld07_03_nearest_6,
  title = {Xu \& Malkin, "A Penny for Your Prompts: Experiments Detecting and Mitigating LLM Usage by Survey Respondents," arXiv:2607.00403 (2026)},
  url = {https://arxiv.org/abs/2607.00403},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld07_03_nearest_7,
  title = {Affonso, "Brief Commentary: A Framework for Detecting AI Agents in Online Research," Journal of Consumer Research 53(3):619-633 (2026), doi:10.1093/jcr/ucag006},
  url = {https://academic.oup.com/jcr/article/53/3/619/8540432},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld07_03_nearest_8,
  title = {Zhang, Xu \& Alvero, "Generative AI Meets Open-Ended Survey Responses: Research Participant Use of AI and Homogenization," Sociological Methods \& Research 54(3) (2025), doi:10.1177/00491241251327130},
  url = {https://journals.sagepub.com/doi/10.1177/00491241251327130},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld07_03_nearest_9,
  title = {Wang, Mamaev \& Leckie, "Towards Detecting AI-Assisted Responses in Online Surveys," arXiv:2609.17317 (2026)},
  url = {https://arxiv.org/abs/2609.17317},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{ro_ld07_03_nearest_10,
  title = {Huang, Sepulvado \& Lerner, "Tool Development and Bias Mitigation: Detecting AI-Generated Survey Responses," AAPOR presentation, NORC (14 May 2025)},
  url = {https://www.norc.org/content/dam/norc-org/pdf2025/AAPOR%20Detecting%20AI-Generated%20Survey%20Responses%20-%20Lilian%20Huang.pdf},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_ld07_westwood_survey_threat_2025,
  title = {Westwood, “The potential existential threat of large language models to online survey research,” PNAS 122(47):e2518075122 (2025)},
  url = {https://doi.org/10.1073/pnas.2518075122},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_ld07_westwood_frederick_reply_2026,
  title = {Westwood \& Frederick, “Reply to Van der Stigchel et al.: Empirical evidence that AI survey contamination is real and substantial,” PNAS 123(8):e2537420123 (2026)},
  url = {https://doi.org/10.1073/pnas.2537420123},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_ld07_asher_keystroke_2026,
  title = {Asher, Gold, Chen \& Carvalho, “Chatbots Are Undermining Crowdsourced Research in the Behavioral Sciences,” AMPPS 9(1) (2026)},
  url = {https://doi.org/10.1177/25152459261424723},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_ld07_zhang_homogenization_2025,
  title = {Zhang, Xu \& Alvero, “Generative AI Meets Open-Ended Survey Responses,” Sociological Methods \& Research 54(3):1197–1242 (2025)},
  url = {https://doi.org/10.1177/00491241251327130},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_ld07_veselovsky_crowdwork_2025,
  title = {Veselovsky, Horta Ribeiro, Cozzolino, Gordon, Rothschild \& West, “Prevalence and prevention of large language model use in crowd work,” CACM 2025},
  url = {https://doi.org/10.1145/3685527},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_ld07_chen_ai_agent_responding_2026,
  title = {Chen, Urminsky et al., “Estimating the threat of AI-agent responding across online survey platforms,” PsyArXiv 2026},
  url = {https://doi.org/10.31234/osf.io/xcg26_v1},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_ld07_gordon_prevalence_2026,
  title = {Gordon et al., “AI Agent Prevalence and Data Quality Across Multiple Online Sample Providers,” PsyArXiv 2026},
  url = {https://doi.org/10.31234/osf.io/pvdjr_v2},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_ld07_xu_malkin_prompts_2026,
  title = {Xu \& Malkin, “A Penny for Your Prompts: Experiments Detecting and Mitigating LLM Usage by Survey Respondents,” arXiv:2607.00403},
  url = {https://arxiv.org/abs/2607.00403},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

@misc{src_ld07_lin_taxonomy_2026,
  title = {Lin, “Artificial-Intelligence-Mediated Contamination in Online Research: Taxonomy, Risk Gradient, and Recommendations,” AMPPS 9(3) (2026)},
  url = {https://doi.org/10.1177/25152459261454231},
  note = {Reference as recorded by SocioLens; authors, title and year are not separated}
}

Back to the opportunity · All research opportunities