Research opportunity · Levelling or hollowing? Skill compression and the reproduction of expertise
Does skill compression reverse when AI acts as an agent rather than an assistant?
In short: has this been done?
Related work: Partly addressed. Related studies cover parts of this question; what they leave open is set out below. Only a first-pass search has been done, so it is not yet known whether a fuller search would find a study that does the whole design.
How far checked: First-pass screen only. A first-pass AI web search on 14 October 2026. No deeper check yet. Every search and check so far was done by AI; no person or domain expert has checked them yet.
The searches, the nearest work found and what the search could not reach
Follow this opportunity: Atom feed of RO-LD04-05 updates (new versions, and revisions of the claims it rests on).
Question
On the same tasks, does the advantage that AI gives less skilled workers when it assists them (a copilot) shrink or reverse when it carries out the work autonomously (an agent), as theory predicts?
Gap type: Untested beyond the original settingOne PhD chapter or paperLow to medium costRelated work: Partly addressed
Why it matters
Most evidence that AI helps novices most comes from 2023–2025 assistants. A model of knowledge work predicts that autonomous AI benefits the most knowledgeable instead, and agentic tools are now spreading. If the prediction holds, the levelling evidence describes a passing phase; if not, it may hold for agents too. Field data compare tool generations without splitting by skill, and the one direct agent-versus-chatbot experiment used students.
The debate lists this among what we still do not know: Will compression reverse as AI agents act autonomously rather than assist?
Evidence that motivates it
- A model of knowledge work predicts that AI which only assists (a copilot) benefits the least knowledgeable workers, while autonomous AI benefits the most knowledgeable, so the direction of skill compression can reverse as AI becomes more autonomous.
How this study uses it: The model predicts that a copilot benefits the least knowledgeable and autonomous AI the most knowledgeable, so the direction of compression can reverse. - In a 2025 randomized trial with 16 experienced open-source developers working on 246 tasks in their own projects, allowing AI tools increased completion time by 19%, although the developers estimated it had cut their time by 20%; METR later called its 2026 follow-up data an unreliable signal because of selection.
How this study uses it: Highly experienced developers were slower with early-2025 AI tools while believing they were faster, and METR's later data were too selected to read: expert gains from newer tools are unmeasured. - A 2026 meta-analysis of 23 programming studies found an overall productivity effect of g = 0.33, large in controlled experiments (g = 0.73) but not distinguishable from zero in enterprise or open-source settings, and no significant effect on learning (g = 0.14).
How this study uses it: Programming gains are large in controlled experiments but not distinguishable from zero in enterprise and open-source settings, and the meta-analysis does not test skill as a moderator.
A possible design
Randomized experiment with three arms on the same task bank: no AI, an AI assistant that suggests and explains, and an AI agent that completes tasks for the participant to review. Recruit participants across a measured range of baseline skill (a pre-test on the task domain), for example professional developers or data analysts. Outcomes are task quality graded blind, time, and errors that reach the final output. The estimand is how the gain from each arm changes with baseline skill.
Data needed
A task bank with gradable outcomes; access to current assistant and agent tools with fixed versions; a participant panel with a validated skill pre-test; graders blind to arm.
Feasibility
One PhD chapter or paper. Low to medium; participant payments and tool subscriptions; tool versions change quickly, so the study must fix and report them. Agents work best on some tasks and fail on others, so the task bank must include tasks outside the agent's competence, where review skill matters. Selection of willing participants biased METR's 2026 data, so recruitment must not depend on willingness to work without AI.
Proposal pack: this page and its references on one printable page, with BibTeX.
Has this been done?
Partly done already: related work covers some of this question (first-pass screen of 14 October 2026; not yet checked in depth).
No experiment was found that gives the same tasks a no-AI arm, an assistant arm and an autonomous-agent arm and estimates gains by baseline skill. Observational studies compare autocomplete, interactive agents and autonomous agents across hundreds of thousands of developers without a skill split in their abstracts, or find experienced programmers capturing most gains without separating agents from assistants. A small experiment compares a coding agent with a chatbot in students and measures comprehension. Two field experiments in the same customer-service operation found the least skilled gaining most from an assistant but report no skill split for the agentic version.
How strong this evidence is
- Who searched and checked: an independent AI reviewer with fresh context (not a person), 14 October 2026: verdicts, nearest-work reading labels and limits checked; query lists corrected to the 8 queries run where they differed. Every check so far was done by AI; no person or domain expert has checked it yet.
- What was read: 4 of the 5 nearest works below were judged from an abstract, summary or search listing only; each is marked. Not every entry says how much of the work was read.
- Where the search could not reach: Screened by web search (8 queries), arXiv abstract pages and one NBER abstract page on the date shown; the bodies of the observational studies were not read and may contain skill splits. SSRN and Google Scholar were not searched directly, and experiments outside software may be under-covered. Absence of a hit is not proof that the work does not exist.
- Before building a project on it, run your own search in a scholarly database such as Scopus, Web of Science, Google Scholar or SSRN.
Nearest work found
- Demirer, Musolff & Yang, "Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools," NBER Working Paper 35275 (2026, revised September 2026) Judged from an abstract, summary or search listing only
Abstract page read; the paper itself was not read. Matched event study using AI-usage telemetry for more than 500,000 GitHub developers: autocomplete, interactive agents and autonomous agents raise commits by 30%, 180% and 240%, with smaller effects on projects and releases. Not randomized; the abstract reports no split by developer skill. Figures differ from an earlier version. - Daniotti, Wachs, Feng & Neffke, "Who is using AI to code? Global diffusion and impact of generative AI," arXiv:2506.08945 (2025) Judged from an abstract, summary or search listing only
Abstract read. AI-written code detected in tens of millions of commits: experienced programmers capture nearly all the productivity and exploration gains. Observational; does not separate agentic from assistant use in the abstract. - Balepur, Baumler, Chen, Choi, Rudinger & Boyd-Graber, "(Im)Paired Programming: Coding Agents Improve Productivity but Harm Understanding," arXiv:2607.26375 (2026) Judged from an abstract, summary or search listing only
Abstract read. Experiment with 54 students building a website with a code-editing agent or a chatbot: the agent aids completion but harms comprehension and the ability to extend the code without it. Direct mode comparison, but students, no no-AI arm and no skill-gradient analysis in the abstract. - Ni, Wang, Feng, Lu, Wang & Zhou, "Generative AI in Action: Field Experimental Evidence from Alibaba's Customer Service Operations," arXiv:2603.29888 (2026) Judged from an abstract, summary or search listing only
Abstract read. Randomized field experiment with an AI assistant: lower-performing agents benefit most and top performers decline in quality. A companion experiment in the same operation (arXiv:2605.14830, abstract read) tests an agentic set-up and reports no skill split in its abstract; the two are separate experiments, not arms of one design. - Becker, Rush, Barnes & Rein, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity," arXiv:2507.09089 (2025)
Full text read for the LD-04 debate. Randomized AI-allowed against AI-disallowed tasks for experienced developers only; no assistant-versus-agent arm and no novice arm.
Search queries used
experiment AI agent versus copilot assistant productivity heterogeneity worker skill experience randomized 2025 2026autonomous AI agents benefit high-skilled workers more experiment "agentic" coding agents experienced developers gains heterogeneityMETR developer productivity study 2026 results AI agents Claude Code experienced developersautocomplete interactive coding agents autonomous coding agents GitHub developers coding activity 40% 140% 180% paperCursor agent adoption experienced developers productivity planning supervising preprint arXivDaniotti 2026 AI coding 160,000 developers experienced developers gains early-careerrandomized experiment chat assistant versus autonomous agent arm knowledge workers high-skill low-skill benefit "agent" treatment 2026coding agents productivity effect by developer experience junior versus senior heterogeneity 2026 study Claude Code Codex tenure
Cite this page
SocioLens project. (2026, October 14). Does skill compression reverse when AI acts as an agent rather than an assistant? [Research opportunity RO-LD04-05, version v1]. SocioLens. https://sociolens-web.pages.dev/opportunities/ro-ld04-05/v1/
@misc{sociolens_ro_ld04_05_v1,
author = {{SocioLens project}},
title = {Does skill compression reverse when AI acts as an agent rather than an assistant?},
year = {2026},
date = {2026-10-14},
howpublished = {\url{https://sociolens-web.pages.dev/opportunities/ro-ld04-05/v1/}},
note = {Research opportunity RO-LD04-05, version v1}
}The link points to this version, whose record never changes. For a finding, cite the original study as well: SocioLens summarizes research and does not replace it.
Provenance
RO-LD04-05 v1. Drafted by Claude (SocioLens Project Lead), AI-drafted from the LD-04 debate on 2026-10-14. Checked by an independent AI reviewer with fresh context (not a person), 14 October 2026: verdicts, nearest-work reading labels and limits checked; query lists corrected to the 8 queries run where they differed. Not yet reviewed by a domain expert. This is a research hypothesis, not a finding.