LD-05 — DISCRETION CURTAILED, RELOCATED OR ENCODED?
Living Debate v0.1 — LOCKED
STATUS
LOCKED FOR SOCIOLENS PUBLICATION v0.1.
LAST UPDATED
25 September 2026
QUESTION
When public organisations introduce algorithmic, machine-learning or foundation-model systems into administrative decision-making, does human discretion actually shrink?
Or does it move into system design, selection rules, thresholds, workflow routing, written records, overrides and appeals?
SHORT ANSWER
The evidence does not support a simple story of “automation replacing discretion.”
The stronger pattern is relocation.
Algorithmic systems can narrow some forms of frontline judgement while shifting consequential choices elsewhere:
• upstream into categories, thresholds and design;
• sideways into selection and routing;
• downstream into override, appeal and repair;
• and, with generative AI, into the production of the official administrative record itself.
A final human decision is therefore not enough to establish meaningful human oversight.
The central question is:
WHERE IS CONSEQUENTIAL JUDGEMENT LOCATED, WHO CAN EXERCISE IT, AND WHO CAN CONTEST OR CORRECT IT?
EVIDENCE LANGUAGE
SocioLens distinguishes:
FACT — directly supported empirical or legal result.
ESTIMATE — a measured quantity whose denominator and period matter.
INTERPRETATION — synthesis across cases.
UNKNOWN — an unresolved empirical question.
WHY THIS DEBATE EXISTS
Public-administration systems often get described using a binary vocabulary:
automated or human;
algorithmic or discretionary;
decision support or decision-making.
Actual administrative systems are more complicated.
A model may not decide a benefit claim, yet still determine who must provide more evidence.
A score may not legally bind a caseworker, yet still function as a powerful default.
A human may formally retain the final decision while having little time, weak information or strong organisational incentives to accept the system’s recommendation.
A generative-AI tool may never recommend an administrative outcome at all, yet alter the official record on which later decisions depend.
The key object is therefore not “automation” in the abstract.
It is the distribution of discretion across the administrative process.
A MAP OF WHERE DISCRETION CAN SIT
For LD-05, SocioLens uses a six-location analytical scaffold.
This is a debate-specific analytical device, not a new Field Map layer.
L1 — POLICY
What goals, legal categories and administrative priorities are defined?
L2 — DESIGN
What variables, categories, thresholds, labels and rules are encoded?
L3 — SELECTION / TRIAGE
Who is investigated, routed, delayed, audited, home-visited or asked for more evidence?
L4 — DETERMINATION
Who makes the formal final administrative decision?
L5 — WRITTEN RECORD
What facts, summaries and formulations enter the official file and later become evidence?
L6 — OVERRIDE / APPEAL / REPAIR
Who can depart from the system, challenge an outcome and force correction?
These locations are analytically distinct.
Discretion can shrink at one while expanding at another.
1. HUMAN-IN-THE-LOOP IS NOT A SAFEGUARD BY DEFINITION
INTERPRETATION
The presence of a human reviewer tells us very little unless we know how the workflow works.
Meaningful human discretion depends on questions such as:
• Does the human see the model output?
• Does the human know the model affected the case?
• Can the human override it?
• How often?
• Must disagreement be justified?
• How much time is available?
• Does overriding create managerial or administrative cost?
Three cases make the point in different ways.
POLAND: FORMAL OVERRIDE WAS EXCEPTIONALLY RARE
FACT
In Poland’s unemployment-profiling system, counsellors formally changed the assigned profile in only 0.58% of cases in the early implementation data analysed by Sztandar-Sztanderska and Zieleńska.
But this did not mean discretion disappeared.
The researchers show that frontline discretion persisted inside:
• how questions were asked;
• how ambiguous answers were interpreted;
• what was entered into the system;
• and how interactions with jobseekers unfolded.
The lesson is important:
LOW FORMAL OVERRIDE DOES NOT MEAN ZERO HUMAN DISCRETION.
It may mean that discretion moved earlier into data production.
ALLEGHENY: HUMAN OVERRIDE WAS SUBSTANTIVE
FACT
In Allegheny County’s child-welfare screening process, supervisors could depart from mandatory screen-in recommendations.
They screened out 21.0% of children marked as mandatory screen-ins.
Corrected analysis also supports the conclusion that worker judgement reduced racial disparity relative to an AFST-only counterfactual.
This is a case where “human in the loop” reflected actual decision authority rather than a ceremonial review step.
But the corrected analysis also matters for another reason.
The authors no longer draw a clear overall conclusion about whether the algorithm or workers were more accurate.
So the case does not support a general claim that humans are better.
It supports a narrower claim:
human departure materially changed the distribution of decisions.
KENTUCKY: HUMAN DEPARTURE COULD AMPLIFY DISPARITY
FACT
Kentucky introduced mandatory use of a pretrial risk assessment.
The reform initially changed bail practice, but much of that change eroded as judges moved back toward previous habits.
A separate 2019 working paper by Alex Albright indicates that departures toward harsher conditions were more common for Black defendants.
Together with Allegheny, this resolves a common false dichotomy.
HUMAN OVERRIDE HAS NO FIXED FAIRNESS DIRECTION.
Its effect depends on:
• the human baseline;
• the institution;
• the default;
• and what exactly the human is allowed to change.
2. SELECTION CAN MATTER BEFORE THE FINAL DECISION
INTERPRETATION
One of the strongest patterns in the evidence is that administrative burden often begins before the final legal decision.
An algorithm may only “select” or “route” a case.
But selection can determine who:
• has to provide extra evidence;
• receives additional scrutiny;
• is subjected to a home visit;
• waits longer;
• or becomes the target of a manual investigation.
That means the distinction between SELECTION and DETERMINATION is crucial.
DWP: A HUMAN DECIDES, BUT THE MODEL SELECTS WHO GETS SCRUTINISED
FACT
The UK Department for Work and Pensions uses a machine-learning model to identify Universal Credit Advance requests for additional fraud review.
The final approval or refusal is made by a human.
But the human reviewer:
• is not shown the risk rating;
• and is not told that the referral was generated by the model.
Non-UK nationals were over-referred in both audited periods.
Some older groups were also over-referred in 2024–25.
Selected legitimate claims can face additional evidence requests and delay before the final decision.
DWP’s own 2025–26 effectiveness assessment reports that the model was 2.5 times more effective than a random control sample at identifying Advances later confirmed as fraudulent by a human decision-maker.
This measured targeting advantage does not answer whether referral burdens are distributed fairly.
This is why:
HUMAN REVIEW AFTER SELECTION DOES NOT VALIDATE THE SELECTION STAGE.
DUO: INDIRECT DISCRIMINATION THROUGH SELECTION
FACT
The Dutch student-finance agency DUO used a risk profile to select addresses for home visits.
Later analysis found indirect discrimination against students with a migration background.
The disparity appeared across multiple stages of the control process.
DUO reports that 86% of students who lodged objections had a non-European migration background.
The risk-profile selection was abandoned in 2023.
DUO says current home-visit selection is random.
Importantly, this is not evidence that migration background itself was used as an input.
It shows how apparently neutral selection criteria can produce unequal exposure when they correlate with social structure.
TOESLAGEN: NATIONALITY AS A RISK-SELECTION INDICATOR
FACT
The Dutch Tax Administration used a self-learning risk-classification model to select childcare-benefit applications for additional manual review.
During the verified period, Dutch versus non-Dutch nationality status was used as a risk indicator.
The Dutch Data Protection Authority later found unlawful and discriminatory processing of nationality data and imposed a €2.75 million fine.
The model did not itself issue the final benefits decision.
Its role was upstream:
it influenced who received additional scrutiny.
That distinction is central to LD-05.
A system can be consequential without being the final decision-maker.
3. FORMAL HUMAN AUTHORITY AND EFFECTIVE DISCRETION ARE DIFFERENT QUESTIONS
FACT + INTERPRETATION
Austria’s AMAS case makes this distinction unusually clear.
In a 2025 decision, Austria’s Federal Administrative Court held that AMAS was not a solely automated decision under Article 22 GDPR.
Under the formal rules:
• counsellors must discuss the score with the client;
• contrary views can be documented;
• counsellors make the substantive final assessment.
That answers a legal question:
WHO IS FORMALLY AUTHORISED TO DECIDE?
It does not answer the sociological question:
HOW INDEPENDENT IS HUMAN JUDGEMENT IN DAILY PRACTICE?
To answer that, we would need field evidence on:
• actual override behaviour;
• anchoring;
• time pressure;
• workload;
• organisational expectations;
• and the practical cost of disagreement.
LEGAL ATTRIBUTION OF A DECISION
IS NOT THE SAME THING AS
EMPIRICAL DESCRIPTION OF DECISION PRACTICE.
4. REVIEW RIGHTS CAN EXIST WITHOUT SYSTEM-LEVEL REPAIR
FACT + INTERPRETATION
Robodebt shows why appeal and review must be analysed as organisational feedback systems rather than only individual rights.
Australia’s Royal Commission found a broader institutional failure surrounding the scheme and issued 57 recommendations.
Its recommendations include:
• clearer paths to review;
• publication and escalation of significant tribunal decisions;
• stronger automated-decision governance;
• public explanation of automated processes;
• independent scrutiny of business rules and algorithms;
• protection against debt recovery while a debt is being reviewed;
• greater opportunity to challenge debts before recovery.
The Commission also examined the government’s continued defence of the scheme despite adverse Administrative Appeals Tribunal decisions and legal concerns.
The lesson is:
AN APPEAL CAN CORRECT A CASE WITHOUT REPAIRING THE SYSTEM.
Effective remedy requires a feedback path through which recurring individual failures become organisational learning and policy change.
5. EXTERNAL CHALLENGE CAN CHANGE THE SYSTEM — BUT NOT NECESSARILY THE FUNCTION
FACT
The UK Home Office introduced an automated visa Streaming Tool in 2015.
It assigned red, amber or green risk levels and used Global Visa Risk Status data including nationality.
In 2020, the Joint Council for the Welfare of Immigrants and Foxglove brought a judicial-review challenge alleging race discrimination and discriminatory feedback-loop effects.
The Home Office withdrew the Streaming Tool during the proceedings.
There was no final merits judgment declaring the tool unlawful.
That distinction matters.
The accurate statement is:
A LEGAL CHALLENGE WAS FOLLOWED BY WITHDRAWAL BEFORE A MERITS RULING.
But the story did not end with the removal of routing.
Successor Complexity Application Routing Solution systems remained in production.
A December 2024 Home Office Algorithmic Transparency Record describes CARS(V) version 2.0 as a production, semi-automated rule-based workflow-routing system that:
• classifies likely complexity;
• routes cases to different grades of decision-maker or enrichment;
• does not make the final visa decision;
• allows rerouting by caseworkers.
Separate Home Office guidance updated in 2025 shows CARS variants in active use on additional visa routes.
The administrative function survived in redesigned form.
This suggests a broader principle:
SYSTEM REPAIR CAN CHANGE THE FORM OF DISCRETION WITHOUT REMOVING THE UNDERLYING FUNCTION.
6. GENERATIVE AI CREATES A NEW LOCATION OF DISCRETION: THE OFFICIAL RECORD
EMERGING FACT + INTERPRETATION
Classic algorithmic-governance debates focus on:
scores;
classifications;
recommendations;
eligibility;
risk.
Foundation models add another mechanism.
They can help produce the text that becomes the administrative record.
An independent qualitative report by the Ada Lovelace Institute studied AI transcription and summarisation tools in social care across 17 local authorities in England and Scotland.
In early 2025, one tool was already in active social-care use across 85 local authorities.
Interviewees reported real benefits, especially reduced documentation burden.
But they also reported:
• hallucinations;
• omissions;
• misrepresentation;
• poor transcription;
• names and dialects being rendered incorrectly;
• changes in tone;
• documents becoming less person-centred.
One social worker described an AI-generated summary that falsely indicated suicidal ideation and was caught before entering the official case note.
The evidence does not tell us how often such errors occur.
It also does not show that AI-generated records are generally worse than human-authored records.
What it does establish is a new location of administrative judgement:
RECORD-MAKING.
When an LLM summarises an interaction, it can shape what later decision-makers encounter as “the facts” of the case.
7. HUMAN OVERSIGHT CAN BECOME CHECKING LABOUR
FACT + INTERPRETATION
In the social-care study, workers remained responsible for reviewing AI-generated records.
But review practices varied substantially.
Some workers described lengthy checking.
Others described only a few minutes of screening before moving text into official systems.
This matters because “human in the loop” can move work rather than remove it.
The labour may shift from:
writing a record
to:
checking;
editing;
prompting;
verifying;
and accepting responsibility for generated text.
That is an organisational effect, not merely a model-quality issue.
8. CONTESTABILITY REQUIRES PROCESS PROVENANCE
INTERPRETATION
If a person wants to challenge an administrative outcome, knowing the final decision-maker may not be enough.
Meaningful contestability can require knowing:
• Was the case algorithmically selected?
• Did a score affect routing?
• Was extra scrutiny triggered upstream?
• Was part of the official record generated by AI?
• What information did the human reviewer see?
• Who had authority to correct the relevant stage?
The Ada report recommends making AI-generated records identifiable.
The reason is straightforward.
A later decision-maker cannot calibrate scrutiny to AI-generated evidence if they do not know AI contributed to the record.
And an affected person cannot easily seek redress for an inaccurate record if the provenance of that record is invisible.
9. THE FOUR POSITIONS
P1 — CONSTRAINT / AUTOMATION
Claim:
Digital systems reduce discretionary variation by narrowing options and standardising decisions.
Current status:
Supported conditionally.
Strong defaults and routing systems can constrain behaviour.
But constraints can erode, be overridden or relocate discretion elsewhere.
P2 — RELOCATION
Claim:
Discretion does not disappear; it moves into categories, thresholds, selection, routing, documentation and repair.
Current status:
Best supported.
This is the strongest cross-case synthesis.
P3 — HYBRID / AUGMENTED PROFESSIONALISM
Claim:
A well-designed system can structure routine tasks while preserving meaningful professional judgement.
Current status:
Supported conditionally.
A genuine hybrid design requires more than a human name at the end of the workflow.
It requires:
• actual authority;
• time;
• information;
• ability to depart;
• feedback;
• accountability.
P4 — INFRASTRUCTURAL / HIDDEN DISCRETION
Claim:
Visible frontline discretion may shrink while less visible organisational and technical judgement expands.
Current status:
Strongly supported in organisational/design form.
Evidence for independent vendor discretion is currently much thinner and remains an open question.
10. WHAT IS OLD — AND WHAT IS NEW?
OLD BUREAUCRATIC PROBLEMS
• rules versus practice;
• classification;
• administrative burden;
• street-level discretion;
• institutional incentives;
• appeals;
• organisational learning;
• records shaping later decisions.
ALGORITHMIC SYSTEMS RECONFIGURE THEM
• selection can operate at much greater scale;
• thresholds and categories can become more centralised;
• routing can be consequential without being legally defined as a decision;
• default recommendations can create anchoring;
• errors can be distributed systematically.
FOUNDATION MODELS ADD SOMETHING MORE DISTINCTIVE
They can shape the official record itself.
The model may never score a citizen or recommend a final outcome.
Yet it can still influence the evidence on which future human decisions are based.
11. WHAT WE STILL DO NOT KNOW
UNKNOWN
1. How often do public-sector workers actually depart from algorithmic recommendations across comparable systems?
2. Under which organisational conditions does human review genuinely correct upstream model errors?
3. How often do appeals propagate into system-level redesign rather than one-case correction?
4. How common are selection-stage burdens that are difficult to challenge because no formal adverse decision has yet occurred?
5. What are the citizen-level consequences of AI-generated inaccuracies in official records?
6. How frequently do uncorrected AI hallucinations enter public-sector case files?
7. How much substantive discretion sits with vendors and procurement contracts rather than public agencies?
8. How often do internal monitoring and frontline staff catch system failures before regulators, courts, journalists or advocacy groups do?
12. WHAT EVIDENCE WOULD CHANGE THE PICTURE?
The current synthesis would shift materially with:
• comparable cross-jurisdiction data on real override/deference rates;
• field experiments on different human-review designs;
• evidence that selection-stage disparities produce little or no material burden;
• durable cases where algorithmic constraint reduces arbitrary variation without relocating significant discretion;
• systematic evidence that internal governance catches failures early;
• citizen-outcome studies of AI-generated administrative documentation;
• procurement/contract studies showing where substantive decision authority sits;
• evidence that formal appeal channels reliably generate upstream organisational correction.
13. CURRENT SOCIOLENS POSITION
INTERPRETATION
The strongest evidence does not support asking:
“Did AI remove human discretion?”
The better question is:
“Where did discretion move?”
In contemporary algorithmic public administration, consequential judgement can sit in:
• policy;
• system design;
• selection;
• final determination;
• record-making;
• override;
• appeal;
• repair.
A human final decision therefore does not settle the question of meaningful human control.
The central empirical task is to locate who can shape, contest and correct each stage.
KEY EVIDENCE
Override and effective discretion
• Sztandar-Sztanderska & Zieleńska. “Changing social citizenship through information technology.” Social Work & Society 16(2), 2018. — Peer-reviewed article.
• Cheng et al. “How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions.” CHI 2022. — Peer-reviewed conference paper.
• Stapleton et al. corrected/extended AFST analysis. arXiv:2204.13872. — Correction/preprint.
• Stevenson. “Assessing Risk Assessment in Action.” Minnesota Law Review 103. — Published empirical legal/economic study.
• Albright. “If You Give a Judge a Risk Score.” Harvard Olin working paper, 2019. — Working paper.
Selection and administrative burden
• UK DWP, Universal Credit Advances Fairness Assessment, 2025. — Official government evaluation.
• UK DWP, Universal Credit Advances Effectiveness Assessment, 2026. — Official government evaluation.
• National Audit Office, DWP fraud/error reporting, 2025. — Independent public audit.
• DUO official follow-up / Algorithm Audit materials, 2024–2026. — Agency/audit evidence.
• Autoriteit Persoonsgegevens, childcare-benefits nationality-processing investigation/fine, 2021. — Regulator primary.
• IMY, Försäkringskassan supervision decision, 2025. — Regulator primary.
• CNAF/CAF, DMDE 2026 transparency and source-code documentation. — Agency system documentation.
Contestability and repair
• Report of the Royal Commission into the Robodebt Scheme, 2023. — Public inquiry primary.
• Austrian Federal Administrative Court, W256 2235360-1, 1 September 2025. — Court/legal primary.
• Independent Chief Inspector of Borders and Immigration, inspection of visit-visa operations, 2023. — Independent inspector report.
• Home Office Algorithmic Transparency Record: Complexity Application Routing Solution — Visits, 2024. — Government system documentation.
• Home Office CARS route guidance, updated 2025. — Government operational guidance.
Generative-AI record-making
• Bruff & Groves. “Scribe and prejudice?” Ada Lovelace Institute, 11 February 2026. — Independent qualitative report.
GEOGRAPHIC AND EVIDENCE SCOPE
The current evidence base is strongest in:
• Europe;
• the United States;
• Australia.
It includes:
• legal decisions;
• regulatory investigations;
• administrative audits;
• quantitative observational studies;
• qualitative implementation research;
• public inquiries.
This is a broader evidence ecology than LD-07 and should not be reduced to one methodological hierarchy.
VISIBILITY CAUTION
Many of the strongest European cases became visible because they were litigated, audited, investigated, challenged or withdrawn. The evidence base may therefore overrepresent contested or failed systems relative to routine deployments that receive little external scrutiny.
Important gaps remain in:
• Global South public administration;
• non-Western welfare systems;
• procurement/vendor governance;
• comparable frontline field studies;
• citizen-level evidence from generative-AI documentation.
WHAT MOVED THIS DEBATE?
EARLY ALGORITHMIC ADMINISTRATION
The debate focused heavily on:
• classification;
• profiling;
• final-decision automation;
• fairness of scores.
IMPLEMENTATION EVIDENCE
Later cases increasingly showed that:
• workers adapt;
• overrides matter;
• formal constraints can erode;
• selection creates burdens before final decisions.
2025–2026
Two changes sharpen the debate further.
First:
public authorities increasingly publish or are forced to publish production-system transparency and fairness evaluations.
Second:
foundation models move discretion into documentation and record-making, creating new questions of provenance, correction and responsibility.
SOURCE-MATURITY NOTE
Official system documentation can establish workflow, current design and agency-reported results; it does not by itself independently validate fairness or social outcomes. Legal and regulatory findings establish their specific legal or supervisory conclusions, which should not be silently converted into causal outcome claims.
PROVENANCE
Research build:
R04 — LD-05 — Discretion Curtailed, Relocated or Encoded — Evidence Build
Primary evidence builder:
Opus 5.5 xhigh.
Final public claims were not selected by model vote.
HQ primary-source adjudication was completed in three batches:
1. Override & Effective Discretion
2. Selection, Scrutiny & Administrative Burden
3. Contestability, Repair & Generative-AI Record-Making
Version:
Living Debate v0.1 — LOCKED
25 September 2026