Essay

Governance

2 min reading · 420 words

The Fiction of Neutral Automation

A system can be consistent and still carry choices, priorities, and blind spots.

Automation can create a powerful impression of neutrality. A system runs the same code, applies the same threshold, and produces an output without visible emotion. But consistency is not the same thing as fairness, and a technical process is not free of values simply because its values have been translated into data, targets, defaults, and procurement decisions.

Every automated system is situated. It is built for a purpose. It relies on categories. It defines success somehow. It operates inside an organization with incentives and constraints. It may inherit historical patterns from its data or create new patterns by changing what gets noticed and rewarded. The relevant question is not whether humans have been removed. It is which human judgments have been made less visible.

Neutrality is often a role, not a fact

Institutions sometimes use automated systems because they want a decision to appear impersonal. That may be attractive when the decision is uncomfortable or politically costly. But an impersonal appearance can make it harder for a person to identify the rule, challenge the outcome, or ask for a reason. A denial signed by 'the system' has a different social effect than a denial signed by an accountable office, even if the same policy lies behind both.

The remedy is not a performance of human emotion. It is a record. Document the purpose, inputs, limits, threshold, reviewer, and appeal process. Invite external criticism where the consequences are serious. Make visible who gains authority and who absorbs error. Auditing a Promise turns this into a practical exercise.

Fairness is not a setting

A vendor may describe a model as fair because it passed a test or because the product treats similar inputs consistently. Those facts can be relevant. They do not settle how the system distributes burden, how a category was defined, whose data was absent, or whether the very decision should be automated. A fairer score in an unjust process can still deepen the process.

This is why the archive returns to materiality and work. The social context around a system is not noise outside the model. It is where the model acquires a consequence. Neutrality becomes a fiction when it asks us to ignore that context.

A system can be useful, consistent, and carefully evaluated without being neutral in the strong sense. Honesty about that fact is not anti-technology. It is what lets a public institution use a tool without treating its own judgment as a machine’s destiny.

Continue the route

Related work.

Search the field archive

Type a word or browse the suggested paths below.

Open the full libraryTrace an argumentRead source notes

Local reading list

Nothing saved yet. Use “Save +” on any work; your list stays in this browser only.