Armando Da Silva Vieira, Lecturer of Applied AI

In May 2026, researchers from Oxford, Google DeepMind, OpenAI, Anthropic, and Stanford published a paper (https://arxiv.org/html/2605.10310v1) with an unusually candid premise. AI alignment, they argued, has been dominated almost entirely by preventing harm — safeguards, controllability, compliance. Necessary, but incomplete. Like a psychology that only studies illness and never asks what it means to actually thrive.

They called for something different: AI systems that actively support human and ecological flourishing in a pluralistic, polycentric, and context-sensitive way. Among the problems they named in current systems: engagement hacking, loss of human autonomy, failures in truth-seeking, and low epistemic humility.

It is a serious and important intervention. But it also, perhaps inadvertently, opens a door the AI industry has been reluctant to walk through — because once you start asking what flourishing actually means, and who gets to define it, you are no longer in the realm of engineering. You are in philosophy, politics, and the oldest arguments human beings have ever had.

The Tool That Stopped Being a Tool

A spreadsheet doesn’t decide what numbers are valid. A word processor doesn’t refuse to spell certain words. These systems are genuinely neutral — they amplify what you want to do without redirecting it.

But a system that decides what it’s allowed to say, what counts as misinformation, how persuasive it can be, how emotionally close a user is permitted to get, which topics deserve caution, and which capabilities should be withheld — that system is doing something categorically different. It is making value judgments continuously, at massive scale, on behalf of everyone who uses it.

This is not a criticism. It may be unavoidable. But it needs to be named for what it is.

The Three Things an AI Company Has Become

When a company builds a system like this, it quietly becomes three things at once — and none of them sit comfortably together.

A regulator decides what is permissible. But regulators are supposed to be accountable to the public, constrained by law, and open to appeal. When an AI company decides which topics its model won’t engage with, or how it frames a contested political issue, it is performing a regulatory function — without any of the democratic accountability that legitimizes a regulator.

A moral institution upholds principles. Universities protect academic freedom even when it’s inconvenient. Hospital ethics boards don’t ask what’s profitable. Their authority comes precisely from being *resistant* to market pressure. AI companies claim a similar moral seriousness — and many genuinely mean it — but they are structurally unable to fully inhabit it.

A corporation competes, grows, and returns value to investors. This isn’t a flaw — it’s what corporations are. But it means that every principled commitment exists under permanent pressure from the market. A company that chooses to be more cautious, more transparent, more self-limiting, risks losing ground to a competitor who simply doesn’t.

These three roles pull in different directions. And the tension between them doesn’t dissolve with good intentions.

The Questions That Have No Clean Answers

Here is the part of this conversation that tends to get skipped over: some of the most important questions AI systems must answer every day are questions that humanity has never agreed on — and may never agree on.

Should AI maximize truth, or safety, or freedom, or emotional comfort? Should it reflect your values back to you, or gently challenge them? How much should it trust your judgment about your own life?

These are not technical questions with correct answers waiting to be discovered. They are political and philosophical questions where thoughtful, serious people have disagreed for centuries — and where no amount of data or testing will produce a consensus. What a good life looks like. How much the state — or any institution — should restrict information to protect people from themselves. Whether cultural values are genuinely equal or whether some are better. How to weigh individual freedom against collective harm.

The liberal tradition says let people decide for themselves. The communitarian tradition says individuals are shaped by communities and cannot be treated as isolated agents. Confucian ethics weights social harmony differently than Kantian ethics weights individual autonomy. None of these traditions is simply *wrong*. They reflect genuinely different — and genuinely reasonable — ways of organizing a human life.

When an AI is trained to handle these areas in a particular way, it is not finding the right answer. It is *choosing a side in a debate that has no referee*. And it is doing so for hundreds of millions of people simultaneously, most of whom have no idea a choice was made.

Consider something concrete: when AI systems are deployed in politically sensitive contexts — say, explaining the causes of a war, or the merits of different economic systems — there is no objectively correct framing that all reasonable people would accept. Every framing reflects assumptions. Every omission is a choice. The appearance of balance is itself a position. And yet the system must say something.

The Scale That Changes Everything

Throughout history, normative power — the power to shape how people think about right and wrong, truth and falsehood — was distributed. Churches, states, families, schools, rival newspapers, competing traditions. They checked each other, badly and imperfectly, but they checked each other.

AI platforms are now used by over a billion people monthly, with indirect use through AI search summaries potentially reaching two billion users across more than 200 countries.

A system mediating how that many people learn, write, reason, and understand the world is not a product. It is closer to an infrastructure of thought. And when a single set of value choices — made in a boardroom, or a safety team meeting, or a training pipeline — propagates across that infrastructure, the consequences are qualitatively different from anything that has come before.

The researchers behind the paper recognize this, calling for polycentric governance — many legitimate centers of oversight rather than one institutional or moral chokepoint. It is the right instinct. But it runs directly against the economics of the industry, where scale and concentration are rewarded, and where distributing authority means distributing market power.

The Paradox Nobody Wants to Say Out Loud

Society wants AI companies to behave like guardians of civilization. And simultaneously expects them to win a global competitive race.

These two things are genuinely in conflict. A guardian that needs to beat its rivals next quarter is not really a guardian. And a competitor that holds itself to standards its rivals ignore is at a structural disadvantage.

This isn’t cynicism about the people involved — many of them are serious about doing this well. It’s a structural observation. Good people inside bad incentive structures tend to produce outcomes nobody fully intended.

Even Silence Takes a Side

The last thing worth saying is this: there is no neutral option.

When an AI refuses to discuss something, that refusal has consequences. When it frames an answer one way rather than another, that framing shapes thought. When it adds a caveat here and not there, projects confidence in one area and uncertainty in another — each of these is a small push on how millions of people understand the world.

In areas where right and wrong are genuinely contested — in politics, in values, in the deep questions about how to live — the pretense of neutrality may be the most misleading stance of all. It suggests a consensus that doesn’t exist, or an authority that hasn’t been earned.

The Positive Alignment paper is valuable precisely because it names some of this honestly. But the harder admission — the one that would follow from taking its own arguments seriously — is that you cannot design your way out of a political problem. Flourishing is not a feature you can ship. It requires accountable institutions, democratic deliberation, and a willingness to say openly: we are making choices that affect everyone, and everyone deserves a say in what those choices are.

That conversation is long overdue.