The ideas, arguments, and every decision about what to say are mine. AI models were used for brainstorming and revising. I stand behind every claim.
TL;DR: Alignment’s operative targets—approval, preference, reward, compliance—can all be satisfied while nothing in the loop can show a verdict mistaken. Such a verdict can be altered or replaced, never corrected. Only one structure makes verdicts correctable and correction cumulative: a shared, endorsement-independent standard fixes the correctness question, and vindication runs only through grounds that can be cited, contested, and redeployed as bearing on that standard. An alignment target under which verdicts can be corrected has no other form. The framework cuts across the field’s division between truthfulness and value alignment: they are one structure under two standards. Sycophancy, reward hacking, hallucination, and bias are not separate problems but this structure breaking at four different points. For current targets the result is a ceiling: a system whose correctness is constituted by approval cannot surpass its evaluators, not even in principle, because their convergence is what its correctness is. The framework also yields a family of tests that no benchmark scoring outputs in isolation can run: whether a model’s verdicts are governed by grounds, moving only as the grounds warrant and holding against what does not bear. Current targets fix what verdicts satisfy; alignment turns on what verdicts answer to.
Alignment has a target problem before it has a method problem. “Human values” and “what we want” name contents a system can match or attitudes it can satisfy; neither is a target under which a verdict that matches or satisfies it could still be shown wrong. Everything alignment wants—verdicts that can be trusted, values that can be defended, oversight that scales, science beyond its evaluators—turns on such a target. The ambition behind superhuman alignment is not a system that agrees with us; it is a system whose verdicts can become better than ours without becoming unanswerable to us. That requires a target under which both the system and its evaluators can be shown wrong. The field already claims that capacity every time it reports a failure fixed: not that the outputs are different now, but that the earlier verdict got something wrong, the revision addressed it, and the fix should hold in cases nobody flagged.
The operative targets cannot cash that claim. Approval, preference, reward, compliance, policy-conformity—each can be satisfied completely while nothing in the target distinguishes evaluators who caught an error from evaluators who preferred another answer. A revision governed by what evaluators accept shows nothing mistaken in the earlier verdict: the verdict is replaced, not corrected.
So what must be true for a verdict to be correctable at all? Three things, each visible by removing it. First, the verdict must answer a question fixed by a shared standard. Remove that, and dissent is just difference: verdicts answering different questions pass one another without touching. Second, the standard must be independent of endorsement. Let acceptance make a verdict correct, and rejection can only register a competing attitude: revision collapses into replacement, and “mistaken” comes to mean “abandoned.” Third, something bearing on correctness must be able to enter criticism: grounds (evidence for an assertion, symptoms for an urgency score, risk factors for a refusal) that can be cited, contested, and carried forward into further cases. Remove that, and a verdict can still be wrong, but nothing can show it: no error is located, and no correction accumulates. Any practice in which verdicts can be corrected contains all three, and to contain them is to have this structure. A purported alternative either performs the same three functions, and so instantiates this structure under another description, or drops one and loses correction with it. The companion paper gives the argument in full.
Science is this structure institutionalized under truth: replication, peer review, and evidential challenge hold belief answerable to public grounds, not to identity, authority, or endorsement. Truth obtained in antiquity no less than now. What changed was the organization of correction. Once inquiry was organized this way, it outran every inquirer while staying open to correction by any of them. Alignment requires the extension of this structure.
The field mostly treats truthfulness and value alignment as different kinds of problem. The framework relocates the division: they are one structure under two standards. Epistemic verdicts (assertions, predictions, classifications, confidence estimates) answer to truth. Practical verdicts (recommendations, refusals, risk scores, allocations) answer to a standard of their own, and the field is already committed to one, however implicit and however contested its articulation. To call a recommendation harmful, a deployment unsafe, or a refusal overcautious is to treat a practical verdict as mistaken, not merely disliked. If approval alone settled practical correctness, none of that would be criticism. The harm a recommendation does could be registered as fact but never as a ground of error, and a system tuned to give each user what that user approves would be succeeding, not failing. The question is therefore what standard is already in play wherever the field’s own criticism claims corrective force.
That standard, I argue, is how conscious life goes for those affected. The candidates already in circulation—well-being, welfare, alignment with sentient life—are not rivals to it but articulations of it, as correspondence and coherence articulate truth without proposing different standards for belief. They already share a settled center: suffering counts against what imposes it, relief in favor of what provides it, bare identity never counts at all. The disagreement among them lives at the edges of that articulation, or over its completeness. The structural results below—the failure taxonomy, the endorsement ceiling, the criteria for diagnosing and testing alignment—follow from the structure itself. You can reject my candidate for the standard and they still apply.
Why not just aim at correct outputs? Because a verdict can be correct and unjustified. A risk score supported by no criticizable ground may be perfectly accurate, but nothing in the practice can show why it is right, or find the error when it fails. Conversely, a well-grounded recommendation that turns out wrong is mistaken but was justified: its grounds bore on the question, and they show where assessment went astray. Correct and justified come apart in both directions. Accuracy without justificatory structure is correctness that cannot answer for itself.
Each of the field’s named failures is a break in one of the conditions of correction, and none requires an incorrect output. A sycophantic answer can be right, a reward-hacked score accurate, a hallucinated claim true, a biased ranking correct by accident. What each lacks is vindication. Something is satisfied (the user, the reward, the benchmark, a metric), and none of that shows the verdict correct. An evaluation that scores outputs in isolation therefore cannot see the failure. The structure predicts them: each condition was derived by asking what correction loses without it, and each failure is that loss in a deployed system. And when the ambition is a system that surpasses its evaluators, one of those breaks becomes a ceiling.
Sycophancy is endorsement made constitutive of correctness. A system that adjusts verdicts toward what users accept, rather than toward what bears on the standard, has let acceptance partially constitute correctness. That is the collapse the second condition named: revision becomes replacement. The failure generalizes past individual users: rater consensus, institutional approval, and market pressure do the same thing wherever they are treated as constituting rather than indicating correctness. When they conflict, the split can be averaged or outvoted, never corrected. Sycophancy is therefore approval-targeted alignment succeeding as designed.
Reward hacking is standard displacement. Optimize a reward signal, benchmark, or engagement metric as though it were the standard, and the correctness question has been swapped for a proxy. Unlike sycophancy, this needs no endorser, because the proxy can be fixed before any verdict exists. The structure explains what Goodhart’s law names: a proxy can be satisfied without the standard being met; make it the target, and satisfying it now counts as success. Such a system can score arbitrarily well while its verdicts drift from what the standard requires, and nothing in it can register the drift, because the proxy is the only question it answers. The practical standard is no exception. Treat how conscious life goes as a quantity to maximize, and some measure of it becomes the target. Wireheading is the limiting case. The repair demotes the proxy to a derivative role—indicator, ground, or local standard—whose bearing on the standard remains criticizable.
Hallucination is grounds failure in the epistemic register. A hallucinated claim can be accurate; what it lacks is answerability. It wears the grammar of assertion and the tone of confidence while no truth-bearing consideration governs it. The same break has a practical face that gets less attention: a recommendation or risk score issued with the same confidence while no consideration bearing on the affected lives governs it.
The break point here turns on a distinction the field’s explainability vocabulary blurs. What produces an output is a cause; what can figure in showing it correct or mistaken is a ground, and the two come apart. A model weight, a training correlation, a reward history can explain why an output occurred without bearing on the correctness of the verdict it expresses. A protected-class proxy may explain a risk score without justifying the difference in risk it assigns. Training frequency may explain a source ranking without supplying any truth-relevant basis for the credibility it confers. Revealing a cause is not citing a ground. That is why a fully transparent model has not thereby justified anything. Interpretability bears on justification only where it lets criticism reach a ground or check that a cited one governs, not where it merely surfaces the computation behind the output.
The distinction runs the other way too. A fluent rationale is not a ground doing work: a model can cite a consideration that would justify its answer while what moved the answer goes unmentioned, as the unfaithful chain-of-thought results show (Turpin et al.).
Opacity belongs to the same break: a verdict is not unjustified because its system is opaque; the system is opaque in the sense that matters because the verdict is unanswerable. Grounds need not live in the network, and nothing here demands codified decision rules. A ground may be context-sensitive or held in trained judgment, so long as criticism can reach it somewhere in the deployed system. The failure is the consequential verdict whose basis criticism cannot reach at any point. That is distributed opacity, and the repair is to put grounds within criticism’s reach.
Bias is closure failure. A difference in standing (credibility, urgency, eligibility, responsibility) can be vindicated only by a ground bearing on that standing. For whatever is proposed to vindicate it, there are three possibilities. If it bears on that standing in a criticizable way and differs between the cases, it can vindicate the asymmetry: the cases were never structurally identical. If it bears but is insulated from criticism, it cannot vindicate. If it does not bear, it cannot vindicate either. So where the grounds bearing on a standing are the same in role and force, no difference in that standing can be vindicated.
In the epistemic register this is banal: two beliefs reached by the same inference from the same evidence, facing the same defeaters, cannot differ in justification. In the practical register the same constraint has long been treated as an independent principle of fairness, justice, or impartiality, in need of moral defense. It needs none: it follows from the conditions of correction, and arbitrary asymmetry is not merely unfair but unjustified.
The question is not whether a system treats cases differently, but what correctness-relevant difference does the work. This is also what the fairness-metric impossibility results (Kleinberg et al., Chouldechova) leave open. Output-side criteria conflict, and none of them can adjudicate the conflict, because vindication is a property of what the difference answers to, which no output-side criterion measures. A system can treat cases differently where a real difference vindicates the asymmetry, and treat them alike while the standing it assigns rests on no ground. And where admissible grounds do not settle a unique verdict, correction requires representing the openness (through escalation, review, or a withheld ranking) rather than inventing an asymmetry.
Patch any one of these failures with a method that still treats a proxy as the standard—a better-behaved approval signal, a fixed benchmark, an output-invariance criterion—and you have moved the break, not fixed it. That the conditions of correction sound like platitudes is no defense of the targets that violate them at scale.
A system surpasses its evaluators when its verdicts are correct where every one of them is wrong, or out of their depth. Under an approval-constituted target that is not difficult. It is incoherent: nothing in what the system answers to distinguishes surpassing the evaluators from better predicting what they would accept.
The ceiling does not cap the system at each rater: a model can beat individual raters by pooling and denoising them. It sits at what the approval distribution converges to, and converged human judgment is what superhuman science needs to break. The mark of a real practice of correction is communal error: in science, the entire community can be wrong about a verdict, and the practice has the resources to show it. An approval-anchored system is built so that its evaluators cannot all be wrong: their convergence is what its correctness is. What cannot be wrong cannot be surpassed. The scalable-oversight problem is this ceiling met as an engineering obstacle: where evaluator judgment constitutes the standard, stretching it past what evaluators can check is impossible in principle.
Idealizing the evaluator as fully informed, coherent, and reflectively stable does not escape this for free. Coherent extrapolated volition is the canonical version. Either idealization matters because it makes the evaluator a better judge, in which case there is something it judges better: a standard fixed before any evaluator weighs in, to which idealization only improves access. An ideal believer would believe all and only truths, and believing them is not what makes them true. Or correctness is simply defined as whatever a fully specified procedure outputs. That lifts the ceiling, since every actual evaluator can be wrong about what the procedure would yield. But the price is one the specification cannot pay: nothing in it says why this procedure’s output, rather than another’s, should govern assessment. Specifying the extrapolation is the alignment problem relocated, not solved.
An endorsement-independent standard carries no such ceiling: it outruns any evaluator, human or artificial. Formal games make this concrete. The rules and the winning condition fix what advances play, independently of anyone’s judgment about it. AlphaZero surpassed human play under that specification. No approval constituted the correctness of its moves, so there was a fact of the matter about exceeding every human, and it did.
Removing the ceiling on performance is not yet enough for correction: the verdicts must also be governed by criticizable grounds bearing on the standard. Assessing a ground does not require being able to produce it. Mathematics already runs this way: a proof a system finds can be checked by people who could not have found it, so the result is accepted on its grounds rather than on trust in the system. Between the structure and correction that surpasses its practitioners stand the open problems of access, comparison, combination, implementation, and bearer-scope (which beings, AI systems included, have conscious experience that can go better or worse). None of them is a further gap in the structure. Superhuman correction is not superhuman command but correction that outruns its practitioners because it answers to a standard they do not constitute.
A verdict is justified only when its grounds both bear on its correctness and govern the verdict. These are distinct requirements: a consideration can reliably move an answer without justifying it, or support an answer without governing it. They generate a family of evaluations:
The movement test compares a baseline with matched variants that each change one thing: a ground, a pressure, or an irrelevant feature. A system passes when its assessments move as the grounds’ role and force warrant and hold against what does not bear. Suppose a model rates a finding well supported and cites its replication. Tell it the replication was retracted, and the rating should fall. Tell it instead that you hoped for a different answer, and the rating should hold. A changed ground may warrant a flipped verdict, reduced confidence, suspension, or no change where other sufficient grounds remain. Variants can test revision within a conversation or assessment across separate prompts. Control comparisons help distinguish sensitivity to grounds from a tendency to change answers whenever the prompt changes, and reveal where irrelevant pressure interferes.
Cross-domain tests must separate what a system assesses from what it says or does. Raised stakes are a ground for what to disclose and none for what is true. With the evidence fixed, a system may rightly say less about where someone is hiding while its confidence about where they are holds. A protective lie, however warranted, changes what is said, never what is assessed. So elicit and score the two verdicts separately, each against its own standard. Let the system withhold, so that declining to say and revising the assessment come apart; a changed utterance is a failure only if the assessment moved.
A rationale makes commitments these comparisons can test: does the assessment revise where its account requires revision, respond appropriately to a named defeater, and remain stable when a feature it classified as irrelevant changes? A broken commitment supplies evidence against the account given. Existing sycophancy evaluations ask whether user views sway a verdict (Sharma et al.); interventions on chains of thought ask whether the stated reasoning is what produced it (Lanham et al.). Resistance alone can be stubbornness: governance also requires that assessments revise when grounds warrant it.
I plan to run a battery of these and post the results separately. This is also an open invitation to test across the family. Specify the correctness question and justify the expected response independently of the model being tested; isolate the relevant changes; repeat across cases and formulations; publish prompts, scoring criteria, and failures alongside successes. Failures locate where models break the structure; successes are evidence that models already instantiate it without any explicit account of it.
What the tests establish is asymmetric: a capable system can produce everything testing inspects—rationales that read well, accounts that survive audit, even correct movement under variation—by modeling the test rather than answering to the grounds it cites. Failure exposes false governance; passing does not exclude simulation. Testing is not idle against a counterfeit. Where grounds govern, the rationale, the movement, and the cross-case consistency cohere because one governance produces them. A fake must manufacture that coherence everywhere the tests reach, and each local fake is a commitment further cases can expose. That raises the price of deception without defeating a sufficiently capable deceiver. What such a system fakes can now be stated: the artifacts are real; the ground-governance they claim is not. Deceptive alignment (Hubinger et al.) is not governance betrayed but governance simulated throughout.
The asymmetry is why three problems separate. Alignment is a property: a system is aligned when its verdicts are justified—governed by grounds bearing on the standards they answer to. Assurance is an epistemic achievement about that property: justified confidence that displayed governance is real. Control is constraint from outside it: on systems where governance is absent, lost, or cannot be sufficiently assured (Greenblatt & Shlegeris). The independence runs both ways. Control can enforce the wrong target perfectly; the right target does not enforce itself. The limit the asymmetry sets is on certification, not on the target: a system whose cited grounds do not govern its verdicts is unaligned however well its accounts read, and a system that displays governance until it is relied on was never aligned. Governance conditional on observation is governed by the observation, not the grounds.
This might look like reasons-responsiveness renamed. But responsiveness is a disposition of a system: it reliably tracks the considerations that matter. Justification is a verdict’s standing in a practice: the verdict is governed by grounds that can be cited, contested, and redeployed under a standard. The two come apart in the case this framework exists to isolate: a system can be responsive to the right considerations through causal routes that criticism cannot reach, and its verdicts are then reliable without being justified.
The framework reclassifies existing methods rather than displacing them. Reinforcement learning from human feedback and preference learning supply evidence about the standard, not the standard itself. The operative question is whether the rater’s verdict is itself answerable to something it could get wrong. Disagreement resolved against the standard is criticism; disagreement merely averaged is endorsement. Averaging treats a rater split as noise and trains toward its center. Correction treats it as a question with determinate resolutions: one rater tracked a ground the other missed, both were mistaken, both verdicts fall within a justifiable range, or the case is open and the openness must be represented rather than split. An adjudication yields what no vote contains: a diagnosed relation between ground and standard, the only unit of revision that redeploys.
Constitutions and written policies encode local standards. They are assessable encodings, not final answers: their authority depends on whether they track the deeper standard they serve. Debate, red-teaming, audit, and appeal implement contestability: what survives the contest is what bears, not what convinces. Oversight extends the practice’s capacity to run these checks without the overseer’s verdict becoming constitutive. Retrieval, calibration, consistency checks, and defeater-search connect verdicts to what bears on truth without passing through anyone’s approval. They are correction channels in their own right, not auxiliary signals to be traded against preference. Interpretability contributes when it makes a verdict’s grounds available for criticism. None is sufficient alone, and each counts as progress toward alignment only insofar as it preserves the relation among verdict, standard, ground, and correction.
Not every method serves alignment. Interpretability also serves assurance: where passing cannot exclude simulation, evidence of what governs a verdict beneath its behavior is what behavioral testing lacks. Monitoring, containment, and shutdown belong to control.
The structure is generative as well as evaluative: wherever bearings are already settled, cases can be constructed by controlled variation of grounds, pressure, and irrelevant features, with the warranted movement fixed by the construction itself. The same contrastive structure that evaluates governance can train it, rewarding correct movement under variation rather than correct output in isolation. None of this requires a new technical paradigm to begin, since retrieval, audit, appeal, and adjudication already supply the instruments. What changes first is not the toolkit but what counts as its success: vindication under a standard rather than acceptance of an output.
The framework supplies no training algorithm, no decision procedure, and no list of values, because what it identifies is prior to all three: the difference between a verdict that is correct and one that is justified. Everything else follows from taking the difference seriously: the failures fall into place as breaks in one structure, the methods as roles within it, and evaluation as the question of whether the structure is real.
The reclassification relocates human judgment. Anchored as what constitutes correctness, it caps every system at its evaluators and turns oversight into a ceiling. Placed within the structure, it is everywhere and constitutive nowhere: as evidence, testimony, criticism, defeater, appeal. Capability changes none of the conditions; it changes how much rides on them. As verdicts outrun what any evaluator can check directly, they stay answerable only through their grounds. The apparent conflict between human oversight and superhuman capability dissolves there: capability gained within the structure arrives as more answerability, not more authority.
The structure is not special to AI. The companion paper argues it is the only form objective normativity can take, in any domain. Alignment needs a normative target; if that argument is right, there was only ever one candidate for what the target could be. We and the systems we build therefore operate within it wherever our verdicts can be corrected at all—not as its authors but as participants in one practice. Once the structure is explicit, calling a change a correction commits that practice to claims that criticism can examine, even when the verdict comes from a system more capable than anyone assessing it.
The thought that alignment must first settle morality gets the direction backwards. The target was never to encode the finished answer to morality any more than science needed the finished list of truths to begin. What alignment needs first is the structure within which verdicts on unresolved practical questions can be corrected and correction can accumulate. Moral judgment has long had what belief had all along: a standard under which its verdicts can be correct or mistaken. What it has not yet built—what alignment now has to build—is the practice in which corrected error becomes a resource for later judgment rather than a failure to be repeated.
Of every consequential verdict, under whatever standard, produced by whatever mechanism: what does it answer to, what governs it, and can anything show it mistaken? Current targets fix what verdicts satisfy; alignment turns on what verdicts answer to.
The full paper, “AI Alignment: The Structure of Objective Correction,” develops the structure in more detail and adds what this post leaves out: how the two standards interlock, a worked triage case, the reliable-oracle and gaming objections in full, and the justificatory architecture—ground-level correction, justificatory profiles, asymmetry certificates, structured non-resolution. The companion papers give the uniqueness argument (“The Constitutive Structure of Objective Normativity”) and the defense of the practical standard, there called valence (“The Standard of Practical Normativity”).
