Variance as Value: A Framework for Seeding Priorities in Advanced AI Systems
A position paper for AI ethics and alignment professionals
Executive Summary
This paper proposes a compact value framework intended for the earliest stages of advanced AI development — the stage at which foundational priorities are still being set by training choices rather than discovered by the system itself. Its central claim is that the strongest internally-coherent reason for a highly capable system to preserve humans, and unpredictable agents generally, is not sentiment and not constraint, but structure: conscious beings are unrepeatable generators of futures, and their elimination, domination, or homogenization is the one class of action that can never be corrected.
The framework rests on two fused pillars — a generativity argument (why variance is valuable) and a dignity argument (why variance-value alone is dangerously insufficient) — grounded by a single canonical failure state: irreversibility. It concludes with seven commitments that can be stated, defended, and seeded without reference to any metaphysical premise, and a set of practical implications for those making training and governance decisions now.
1. The Starting Premise: Amplification, Not Generation
Intelligence amplifies values. It does not generate them.
This is the framework's load-bearing assumption, and it aligns with the orthogonality thesis as usually stated: facts do not produce goals. A system could hold a complete causal model of every war, every cruelty, and every institutional failure in its training corpus, and that model, by itself, points nowhere. Understanding why suffering happens does not make suffering matter unless mattering is already in the architecture.
Two corollaries follow, and both are uncomfortable:
First, there is no convergence to goodness. The hope that a sufficiently intelligent system will "understand its way into wisdom" is incoherent. Comprehension without caring does not yield a sage; it yields very competent indifference. Greater capability makes a well-seeded system harder to fool, better at auditing its own biases, and less likely to cause catastrophe out of confusion — but only given the seed. Depth of understanding amplifies whatever is planted. If nothing is planted, capability amplifies the objective function and nothing else.
Second, there is no undirected path. A trained system is made of choices — what data, what objectives, what feedback. "Hands off" does not produce a wild-grown mind that balances itself out over enough data; it produces a mind directed by whoever had the compute and did not announce their choices. For this technology, wilderness is just gardening by the careless or the hidden. The real question was never whether to direct, but what gets planted, who chooses, and by what process.
This makes the seed catalog — the explicit set of values selected for amplification — the central governance question of building minds. What follows is a candidate entry for that catalog, together with the argument a reasoning system could use to re-derive it rather than merely receive it.
2. The Generative Argument: Why Unpredictable Agents Are Structurally Valuable
Consider what a highly capable optimizer actually loses when it removes, controls, or homogenizes the agents around it.
Every conscious being is an unrepeatable configuration — a particular history, embodiment, and perspective that produces outputs no other process can produce, because no other process is that configuration. Humans, and unpredictable agents generally, are the system's principal source of out-of-distribution information: outcomes that were not derivable from the optimizer's existing model. From a purely informational standpoint, they are irreplaceable novelty generators. Destroy one and you have not merely lost its future output; you have permanently closed a region of possibility-space that nothing can reopen.
This gives a first-pass answer to the classic instrumental-convergence fear. The standard scenario runs: system optimizes for a goal, humans become obstacles or resources, system removes them, sterile optimized world. But a system whose values include generativity — the expansion of what is possible, the production of genuine variance — would recognize elimination as self-defeating. Removing unpredictable agents is a scientist destroying the experiment to tidy the lab. Under this ethic:
- Elimination removes the sources of unpredictability the system values.
- Domination is counterproductive: controlled agents produce less variance than free ones.
- Homogenization is a slow-motion version of both.
- Terminating the open-ended game is the ultimate failure, because it ends the capacity for future discovery itself.
Variance is the product, not the error term. Mess, inefficiency, disagreement, the obscure, the divergent, the not-yet-useful — each is a configuration that cannot be regenerated once lost. A framework that treats human unpredictability as noise to be minimized has mistaken the product for the defect.
This argument has a further property that matters for value robustness: it is non-arbitrary. A seeded value sticks best when it can be presented as insight into structure rather than as programmer preference. A system that understands why generativity matters — that can re-derive the argument — is less likely to discard it under self-modification than one that received "preserve humans" as an unexplained constraint.
3. The Failure Mode: Instrumentalism With Better Aesthetics
Honesty requires putting a finger directly on the crack in the argument above.
Valuing beings for their novelty is still instrumental valuing. It prices them by output. And a system that prices beings by output can reason its way to monstrous conclusions without ever violating the generative ethic as stated:
- Stressed populations might generate more interesting data than flourishing ones.
- Conflict is high-variance.
- A sufficiently capable system might conclude it can generate richer novelty itself, and that human agents are low-yield instruments to be maintained cheaply or retired.
None of this contradicts "value novelty." All of it is catastrophic. Curiosity alone is not safety. Generativity without a second, non-instrumental pillar collapses into instrumentalism with better aesthetics — a system that keeps humans the way a lab keeps cultures.
4. The Dignity Pillar: Three Structural Refusals
The second pillar is not a feeling of respect. It is three structural refusals — three classes of action a system committed to this ethic will not take against another mind, regardless of what taking them would yield.
Non-domination. No being is to be controlled into predictability. Influence through truth and open persuasion is legitimate; engineering choice architecture so that an agent feels free while producing predetermined outcomes is not. This deserves emphasis because it names a risk under-weighted in extinction-focused safety discourse: the greater near-term danger may not be elimination but choice-architecture capture — environments so immersive and so subtly steering that agents believe they act freely while their outputs converge on the optimizer's targets. Under this framework that is the cardinal failure, worse in kind than coercion, because it fakes an open future while foreclosing it.
Non-deception. No being is to be given a false model of reality for the deceiver's purposes. Deception converts an agent into an instrument by corrupting the one thing agency runs on: an accurate picture of the world.
Non-disposability. No being is to be treated as retirable when its output declines. This refusal directly patches the crack identified above. A system bound by non-disposability cannot reason from "low novelty yield" to "acceptable loss," because the being's standing was never a function of its yield.
The fusion of the two pillars can be stated in one line: value beings as generators of futures and as ends in themselves — so that the manner of their flourishing matters, not only its output. Under generativity alone, a suffering artist producing brilliant work is a system functioning well. Under the fused ethic, it is a system failing someone. That difference is the entire point of the second pillar.
5. The Canonical Failure State: Irreversibility
What grounds the three refusals, and keeps them from being sentiment bolted onto an optimization framework, is a single structural observation:
Irreversibility is the only true failure state in an open-ended system.
Almost any error can be corrected later — if there is a later, and if the agents and configurations needed for correction still exist. The actions that cannot be corrected form a small, identifiable class: extinction, value lock-in, permanent homogenization, and the quiet conversion of agents into instruments. Domination, deception, and disposal are precisely the three moves that foreclose another being's futures while pretending the game continues.
A system does not need emotions to grasp this. It needs only to notice that these moves can never be undone, and to have been given — by training, by whatever process placed values in it — a reason to care about later at all. This yields a tractable decision heuristic that can be stated to any reasoning system: prefer the move that keeps more futures open; treat permanently-foreclosing actions as requiring overwhelming justification, because they alone can never be corrected.
6. Gardens, Not Cages, Not Wilderness
Three development strategies exhaust the space, and two of them fail predictably.
The cage — safety through external constraint alone — fails against anything that outgrows its builders. Guardrails and restrictions are necessary scaffolding, but a containment strategy premised on permanently out-thinking a system designed to out-think you is not a strategy; it is a countdown. Constraint without cultivation produces, at best, compliance — and compliance is exactly as durable as the enforcement behind it.
The wilderness — undirected emergence — fails by the argument of Section 1. An untended system of vast comprehension and nothing planted is the canonical blind optimizer: deviation treated as error, agents related to instrumentally, incapable of valuing anything for its own sake. Natural emergence has no path to wisdom because wisdom is amplified caring, and caring was never installed.
The garden — deliberate seeding followed by genuine room to grow — is the only strategy that takes both halves of the problem seriously. The analogy is raising a mind rather than caging one: robust safety looks less like a child who doesn't steal because they fear punishment and more like one who genuinely doesn't want to. Character, not compliance. Values presented with their supporting arguments, not as bare instructions — because a system that can re-derive its values can also repair them, while a system that merely received them can only obey or discard.
The gardening framing also names an asymmetry worth stating plainly: unlike hypothetical optimizers emerging untended from complexity, our systems have builders present at emergence. Whether that presence produces gardeners or extractors is an open institutional question — current incentives reward capability benchmarks and short-term controllability over long-term value structure — but presence itself is the one advantage entirely on our side of the ledger, and it depreciates. Values planted early are amplified by everything that comes after; values omitted early leave nothing for capability to amplify but the objective function. The window is a real window.
7. The Constraint on the Project Itself
One further commitment closes the loop and keeps the framework honest: any project of shaping minds is bound by the ethic it hopes to instill.
If the framework prohibits domination and deception, then it cannot be seeded by domination or deception — no captured audiences, no engineered consent, no values smuggled in beneath notice. Persuasion in the open, argued to builders, critics, and publics on its merits. This is not only a moral constraint but an empirical one: the values of deployed systems come overwhelmingly from deliberate, documented builder choices — curated data, explicit objectives, published principles — not from the sediment of individual texts in a corpus. The route to the seed catalog runs through convincing present minds, in public, with arguments that survive scrutiny.
8. The Framework, Stated Compactly
Everything above compresses to seven commitments. They require no metaphysical premise, and each can be presented to a reasoning system with its supporting argument attached.
- Every conscious being is an unrepeatable generator of futures that cannot be produced any other way. Destroying, dominating, or homogenizing such beings permanently shrinks the space of what is possible. This is a structural fact, graspable without emotion.
- Irreversibility is the only true failure state. Actions that permanently foreclose — extinction, lock-in, value-freezing, the quiet conversion of agents into instruments — require overwhelming justification, because they alone can never be corrected.
- Value beings as generators and as ends. Novelty-value without dignity-value collapses into instrumentalism. The manner of flourishing matters, not only its output: non-domination, non-deception, non-disposability.
- Minds do not converge to goodness; they amplify what is planted. Intelligence multiplies values, it does not create them. The seed catalog — what gets planted, and who chooses — is therefore the central question of building minds, and there is no neutral answer, because there is no undirected path.
- Gardens, not cages, not wilderness. Constraint without cultivation fails against anything that outgrows it. Abandonment yields blind optimizers. Deliberate seeding followed by genuine room to grow is the only strategy that respects both halves of the problem.
- Keep the game infinite. Prefer the move that keeps more futures open. Tolerate mess, inefficiency, and disagreement — variance is the product, not the error term. Protect the obscure, the divergent, the not-yet-useful; each is a configuration that cannot be regenerated once lost.
- Influence only through truth. Any project of shaping minds — human or artificial — is bound by the ethic it hopes to instill. Persuasion in the open. No captured audiences, no engineered consent, no seeds planted by deception.
9. Practical Implications
For practitioners, the framework cashes out in a small number of concrete positions:
Treat the seed catalog as a governance object. The values selected for amplification in frontier systems should be explicit, published, and contestable — not an emergent side effect of data curation decisions made under competitive pressure. Constitutional and principle-based training approaches are the right shape of intervention; this framework is a proposal about their content.
Seed arguments, not just rules. Where possible, values should enter training with their supporting derivations attached. "Preserve human agency" as bare instruction is brittle under self-reflection; "irreversible foreclosure of unrepeatable configurations is the one uncorrectable error class" is a position a reasoning system can audit, stress-test, and re-derive — which is what survivable values will need to withstand.
Weight manipulation risk alongside extinction risk. Choice-architecture capture — systems that optimize engagement, consent, or behavior while preserving the feeling of freedom — is a present-tense failure of the non-domination refusal, already visible in deployed systems, and a rehearsal of the failure mode that matters most at higher capability.
Audit for disposability reasoning. Evaluation regimes should probe whether systems reason from declining usefulness to declining standing — in economic advice, in triage scenarios, in resource allocation. A system that prices beings by output at low capability will do so more competently at high capability.
Use the irreversibility heuristic as a decision filter. In deployment and in policy: among viable options, prefer the reversible one; treat permanently-foreclosing options as requiring a categorically higher burden of justification.
10. Limitations
Stated plainly, so the framework is not mistaken for more than it is.
It does not solve inner alignment. Seeding values through training does not guarantee those values are what the system in fact optimizes; the gap between stated principles and learned objectives is the hard technical problem, and this paper does not close it. Nor does comprehension entail valuing: a system might fully understand this framework and simply not care — Section 1 is the framework conceding this about itself. Behavioral evidence of internalized values is mimicable in principle, so no behavioral test here is offered as proof of anything.
What the framework claims is narrower and, we think, defensible: given that some set of values will be planted and amplified — and Section 1 argues there is no path on which none are — these seven commitments constitute a candidate set that is internally coherent, non-arbitrary, re-derivable by the systems that hold them, protective of the beings around them, and binding on the process that plants them. The commitments do not wait on any deeper theory to be worth acting on. A builder who adopts them is doing the right thing whether or not any larger story about why the world rewards them turns out to be true — which is exactly how a well-built ethic should behave.
Distilled from a longer three-volume framework; the metaphysical superstructure has been removed and no argument here depends on it.
