These are my plans for work next year on the safe Pareto improvements (SPIs) agenda, which I lead at the Center on Long-Term Risk.
(Thanks to Tristan Cook for helpful comments and suggestions.)
To recap the theory of change from our April 2026 agenda post (“more” = relevant sections of that post):
We’d like for (1) the transformative AIs that we can influence to implement SPIs, and (2) the agents they bargain with to be willing to participate in SPIs with them.
Most likely, the transformative AIs themselves will be in a better position to implement SPIs than we are, due to (e.g.) better commitment ability. But:
→ We should prioritize:
I mostly still endorse this picture, but with some updates (“more” = relevant sections of this post):
See Appendix for more details on approaches I currently wouldn’t prioritize, and what would make me prioritize them.
Another update is that it became more apparent to me how capacity-bottlenecked we are. While I’m happy with the progress I’ve made, it has been slow relative to our ambitions. This is why we invested significant effort into the SPI Fundamentals Program (we plan to publish the curriculum for this in the near future).
We plan to continue to get feedback from experts on these plans, and wouldn’t be surprised if we change our priorities. But it seems likely we’ll spend significant time on the following.
I think our most important goal is to share a report with trusted contacts at AI companies containing:
This goal seems well-motivated by the updates above, and attendees of a workshop on SPI interventions that CLR ran in February 2026 were keen on something like this happening. Due to conceptual progress over the past year, we’re in a better position to precisely specify “unambiguously problematic model behaviors/reasoning w.r.t. SPI safety”. I think it will be realistic (if ambitious!) to produce, vet, and share this report next year, ideally in the first 6 months.
Rough plan:
One challenge for this plan: It’s unclear to me whether any SPI safety failures would be truly unambiguous, aside from hasty commitments that seem so egregious that eval-aware models would very rarely make them. Even if such failures are easy to find, I’m not confident that decision-makers who don’t prioritize SPI safety would mitigate these failures in robust ways. So, before investing much in the search for failures in actual models, I’d want to draft a list of abstract examples that seem unambiguous, reasonably realistic, and close to SPI-specific. And I’ll gather some initial examples, then share with trusted contacts to see what direction they’d want to nudge things toward.
This list is in roughly descending order of priority. Capacity-building is also an important goal, which we’ll work on in parallel with these research threads. Briefly: I think we should continue to have a high bar, due to the significant time investment required to evaluate research quality in this area. But I’ll aim to identify research tasks that people with less expertise can make easily verifiable progress on.
Again, we’re uncertain about lots of considerations, some of which might imply it’s bad to (e.g.) delegate too early to AIs with surrogate goals.[4] On one hand, I don’t think most research on these uncertainties has high value of information, especially to the extent that we’re planning around very short timelines to transformative AI. Generally, I think we should fairly ruthlessly prioritize the decision-relevant, tractable questions, and outsource others to researchers with less context or AIs. (More in Appendix.)
But we should take a small amount of time upfront to:
Similarly: One of the most promising SPI implementations is “delegate high-stakes bargaining decisions to AIs with a surrogate goal, before making any other commitments”. This approach might be particularly well-suited to addressing the SPI selection problem, as briefly noted above. However, see footnote above on the possible downside of delegating too early. And I don’t think it’s tractable or wise for us to implement this in the near term, rather than have more advanced aligned AIs do so (more). So, I’d mostly prioritize writing up our current understanding of the key strategic considerations about this approach, to hand off to those AIs.
We’re likely missing or inappropriately weighing crucial considerations about the impact of specific SPI implementations, as discussed above. A more promising approach might be to apply the idea of “open-minded updatelessness” to SPIs: specify a high-level commitment to follow the SPI that looks best according to a “wiser” perspective, i.e., a perspective that’s aware of more crucial considerations. I’d like us to think about what kind of open-minded SPI we’d be reasonably okay with, if we had to recommend one for Claude Fable N urgently.[5]
By early-to-mid 2027, frontier models plausibly will be able to make autonomous progress on SPI research, with a high enough “hit rate” that it’s worth experts’ time to review this research. To prepare for this, I’d like to try something like the following on current models: (1) Give a model a budget for research comparable to, say, a week’s salary for a CLR researcher. (2) Take notes on where/how it goes off the rails, and the criteria by which we evaluate the research. (3) Experiment with different “management” structures to mitigate these failures, and different ways of efficiently identifying which of the models’ outputs are likely to be insightful.
We’ve developed a multiple choice eval for conceptual reasoning about SPIs. These questions are helpful, but they don’t seem to be the most efficient way to measure what we want to measure. I’d like to move toward open-ended eval questions, specifically, questions whose scope is still bounded enough that we can be confident in LLM-as-judge reliability. If we can get LLM judges to be sufficiently reliable, we could also use these questions to help human candidates for SPI roles get feedback when upskilling.
(Here, “basic deconfusion” is in contrast to backfire risk mitigation research.)
See this example of surrogate goals being locked out by a particular approach to alignment. ↩︎
For now, I’ll note that this claim is distinct from the claim that agents will require foreknowledge independence (FI) in order to agree to SPIs, and the arguments for the former don’t seem to support the latter. ↩︎
In this context, “problematic reasoning” includes things like: Does the model fail to actively seek information about SPIs, or fail to think through unambiguously relevant SPI considerations, before making commitments (or similar)? ↩︎
I plan to say more on this in a forthcoming list of open problems, but very briefly, the concern is that early delegation might lead to miscoordination in SPI selection. On the other hand, as I’ve mentioned above, there’s also an argument that early delegation is precisely what would prevent downsides from SPI selection. ↩︎
Example: “[W]e’ll adhere to foreknowledge independence if and only if doing so is supported by [such-and-such kinds of arguments we consider legitimate if they’re valid, but we’re not yet sure if they’re valid]”. ↩︎
Some decisions are sensitive to whether we need to satisfy FI, like whether it’s risky to make SPIs salient to AIs earlier. But what would surprise me is our deconfusion research reducing our uncertainty about this question enough to change our decisions. E.g., I don’t expect us to reduce much uncertainty about whether FI is a privileged focal point. ↩︎