I remember that when I first became interested in alignment there was this belief that in order to make alignment progress, you needed to be a genius. With the rise of interpretability, this expectation has moderated somewhat, but I still think it has influence and that it often leads people to overcomplicate.
Many recent alignment breakthroughs have been found by keeping it simple:
Lie Detection:
• Contrast-consistent search (better known as discovering latent knowledge) reads the truth out of a model without looking at the outputs, simply by using a linear probe with only two elements in its activation function.
• Pacchiardi, Owain and others use a logistic regression classifier to predict whether a mis al was truthful from its answers to further questions. They were able to demonstrate this using three different kinds of questions and the results even generalised across models. This technique seems like it could be useful in practise, even if it doesn't scale all the way to superintelligence.
Superposition: Dictionary learning makes substantial progress on superposition by simply using a one hidden layer multi-layer perceptron as a sparse auto-encoder with L2 reconstruction loss and a L1 loss on hidden layer activations. Anthropic actually chose to avoid more sophisticated methods, as they were worried they might recover features that the model doesn't actually utilise.
Model-steering: Activation Engineering: Activation addition allows us to steer models by simply performing a forward pass for the concept you want to activate, saving the activations of a particular layer as a vector and adding them to future forward pass. See also Turntrout's prior work on steering a mouse by subtracting a cheese vector and inference-time intervention.
World model location: Neel Nanda found a linear representation in Othello-GPT. This one was a bit trickier: a previous paper had failed to recover it using linear probes. However, Neel managed to do this by searching for a representation of "my color" and by applying a bunch of other tweaks. See also Wes Gurnee's work with Max Tegmark, which used linear probes to find linear representations of latitude and longitude; in addition to one representing time.
Scalable Interpretability: OpenAI used a language model in order to label neurons based on the activations for some relevant examples using few-shot prompting.
Understanding Implications: For a long time it appeared as though it would be challenging to prevent an AI from taking our instructions too literally. Essentially what seems to have happened is that they decided to just use RL on a generative base model and the problem mostly solved itself. Reinforcement Learning from Human Feedback is admittedly complex - proximal policy optimization isn't simple - but I still think it's quite notable than OpenAI, seems to have almost one-shotted it with InstructGPT. Even if they hadn't found a solution the first time, it seems likely that we were always going to end up there by iterating on a baseline.
I don't want to dismiss the challenges of such research. Even if you have the general idea, making it work requires getting a lot of details right. Verifying it works takes a lot of work too. Even picking the right question is hard. However, simpler techniques have gone further for these problems than I would have originally expected.
I think it's worthwhile speculating why reasonably simple solutions have worked for these problems, and checking whether there are any similarities between them.
Firstly, I think it's worth noting that all of these techniques except for the last two look for linear representations in neural networks. Beren has made a strong case that there is strong evidence of deep learning models being almost linear. In retrospect, it seems intuitive to me that neural networks would store sparse features in linear combinations, rather than with some more complicated form of compression, as this makes it easy to write to and read from these features.
In contrast, the last two techniques listed rely on current AI models being very powerful and quite steerable. I admit that two examples aren't very many, but I expect we'll see more in this vein soon.
Looking more broadly, I think it's worth noting that all of these results are in empirical alignment research. I don't think this is a coincidence. Whilst there may be simple solutions to some of the problems of agent foundations, the problems have been around for enough time that if there were simple solutions, they likely would have been found by now. In contrast, the last two directly rely on a certain level of capabilities and the linear representation techniques arguably rely on this indirectly as well. To wit: it is beneficial for a model to have a linear representation of particular features, the more powerful our optimization techniques, the more likely our trained models are to produce such a representation.
An Aside on Contrast-Consistent Search:
When I first encountered this technique, I was awed by it and I felt like I had no conception of how anyone would ever think of something like this. While I still acknowledge the brilliance of this work, it no longer feels completely unthinkable.
I assume Collin already had some intuitions that neural networks were vastly more linear than you might expect. If you buy this and you're trying to read the truth directly out of the network weights, it's then entirely natural to look for a linear direction.
Linear probes were an already existing technique, so if you knew of this technique and you were making a shortlist of which techniques to investigate, this would make it onto the shortlist.
Suppose you end up in the position of asking "how can I use a linear probe to find a direction corresponding to truth?" in a neural network. It'd then be natural to ask what some of the properties of truth are, which would then lead directly to the consistency property Collin leveraged.
So, smart, brilliant even, but not completely unthinkable, if you ask the right questions.
And even just knowing to look for linearities would be a massive headstart.
Final thoughts:
I suspect that all of these approaches are still very far away from where we need to be. I consider them substantial advances nonetheless for two key reasons: having a baseline helps people choose an appropriate level of ambition, and also makes allows us to empirically discover the key bottlenecks to a fuller solution.
I'm hoping that increasing awareness that relatively simple techniques have been successful spurs more research progress, by giving people the confidence to actually try to make progress and increasing people's motivation to explore these kinds of approaches for longer before they toss in the towel.
It may very well be that we're just in a particular stage of the field where there's substantial this low-hanging fruit to pick. Perhaps we'll soon pick all the low hanging fruit such that we end up requiring brilliant conceptual breakthroughs for further progress. However, we aren't there yet and I think it's a mistake not consider simple paths forward on the basis that if they were fruitful then someone would have already realised this.
