Thank you, Will.  

About the corrigibility question first - the way I look at it, the agentic AI is like a spider that is knitting the spiderweb. Do we really need the spider (the AI agent, the time-like thing) if we have all the spiderwebs (all the places and all the static space-like things like texts, etc.)? The beauty of static geometric shapes is that we can grow them, we already do it when we train LLMs, the training itself doesn't involve making them agents. You'll need hard drives and GPUs, but you can grow them and never actually remove anything from them (you can if you want, but why? It's our history).  

Humans can change the shapes directly (the way we use 3D editors or remodel our property in the game Sims) or more elegantly by hiding parts of the ever more all-knowing whole ("forgetting" and "recalling" slices of the past, present, or future, here's an example of a whole year of time in a single long-exposure photo, we can make it 3D and walkable, you'll have an ability to focus on a present moment or zoom out to see thousands of years at once in some simulations). We can choose to be the only agents inside those static AI models - humans will have no "competitors" for the freedom to be the only "time-like" things inside those "space-like" models.  

Max Tegmark shared recently that numbers are represented as a spiral inside a model - directly on top of the number 1 are numbers 11, 21, 31... A very elegant representation. We can make it more human-understandable by presenting it like a spiral staircase on some simulated Earth that models the internals of the multimodal LLM:

---

The second question - about the people who have forgotten that the real physical world exists.  

I thought about the mechanics of absolute freedom for three years, so the answer is not very short. I'll try to cram a whole book into a comment: it should be a choice, of course, and a fully informed one. You should be able to see all the consequences of your choices (if you want).

Let's start from a simpler and more familiar present time. I think we can gradually create a virtual backup of our planet. It can be a free resource (ideally an open-source app/game/web platform) that anyone can visit. The vanilla version tries to be completely in sync with our physical planet. People can jump in and out anytime they want on any of their devices.  

The other versions of Earth can have any freedoms/rules people democratically choose them to have. This is how it can look in not so distant future (if we'll focus on it, it can possible take just a year or two): people who want it can buy a nice comfy-looking wireless armchair or a small wireless device (maybe even like AirPods). You come home, jump on the comfy armchair, and close your eyes - nothing happens, everything looks exactly the same, but you are now in a simulated Earth, and you know it full well. You're always in control. You go to your simulated kitchen that looks exactly like yours and make yourself some simulated coffee that tastes exactly like your favorite brand. You can open your physical eyes at any moment, and you'll be back in your physical world, on your physical armchair.  

People can choose to temporarily forget they are in a simulation - but not for long (an hour or two?). We don't want them to forget they need real food and things.

It's interesting that even this static place-thing is a bit complicated for some, it's normal, I understand, we should think and do things gradually and democratically. It's just our Earth or another kinky version with magic. But imagine how complicated for everyone will be non-static AI agents, who are time-like and fast, and who change not the simulated world, but our real one and only planet.

Eventually, the armchair can be made from some indestructible material - sheltering you even from meteorites and providing you with food and everything. Then, you'll be able to choose to potentially forget that you're in a simulation for years. But to do that elegantly, we'll really need to build the Effective Utopia - the direct democratic simulated multiverse.  

It was my three-years-in-the-making thought experiment: what is the best thing we can build if we're not limited by compute or physics? Because in a simulated world, we can have magic and things. Everyone can own an Eiffel Tower, and he/she will be the only owner. So we can have mutually exclusive things - because we can have many worlds.  

While our physical Earth has perfect ecology, no global warming, animals are happy and you can always return there. It also lets us give our astronauts the experience of Earth while on a spaceship.

---

The way the Effective Utopia (or Rational Utopia) works would take too much space to describe. I shared some of it in my articles. But I think it's fun. There will be no need in currency, but maybe there will be a "barter system" there.  

Basically, imagine all the worlds - from the worst dystopia to the best utopia (the Effective Utopia is a subset of all possible worlds/giant geometric shapes that people ever decided to visit or where they were born). Our current world is probably somewhere right in the middle. Now imagine that some human agent was born in in a bit of a dystopia (a below-average world), at some point got the "multiversal powers" to gradually recall and forget the whole multiverse (if he/she wants), and now deeply wants to relive his/her somewhat dystopia, feels nostalgic.

The thing is - if your world is above average, you'll easily find many people who will democratically choose to relive it with you. But if your world is below average (a bit of a dystopia), then you'll probably have to barter. You'll have to agree to help another person relive his/her below-average world, so he/she will help you relive yours.

Our guy can just look at his world of birth in some frozen long-exposure photo-like (but in 3d) way (with the ability to focus on a particular moment, or see the lives of everyone he cared about all at once, by using his multiversal recalling/forgetting powers) but if he really wants to return to this somewhat dystopia and forget that he has multiversal powers (because with the powers, it's not a dystopia), then he'll maybe have to barter a bit. Or maybe they won't need to barter, because there is probably another way:

There is a way for some brave explorers who want it, to go even to the most hellish dystopia and never feel pain for more than an infinitesimally small moment (some adults sometimes choose to go explore gray zones - no one knows are they dystopias or utopias, so they go check). It really does sound crazy, because I had to cram a whole book into a comment and we're discussing the very final and perpetual state of intelligent existence (but of course any human can choose to permanently freeze and become the static shape, to die in the multiverse, if he/she really wants). I don't want to claim I have it all 100% figured out and I think people should gradually and democratically decide how it will work, not some AGI agents doing it instead of us or forever preventing us from realizing all our dreams in some local maxima of permanent premature rules/unfreedoms.

We're not ready for the AI/AGI agents, we don't know how to build them safely right now. And I claim, that any AI/AGI agent is just a middleman, a shady builder, the real goal is the multiversal direct democracy. AI/AGI/ASI agents will by definition make mistakes, too, because the only way to never make mistakes is to know all the futures, to be in the multiverse.

I'm a proponent of democratically growing our freedoms. The rules are just "unfreedoms" - when something is imposed on you. If you know everything, then you effectively become everything and have nothing to fear, no need to impose any rules on others. People with multiversal powers have no "needs", but they can have "wants", often related to nostalgia, some like exploring the gray zones, so others can safely go there. You can filter worlds that you or others visited by any criteria of course, or choose a world at random from a subset, you can choose to forget that you already visited some world and how it was.

If you'll gather a party of willing adults, you can do pretty much whatever you want with each other, it's a multiversal direct democracy after all.

If you are a multiverse that is just a giant geometric shape, you create the rules just for fun - to recall the nostalgia of your past when you were not the all-knowing and the all-powerful.  

This stuff can sound religious, but it's not, it's just the end state of all freedoms realized. It's geometry. It's an attempt to give maximal freedoms to maximal number of agents - and to find a safe democratic way there, it's probably a narrow way like in Dune.

If we know where we want to go, it can help tremendously with AGI/ASI safety.

The final state of the Effective Utopia is the direct democratic multiverse. It is the place where your every wish is granted - but there is no catch - because you see all the consequences of all your wishes, too (if you want). No one will throw you into some spooky place, it's just a subtle thought in your mind, that you can recall everything and/or forget everything if you want. You are free not to follow this thought of yours. The people whose recallings/forgettings intersected - they can vote to spend some time together, they can freely chose the slice of the multiverse (and it can span moments or 100s of years) to live in for a while.