Graphical abstracts

Summary

A true human whole brain emulation would be very helpful to humanity. The WBE could increase their own intelligence through self-modification and then somehow prevent AGI from killing everyone. However, if a research project made any serious progress towards WBE, it would likely contribute to existential risk from AI by contributing to AI capabilities progress.

Here is the argument that a successful WBE research project would accelerate AI capabilities:

• Creating a WBE is very hard. Therefore, it's very unlikely that a research project, unless extremely well-resourced, could reliably create a WBE very quickly (in less than five years, say).
• There is a wide spectrum of difficulty within the set of problems building up to WBEs. Some are easy, some are pretty difficult, some are very difficult, some are extremely difficult. Therefore, a project is likely to make some initial progress partway to WBEs, and then to stall and not quickly get all the way to WBEs.
• Therefore, it's very likely for a WBE project to spend a significant amount of time having already made partial progress, but not being very close to WBEs.
• If a project makes serious partial progress towards WBEs, it will have produced artifacts (whether data, methods, apparatuses, ideas, algorithms, or running simulations) that enable one to partially emulate some powerful aspects of the human brain. In other words, it is not feasible to eventually get a computer emulation of all the immensely powerful algorithms of the human mind, without, in the interim between here and there, being able to create "partial brain emulations" that embody some but not all of the powerful algorithms of the human mind.
• The AI capabilities research sector would probably succeed at expropriating those "partial brain emulations" and using them to accelerate AI capabilities research.

In other words, solving approximately everything about WBEs would be great. Solving anywhere between 5% and 95% of the problems leading up to WBEs would be quite bad. The latter is much more likely to happen from a given WBE research program, if it makes much progress at all. It's much easier to get some capabilities out of the brain and into the computer than to get all of them into the computer.

This situation induces a bad "knee curve" of utility for WBE research: It's net-negative up until it fully succeeds, and it's difficult to fully succeed. Thus, WBE research is net-harmful by default.

Prior art (not necessarily in concurrence):

• Byrnes 2020: "Building brain-inspired AGI is infinitely easier than understanding the brain"
• Byrnes 2023: "8 examples informing my pessimism on uploading without reverse engineering"

Caveats

I don't imagine this article will materially impact too much. But if it did, I would think that impact to be at significant risk of being mistaken, unless there was appropriate pushback somewhere (here in the comments or elsewhere).

I state arguments fairly strongly in this article. My hope is to do this in the spirit of debate, rather than shutting anything down. I stand by the arguments I make here, but this is coming from mostly recently formed opinions. In particular, I haven't spent a lot of effort trying to find a workable WBE plan that responds to the critique I make here; maybe such effort would bear fruit.

I'm not remotely close to being an expert in neuroscience.

Some of the ways this article could be wrong

There are some settings of background strategic variables that would make WBE research good even if everything argued in this article is true. For example, if you think all of:

• AGI is very likely very near, even given all efforts to stop it.
• There's a nonnegligible chance of making the research "leap" all the way to WBEs in a few years.
• It's not worth changing course based on unilateralism / cooperation concerns saying not to add to AI capabilities even when it seems like a good idea, because you degrade our ability to stop AGI research.

Then a crash project to get WBEs could make sense.

It could also be that making a quasi-WBE that emulates electrical structure only (no learning) is already useful and pretty easy.

It could be that some kinds of "WBE research", especially ones such as connectomics that don't try to reverse engineer the brain's learning algorithm, are net-helpful. See Byrnes 2023: "Connectomics seems great from an AI x-risk perspective". I a little bit believe this regarding connectomics. I don't strongly believe this about connectomics or about any research, a priori, because either it doesn't lead to true, extremely helpful, human WBEs, or else it does and it's powerful and therefore it throws off powerful PBEs on the way there. However, see the next paragraph:

It could be that Flourish or another group will have already cracked the dangerous brain algorithms, but not yet have finished turning that into a world-destroying AI. In that case, a WBE project couldn't add much dangerous information, so it could be worth trying to race and get a WBE by filling in the human connectome and other needed information. (It's hard to imagine that information not being dangerous, because it would unlock a powerful WBE. However, it may be more feasible to get a WBE quickly in this scenario, making a siloed project more feasible.)

It could be that PBEs are much less expropriatable than I think they are.

Things I'm not saying

• I'm not saying WBEs are bad. I think true human WBEs would be risky but great.
• My point is not that WBEs are risky because we might get a near miss and have a human-like but strange creature (though that would be bad too).
• I'm not saying WBEs or even partial brain emulations (PBEs) are easy to make. In fact I think they're probably both quite difficult to make. I'm only saying that if someone makes serious progress towards WBEs, they will most likely also produce PBEs years before WBEs.
• I'm not saying neuroscience has contributed a lot to AI capabilities in the past. I'm only saying that this would happen if someone were making serious progress towards WBEs.
• I'm not saying existing "brain emulations" are dangerous, or that any sort of "partial brain emulation" (PBE) is automatically dangerous. I'm just saying that if a project makes serious progress towards real WBEs, which contain lots of dangerous stuff, then it first produces partial brain emulations that are actually dangerous because they contain some of that stuff.
• I'm not centrally saying that WBE research is bad because people might reverse engineer the brain, extract clean abstract algorithms for intelligence, and then use those algorithms as AI. That is one pathway through which WBE can have the bad effect I'm talking about, but it's far from the only one. I'm making a more comprehensive argument: There's no feasible pathway to true WBEs that doesn't first produce expropriatable things which can (and will) be used to accelerate AI capabilities.
• I'm not centrally saying that WBE research is bad because people might copy the weights of a partial brain emulation and run it elsewhere. That's one path among several others. For example, if you produce the weights for a PBE and successfully keep the weights secret, you may still leak the method you used. That method can then be used by others to produce comparable PBEs for misuse as parts of AI systems. (A similar thing seems to happen with current frontier AI innovations.)
• Whether or not an electrical-only quasi-WBE is extremely useful is not a central crux. This is somewhat cruxy, but not my central crux.
• Whether making WBEs is extremely hard or only very hard is not a central crux. My central crux is the infeasibility of a "leap" from here to WBEs, without intermediately producing partial brain emulations for several years, which accelerates AI capabilities substantially in expectation. (But, this crux does collapse if you think making true human WBEs is pretty easy or only kinda hard, rather than at least very hard.)

Context: AI capabilities research expropriates any partial understanding of intelligence

If someone were to build superhuman artificial general intelligence, then most likely humanity would be wiped out. Research towards creating AGI nevertheless has been initiated, and has continued for the past 70 years through two winters.

More recently, many researchers have set out with the intention of decreasing the risk of extinction due to AGI. One of the main plans they've tried is to create an AGI that is extremely useful (as an artificial general intelligence) but does not kill everyone. This research is called AI alignment, AI safety, AI control, or Friendly AI research. To have a chance at succeeding, this research has to pass through expropriatable AGI capabilities.

Order-dependency: AGI alignment research passes through partial understanding of intelligence

Some of that AI safety research engaged, in public, with AGI precursors. That is, it produced and shared ideas that engaged with the conceptual or computational elements that will lead to AGI, such as deep learning systems, theories of powerful agentic or epistemic engines, and so on.

AI safety research that didn't engage publicly with AGI precursors did not succeed at figuring out how it would be possible to create aligned AGI. Such research probably cannot feasibly do that. You can't understand AGI well enough to make an aligned one without also understanding new things about AGI; and that knowledge, once found, tends to get out into the broader sector of people trying to advance AGI capabilities. And if it doesn't get out, then you probably won't understand enough to actually figure out AGI alignment, because you're isolated and on your own, and the problem is too hard to solve by yourself (or within your small group).

AI safety research that did engage publicly with AGI precursors has also failed to figure out aligned AGI. However, this research has meaningfully accelerated AI capabilities.

This pattern comes from two basic facts:

• There is an apparently unavoidable order-dependency. You cannot understand aligned AGI before you understand AGI. To succeed at figuring out aligned AGI, you had to first succeed at understanding AGI, in part or in whole. There's no way to get from here (no aligned AGI) to there (understand aligned AGI) without passing through understanding AGI. If you're "aligning" something that isn't AGI, it's probably not useful enough to help prevent risk from unaligned AGI—it misses the point of making aligned AGI because it's not powerful enough.
• The sector of AI capabilities research will happily suck up any partial understanding of AGI and use it to advance AI capabilities. When you pass through understanding AGI on your way to aligned AGI, you're throwing stuff into the bonfire.

There is a parallel fact pattern for WBEs: You can't get to WBEs without producing partial brain emulations, which make dangerous AI capabilities available to AI capabilities researchers.

What is whole brain emulation research?

By whole brain emulation, I mean a computer program running on a computer that behaves "truly as a human", even over long timescales. (But the WBE is allowed to behave "truly as a human who is running as a computer program", and in particular they're allowed to modify themself.)

The contention of this article is that research that aims at, or progresses towards, whole brain emulation is likely to be very net-harmful to humanity.

The harm caused by WBE research is not because whole brain emulation would be bad. Whole brain emulation does carry large risks: An uploaded human would be in a unique and extreme state of being both very powerful (through fast, self-modified cognition) and also disconnected from humanity. However, a WBE would constitute aligned or mostly-aligned AGI, in a sense. That person or group of people might have a solid shot at ending the acute risk of unaligned AGI, and navigating to a good future for humanity. Given the risk already posed by the possibility of AGI, the risk of WBEs would probably be well worth it.

The harm caused by WBE research comes from the fact that, in order to get to WBEs, you have to pass through partial understandings of the human brain. Just as in the case of AI safety research, these partial understandings are highly dangerous. If they are not powerful, they aren't very far along the way to WBEs. If they are powerful, they will likely be expropriated by the AI capabilities research sector. Accelerating AI capabilities increases the chances that humanity will be wiped out by AGI.

Thus, for the purposes of this article, we have this functional definition of WBE research:

Whole brain emulation research is research that contributes to our ability to emulate in silico the algorithmic structure of the brain that produces intelligence in humans.

Is this a fair definition of WBE research? Isn't that a somewhat differently-expressed goal than WBE research that is aimed at emulating a whole person?

The following sections will argue that researchers are very unlikely to be able to get an actual whole brain emulation without many steps along the way of partial brain emulation (PBE). That is, at many points along the way to WBE, the WBE research community will have produced enough knowledge to emulate parts or aspects of the human brain in silico, without being at all able to emulate a whole human brain.

(To be clear, a PBE need not look much like a WBE in any "phenotypic" way; it may not speak, think, or learn. A PBE is anything that embodies some of the aspects of the human brain that make human brains very powerful in the world. A PBE could be, for example, a very accurate long-term simulation of a single neuron. I also use "PBEs" as a metonymy for "PBEs or methods, tools, and other knowledge that can be used to create PBEs".)

On the other hand, I expect genuine WBE to be a very difficult research goal to achieve. My guess is that it's difficult enough that the normal course of neuroscience research is fairly unlikely to produce PBEs. In other words, normal research seems fairly unlikely to produce dangerous in silico emulations of intelligence-producing algorithms copied from the brain, any time soon. However, WBE research specifically is aimed at a goal that must pass through PBEs. So, the goal of WBE is a dangerous goal, because the research it prioritizes has dangerous outputs if successful.

There are many ways to have strictly partial brain emulation

My claim is that, on the way to WBE, research will produce partial brain emulations (PBEs)—or at least, tools and understanding sufficient to make various sorts of PBEs. Subclaim: There are many ways to have PBEs that aren't WBEs. In other words, there are many intelligence-load-bearing aspects of human brains that we could figure out how to emulate, without figuring out how to emulate a whole human brain.

There are at least two reasons there are many points of PBE, lying in the field of possibilities between here and WBEs. One is that there are many different elements of a WBE. You'd likely have to emulate, for example:

• the full connectome,
• the single-neuron learning algorithms,
• the algorithms for glial-cell growth and support,
• the production and spread of neurotransmitters,
• myelination,
• perineuronal nets,
• vasculature and nutrition,
• and so on.

The other reason is that, for each of these dimensions, we could make partial progress without complete comprehensive success. Because of these two reasons, there are many ways we can end up with the ability to emulate substantive (and dangerous) elements of the brain, without WBEs. For example, we might have:

• A partial connectome, which for example describes the cortex well but doesn't describe the amygdala, or drops regions due to aberrations in the expansion process.
• A partial understanding of the learning algorithms that a neuron implements, which for example doesn't explain how the neuron responds to certain neurotransmitters or responds to a demanding context with lots of feedback from surrounding connections.
• A partial understanding of the algorithms embodied by a cortical microcolumn, which doesn't describe how they compete for blood flow (which could be algorithmically important).

There will be incremental progress towards WBEs with many stages of partial understanding

What will most likely happen is that scientists will, over time, incrementally gain more and more access to brain anatomy, functioning, and abstract algorithms. Correspondingly, scientists will incrementally gain more and more ability to emulate more and more aspects of the brain in silico (PBEs).

(Aside: This may sound somewhat tautological. Of course science proceeds incrementally, gradually gaining more information and dissolving more and more questions, right? But the counterfactual is imaginable: It could hypothetically be the case that, in the course of developing a brain scanning method, for the first 95% of the research you have 0 brain scan data (but some other kind of progress, e.g. design or fabrication of the scanning apparatus); and then at the end of the research, you suddenly get 100% of the data about the brain that you wanted. This seems very unlikely (which is my point), but it's possible. And indeed, this is the hypothetical that is seemingly being posited, if we say "We can just build actual human WBE, and not build PBEs which will be dangerously expropriated into AI capabilities.". That leap would be nice, but is implausible.)

In other words, we won't observe zero data and zero ability to emulate aspects of brain functioning, followed by a breakthrough where we get all the data and a WBE. Rather, we'll observe a messy, long process of expensive, noisy, partial results; followed by cheaper, higher-signal, more complete results; and this will proceed along many dimensions of progress. There has been, and probably will be, an expanding pareto frontier of brain scanning devices in terms of expense, spatial resolution, time resolution, and depth; and an expanding frontier of cell modeling in terms of expense, functional accuracy at different timescales, and dependence on high-level / end-to-end reference data.

There is a wide spectrum of access difficulty for brains and algorithms

There is probably a wide spectrum of difficulty of access, for various aspects of brains. This is a slightly stronger statement than just the prediction of incremental progress by default. In particular, there are many aspects of the brain that will be hard to access.

An implication of the difficulty spectrum is that even with some more concentrated effort, but in the absence of a truly hugely resourced, very concentrated effort, there will likely be many holdouts. You'll accelerate the incremental progress, but you won't skip to the end in one jump. You'll land somewhere along the spectrum, and a bunch of the aspects of brains that are harder to access will still be out of reach for some time.

(A stronger version of this section might pull from historical attempts to model a single living cell in detail, well enough to predict most or all behavior in a wide range of environments and timescales. I suspect this is a better vague anchor point for the difficulty of WBEs compared to, say, the difficulty of reading out a computer program off of an extremely inconvenient hard drive and then running it.)

I'm going to give lots of examples here. Feel free to skim or skip. But I want to hammer this point very firmly, because this point provides much of the force for my argument that quite substantive PBEs will inevitably come before WBEs. Examples of difficulty spectra:

• The brains of different species have different sizes.

  • A common plan held by people proposing WBE research is to first emulate the brains of small organisms, then work their way up to bigger brains, through mouse brains, up to a human brain.
  • The smaller brains are easier to emulate for many reasons, so would likely be done significantly before a true human WBE.
  • The emulations of smaller brains would constitute a dangerous PBE. For example, you could take a mouse brain emulation and then scale up its cerebral cortex by some large multiplier (${10}^4$, say, or ${10}^5$—to a greater size than a human brain). This would almost certainly not be a safe human-like WBE, but it may very well become a true general intelligence or superintelligence, or be usable as a major element of a subsequent artificial general superintelligence.

• Different brain elements are more or less uniform and therefore generalizable.

  • The cortex is somewhat repetitive in terms of structure, though probably far from totally uniform. That means it may be relatively easy to get a partial model of a typical cortical column (or one type of column, if they vary a lot between cortical regions). Such a partial model could then be scaled up many-fold, to get a PBE that is functionally successful but far from a WBE. On the other hand, it may be relatively much harder to get all or most of the detailed differences in structure found in different cortical regions, or found in non-cortical brain tissues.
  • There may be common types of brain cells, which share much of their functional behavior. Getting a PBE involving those types of brain cells would be significantly easier than getting a WBE, requiring good models of most or all types of brain cells.

• Different brain areas are more or less easily accessible.

  • For example, the cortex is relatively easy to mechanically or optically access (though difficult in absolute terms). Lower / innerward areas, such as the basal ganglia or hypothalamus or brainstem, are harder to access.

• Different scanning (and recording) methods trade off scale (how much of the brain is covered), spatial resolution (how small a region is lumped together), time resolution, signal/noise ratio, expense, and type of scanning (or recording).

  • These different scanning methods "complement" each other in the colloquial sense that they each have aspects of the brain that they help with, that other methods aren't as good for. But they do not complement each other in the sense of covering the whole space. All the current scanning methods put together can't scan a live human brain with high resolution and full scale, and this will very likely continue being the case.
    • The State of Brain Emulation Report 2025 states:
    • "Conversely, invasive recordings can resolve individual neurons but are restricted to tiny fractions of brain tissue, capturing thousands of neurons at best compared to the brain's ~86 billion. This trade-off between resolution and coverage appears fundamental rather than technological, making comprehensive recordings with full human brain coverage at single-neuron resolution permanently out of reach."
  • In vivo, electrical scanning is relatively easy compared to chemical / molecular scanning.
  • Longitudinal data (e.g. brain scans for hours rather than seconds) may be important for understanding some brain aspects, and is much more expensive.
  • This variety displays spectra of access difficulty: Each scanning method has information it can and can't access, and the whole ensemble of scanning methods has information it can and can't access.

• Different brain elements have very different sizes and other physical properties.

  • Glial cell somata are smaller than cortical pyramidal somata. Cerebellar granule cell somata are smaller than glial cell somata. Axons, dendrites, and synapses are smaller still (in diameter). Then vesicles, ion pumps, transcription factors, cytokines, hormones, neurotransmitters, etc. (That's over 4 orders of magnitude.)
  • This size spectrum sets up a clear spectrum of access difficulty. Smaller things are harder to get in contact with, harder to measure, harder to isolate for study, and more numerous (thus more costly to model in full).
  • Other physical properties also affect measurability. E.g. it's very challenging to measure proteins or lipids, especially when you add constraints like in vivo, in 3D, at scale, and with high resolution; unlike flows of charge, they can't be measured electrically.

• Different brain elements have very different characteristic timescales.

  • E.g. some neural functions happen at the millisecond timescale (action potential). Some crucial neural functions, such as morphological changes, happen over the course of minutes, hours, or days.

• Different brain elements have different rarities.

  • Examples of classes, where different members of the class have different rarities:
    • Cell types
    • Cell states (gene-regulatory, morphological)
    • Neural circuit structures
    • Types of interactions between elements
  • The rarer an element is, the harder it is to learn about that element. E.g. the harder it is to benchmark your emulations that involve that element. (And the more likely it is that you won't even notice its existence or relevance.)

• Different brain elements are more or less well-understood scientifically.

  • For example, many proteins in cells are known, some are not (if you include alternative splicing and post-translational modifications).
  • For example, many interactions between functionally important molecules are not known or understood.
  • The ones that are poorly understood may take novel basic science to detect, isolate, characterize structurally, characterize functionally, and place within full functional pathways. This is hard work.

• Different brain algorithms are more or less complex.

  • Neurons are reasonably well-understood as some sort of integrator of electric signals. But they are very poorly understood as living cells, with regulatory and morphological changes. As living cells, they are very algorithmically complex. Many short-term changes seem fairly simple; longer-term changes are probably more complex (e.g. non-linear, dependent on hidden state, dependent on several kinds of inputs).
  • Some neural circuits are somewhat understood. But there are also large, complex circuits. How cortical columns work on their own, and how they are wired together, and how they ensemble together for performances, are not well-understood and are likely complex.

• Different brain elements require more or less compute to emulate.

  • Thus, you're more likely to get an emulation of a small piece of cortex, before getting a WBE. The emulated cortex may be compilable into something much cheaper, and still performant, but inhuman, for use in AI capabilities research.
  • Note that the compute constraint means you can't leapfrog other difficulty spectra by brute-force emulation.
  • (You could try to scale up compute to the point where you can "simply" emulate the brain at an atomic or quantum level, but that's a truly massive amount of compute, which would accelerate AI capabilities on the way there.)

• Meta: different difficulty spectra will fall at different times.

  • Even if you can leap to the end of some of the spectra, or avoid them, there are a lot of them.
  • You'll probably get, say, half of them, well before you get all or nearly all of them.
  • For example, you could get a very good model of a single neuron, without having a good model of large ensembles as found in the human brain. Then someone could emulate large numbers of neurons in non-human configurations, selected and trained to be effective at task performance.

Filling in gaps with learning creates the capabilities expropriation pipeline

A central pillar of the argument in this article is that some aspects of the brain will be difficult to build accurate emulations of. If it's actually the case that creating WBEs is fairly easy, or has a very direct path with a jump from nothing to WBE without intermediate PBEs, then such WBE research may be net-helpful rather than net-harmful.

Part of the case that some brain aspects are difficult to emulate is that there's a wide spectrum of how difficult it is to get useful data that directly pins down neural mechanisms. Some people respond to this case by saying something like:

Yes, there will be issues with access for recording, quantity and quality of data, and our understanding of specific mechanisms. But we can get lots of data that constrains the functional behavior of specific mechanisms—enough data to learn an emulation that encompasses those mechanisms, insofar as they affect the behavior of the brain at the level we care about. This data can come from brain recording at lower resolution, from observations of behavior, and from training the emulations based on functionality.

I've heard this argument made in person. It also seems to be suggested by the State of Brain Emulation Report 2025 as a plausible element of a research pathway; I'll describe that report a tiny bit here, as it's relevant in general to this article and in particular it hints at this "fill in the gaps with training" strategy. In the "Overview" section, that report states:

"The path forward for computational neuroscience involves three coordinated strategic objectives. First, achieve high-fidelity emulations in small, tractable organisms by fully integrating their complete connectomes with rich, whole-brain functional and causal perturbation datasets. Second, within these same systems, develop and validate generative models that map molecularly-annotated structure to function. Such a mapping allows functional parameters to be inferred for larger mammalian nervous systems where comprehensive functional recordings are largely unavailable."

I think their plan centers around generalizing from models of animals and single human neurons. I'll address this in a later section. For the purposes of the present section, I take it that some of the feasibility of this plan is meant to come from the possibility of machine learning. For example, "develop... generative models" is I think supposed to come from causal / perturbation data, but also presumably from search over a space of computational models not strictly tied to biological structure. (If instead it is tied to biological structure, then we're back to this project being very difficult.) They give a tepid endorsement of this possibility:

"Nonetheless, in smaller, relatively stereotyped organisms where single-neuron resolution data could be collected in large quantities, purely "black-box" models with minimal biophysical assumptions could at least theoretically thrive."

(It seems their preferred alternative is inferring the function of neurons from their observable physical structure. However, the main example they cite is Holler et al., 2021, which only recovers a specific parameter of electrical behavior—which, even if generalized to all such electrical parameters, would not be sufficient for a WBE-worthy neural model. The remaining aspects of neural behavior, such as gene-regulatory and morphological changes at the minutes / hours / days timescale, would be harder to discover. Maybe I'm reading too much into their mentions of black box learning, and it's more accurate to say that they simply don't state a plausible plan for discovering those aspects of neural behavior. Their notion of minimal brain emulation, the bar for them calling something an emulation, does not seem to have any requirement for modeling plasticity.)

That report later cites, for example, Lappalainen et al., 2024 and Cowley et al., 2024 as prominent recent examples of Drosophila brain emulation (though not necessarily as an endorsement of that strategy). Both of those papers trained parameters of their models based on some sort of overall functional behavior (Cowley) or on overall task performance (Lappalainen).

Later, in their table on "Gaps and opportunities" in human brain emulation, they describe the problem:

"The most fundamental limitation is the lack of adequate functional and structural data to constrain human brain models. While future technologies may eventually provide detailed structural data through connectomics, functional data at cellular resolution will likely remain permanently out of reach due to physical and ethical constraints. This forces models to rely on massive extrapolations from animal studies or indirect measurements, severely limiting their biological validity."

It's true that the paucity of data can be made up for using machine learning and by generalization of mechanisms from smaller units. However, this strategy increases the likelihood of producing dangerous PBEs and non-human WBE-like emulations.

Whenever a WBE project fills in gaps in their ability to directly emulate the mechanisms of a human brain by training on signals of end-to-end functionality, the project degrades its safety case. Instead of copying all the algorithms from the human brain, you're admixing some additional, non-human-brain capabilities into the partial human brain capabilities that you got from studying human brains. One could imagine creating a kludge almost-WBE, which is just like a human on the scale of 30 seconds, but slowly diverges as it applies inhuman learning algorithms trained to be functional peers to the learning ability of human neurons—leading to a gradually more alien general intelligence.

In short, the more you use machine learning to fill in unknown aspects of your brain emulation, the less you maintain the alignedness-by-default property of human brains.

To say this another way, when you use ML to fill in gaps, you're likely to increase capabilities while only slightly (or not at all) increasing the degree to which you've accurately emulated the human mind. It's much easier to find some functional algorithm-elements, than to find the appropriate human functional algorithm-elements. It's much easier to get training signals for usefulness than for being human-like or good; and even if you train on imitation, as we've seen with LLMs, it's much easier to get somewhat-accurate imitation than to get something that's mechanistically human in a way that generalizes like humans do.

Thus, what you're really doing is taking the human algorithm-elements that you did more directly recover from brain data, and then adding non-human stuff, to produce something more competent. In other words, you've constructed a PBE to AI capabilities pipeline. That pipeline turned a human PBE into a more powerful AI that's less human on average; and that pipeline could continue doing so, as you improve the PBE. You've also demonstrated to AI capabilities researchers that they could do similarly.

Avoiding emulating some neural details doesn't make WBEs easy

There's a sister argument to "we'll fill in the gaps with ML". This sister argument also supports the idea that, actually, it's not as hard as it may seem to get a working WBE; and therefore maybe we actually can get there in one leap. The argument is something like this:

You don't have to solve all the hardest parts of neuroscience. The brain is implemented on living cells and neurotransmitters because the brain evolved in cellular organisms, but that doesn't mean brains are about cells and neurotransmitters. The brain is an information processing system, and it operates on a somewhat higher level of structure. It's that structure that matters. We don't need to perfectly model the atoms in every cell, to get a good enough emulation of the neural and interneural structures and interactions that give rise to the brain's information processing abilities, which add up to a human. Since we can abstract away from much of the detail, we can avoid the difficulties with modeling all that detail, and instead solve the easier problem of modeling the meso-scale structure that's relevant to the operations of the human mind.

This argument holds some water. Indeed, it would be extremely hard to get WBEs if the only way were to simulate every protein and lipid molecule in every brain cell. The actual level of difficulty is somewhat below that level of difficulty.

However, I think this argument just isn't very strong. How are you going to get an emulation of a neuron that includes all the functional consequences of lipid membrane shape changes? (This is just an example for illustration; pick something else such as surface protein varieties and densities, or gene-regulatory states of brain cells that affect hours-long changes in millisecond-scale functional properties, if you think membrane shapes don't matter too much.) The approaches I see are:

• From below.
  • That is, emulate the molecules involved in constructing and reshaping the membranes. The behavior of the membranes emerges as a bottom-up consequence of the behavior of the smaller elements.
  • The "from below" method is the expensive difficult method, requiring measuring nanoscale molecules, which the above argument tries to avoid.
• From the side.
  • Measure the membrane shapes themselves, or other similar-scale elements that reveal the membrane shapes.
  • But this is already probably extremely difficult. How would you measure these changes at scale, with reasonable time and space resolution?
• From above.
  • Get data about adjacent phenomena—e.g. consequences of membrane shapes, or higher-level phenomena (such as a neuron's overall connectivity or excitability or similar) that are influenced by membrane shape. Then infer the shapes from the adjacent data.
  • But this is back to filling in the gaps using learning. In particular, you're filling in consequential unknown mechanisms with computer-generated hypotheses for how those unknowns work. This is adding non-human algorithmic content to human content in order to make it more useful.
• Some combination of these approaches, and thus some combination of their problems.

Generalizable models of small components would be dangerous PBEs

A third sister argument, which tries to support that WBEs aren't so hard, goes like this:

You can get high leverage by understanding some small element of brains, and then generalize that across the whole brain. For example, you could get a detailed model of a single neuron, and then copy-paste that model for many neurons. Or you could do that for a microcolumn, and then copy-paste that model for all the cortical microcolumns.

Problems:

• You still have to get a very good model of a single neuron (good enough to copy-paste billions of times). So you still have to get deep into the details, and this is probably a hard task with many intermediate PBE states.
• If that actually worked, the small model would already be a dangerous PBE that can contribute to AI capabilities. For example, the detailed single-neuron model could be copy-pasted 1 trillion times, or the cortical microcolumn model could be copy-pasted 1 billion times. Note that this is still dangerous even if the small component model is not very good—not good enough to use for a true human WBE. Even if you are only trying to copy from the human brain and so you would not use machine learning to fill in the gaps, an AI capabilities person would do so; so they'd have a good shot at using the small component model as part of an ensemble which they train up to high performance.
• Unless it's a really good model (which would require more deep detail), you'll still have to do some fitting to match other components (neurons, microcolumns) in their variegated contexts. That's more filling in the gaps with learning, rather than copying from the human brain; or else that's more difficult neuroscience.

A siloed one-shot leap to WBEs is highly implausible

Making WBEs is a big, complex, difficult, many-years technoscientific research project. A true human WBE would be far and away the most complex structure ever created by humans. The foregoing text argued that a project aimed at WBEs is likely to take a long time or stall out partway through, and in the meantime, if the project had any actual chance at making WBEs, it would have produced PBEs expropriatable by AI capabilities research.

But, what if the WBE project did the hard research all in one go? You could have a single, large, well-resourced, well-contained project that goes all the way to WBEs. As long as it doesn't leak, you don't contribute to AI capabilities.

There is a big basic problem with this plan: It's somewhat implausible to have a leakproof silo like this that goes all the way to WBEs. There are just too many serious practical difficulties:

• Secrecy is hard.
  • Ideas leak, people leak, people spy, etc.
  • Diffusion of ideas within technoscientific research is the default; cf. the history of AI researchers moving between projects.
  • Further, there would be immense financial incentive for espionage from the outside, leaks from the inside, and potential government pressure to turn over secrets.
• Secrecy is painful. If you don't publish, you don't get attention, funding, and talent. You have to trade off talent vs. willingness to keep secrets when hiring. You have to forego collaborations with other groups that are less strict about secrecy.
• It's easy to get partway through your research and then get stuck / lose funding / get regulated / whatever, and be forced to disband. At that point your secrets are feathers spilling from an unzipped pillowcase into the wind, as the scientists scatter, carrying the PBE-creating ideas with them.
• Even if the project does not stall out, it would still probably take a long time. This directly increases the overall chances of leaks by increasing exposure.
• It's easy for your whole research organization to get coopted. Cf. OpenAI and so on and so on. If the project does start approaching WBEs, it would be at a heightened risk of being coopted.

What if you entirely avoid making dangerous, expropriatable PBEs in the first place?

One broad strategy you could try to take would be "copy only". You avoid understanding anything about the brain, you avoid extracting any abstract algorithms, you avoid compressed descriptions of how aspects of the brain work, you avoid highly accurate emulations of small chunks of the brain. You just... Well I don't really know what this looks like in practice. But it's logically coherent at least; e.g. if you had a hypothetical atomic scanner, you could scan all the atoms in a human brain and then run an atom-level simulation of the brain. And if you don't get the atom-level simulation, you dissolve your project and don't share any discoveries.

In practice, though, a project would not be successful while sticking to copy-or-bust. The copy-only method wouldn't work. (I'm asserting this without much argument. But consider that counterarguments to "WBEs are extremely hard to make" usually come in the form of some proposal for a method to gain efficiency—by moving away from copy-only.) So, to have any chance of succeeding, the project would be forced to use learning to fill in gaps, use small emulations to test its current draft algorithms, build partial models of neurons and other structures, assemble partial datasets, and so on.

More importantly, even if you don't create PBEs as such, you're still creating the tools to create PBEs. If you figure out how to "copy" the whole human brain, then you also figured out how to "copy" a small chunk of the cortex. If at some point you made a true WBE, then probably some significant time before that point, it was feasible to make a PBE—which, when copied and trained as part of AI capabilities research, would be very dangerous. This kind of knowledge—methods, tools, confident knowledge that something can be done—is, I think, pretty prone to diffusion of personnel and to stimulus diffusion.

This is similar to the situation with AGI alignment and AGI capabilities: There's no practical way from A to C without passing through (P)B(E).

Getting a powerfully intelligent WBE requires emulating powerful algorithms

One way to avoid creating PBEs on the way to WBEs is to avoid getting a detailed model of the learning / updating dynamics of neurons. A pastiche of arguments that I've heard could go like this:

Longer-timescale neural dynamics, mediated by complex molecular interactions, embody the dangerous brain algorithms. But we don't need to crack those dynamics. In big brains, protein and neurotransmitter and other molecular signaling is way too slow. All important computations are done with fast electrical signaling. So to emulate a big brain, we just have to emulate the electrical properties and behaviors of neurons. This way, we're getting the benefit of having an emulation of a whole human brain, but we haven't made neural dynamics available for AI capabilities to expropriate.

But in this scenario, the "emulation of a whole human brain" that we get is not really a WBE. It may be accurate at the 1 second or 30 second timescale, but it will not be accurate at the 10 minute timescale, let alone the 10 hour or 10 day timescale. Without the longer-term neural dynamics of neurite changes, gene-regulatory changes, and so on, you don't get learning, you don't get thinking, you don't get new concepts and new long-term memories, and so on.

Such an emulation is just not that useful. I would imagine that a one-minute-human would be comparably useful to a modern LLM, really, for roughly the same reasons; and we already have those.

In general, I expect that there are many different brain algorithms (at various levels of abstraction) that a WBE must have, in order to be a powerful mind the way a human is a powerful mind. Imagine deleting certain kinds of brain cells, certain kinds of circuits, certain long-term neural dynamics, certain brain regions, and so on. Some of them might leave behind a somewhat-functioning mind, but many of them would be pretty damaging. You need to get all of those.

So, you could avoid creating the dangerous PBE stuff that comes with unlocking neural dynamics. But then you also give up much of the usefulness. This isn't a coincidence; to make a true WBE, you have to make something very powerful, so you probably made something kinda powerful first.

Getting a truly human WBE is probably a very high bar

The idea of making WBEs is that humans are aligned with human values. So if your WBE is basically a human, you've made an aligned general intelligence. That person, being in silico, can much more easily self-modify and self-improve and think fast, compared to an organic human. They can use their increased cognitive capacity to get humanity out of danger.

A WBE modifying themself in order to self-improve is a pretty risky process. They could mess up their values accidentally through self-modification. Even if they don't, they could become very superhumanly intelligent, and thence corrupted by power. But it could probably be done well, with safeguards, and may be well worth the risk given the high default risk of extinction by AGI.

In order for the safety case to go through, the WBE has to at least start off being basically a human. How close? I think there are probably many different aspects of human brains that a WBE must have, in order to be aligned. In other words, there are many brain aspects such that if a WBE is missing that aspect, the WBE doesn't have humane values and is not even safe to run, let alone likely to save humanity. Consider:

• One might think it's ok to be missing a piece of cortex. (E.g. imagine that the connectome construction is not fully reliable, and goes wrong in one area.) At least, maybe that would be largely morally neutral, even if it would be a hit to cognitive capability. Maybe plasticity will help smooth over missing patches? Leaving aside that this relies heavily on the emulation of neurons being accurate enough to emulate learning, consider that crystallized knowledge (broadly speaking) could be pretty bad to delete. In particular, the deleted portion could be morally loaded. E.g. you might have deleted areas that embody structures of wisdom, ethics, strategies for caring for others and getting along, and so on.
• Misconnected or diminished ventromedial prefrontal cortex -> altered relationship to fear, sense of obligation, and other emotions -> less loyal to organic humans.
• Missing some neurotransmitter -> doesn't maintain or form bonds of love, caring, or friendship with others -> pursues selfish ends that don't account for other people's preferences.
• Missing gene-regulation pathway -> responds very unusually to shame, guilt, or social pressure -> not bound by norms.
• Altered learning algorithm -> has substantively different patterns of understanding and non-understanding -> cannot feasibly empathically model, and be empathically modeled by, organic humans -> becomes isolated and hostile.
• Altered prefrontal cortex connectivity -> weaker self-regulation -> gets set off on rage-driven or spite-driven pursuits.

These are all somewhat fanciful, mainly just to illustrate the sort of thing I mean. (A better version of this section would cite actual cases of people having genetic, anatomical, chemical, or other kinds of brain abnormalities, and consequently having unusual and harmful interactions with other people.)

Perhaps one such aberration is not enough to render a WBE net-negative in expectation. (Though keep in mind that even a true human WBE would then be under immense pressure in a very unusual, unstable, destabilizing situation.)

But WBE research starts with nothing emulated, and has to work hard, counting up from zero, figuring out how to emulate various brain aspects more and more accurately, until it gets almost all of these things right. That leaves a large region of ways that WBE could get partial, but far from complete, emulation of brain aspects that are needed to be humanely human.

To be very clear: I'm not saying "WBEs would be risky because they might be missing some human elements; therefore we must have very stringent validation of accuracy for WBEs.". That's a reasonable argument to make; I think WBEs would hypothetically be worth the risk, though of course also you'd want lots of stringent validation. My point here is not that WBEs might accidentally fail to be quite good enough. My point here is that the bar is high, therefore getting there is very difficult, therefore you're far more likely to land somewhere partway to WBEs—making dangerous PBEs available to AI capabilities researchers—rather than getting all the way to WBEs.

To summarize this section in a different way: It seems very unlikely that goodness / humanness on the one hand, and pure effectiveness / intelligence / capability on the other hand, are so tightly coupled that you either fail to make something useful or else make something that's basically a human. It seems much more likely that you'd get stuck or bogged down in your research after having made something that is kinda functional, and dangerous if expropriated by AI capabilities research, but far from being a human.

Brain elements will be expropriated by AI capabilities even if they haven't been historically

I make no strong claim about the history of ideas flowing from brains and neuroscience to AI research. My impression is that there has not been much flow, and this is not a crux for my viewpoint.

My explanation for the lack of transfer so far would simply be: brain emulation is hard, neuroscience is hard, and it hasn't progressed very far in an absolute sense. So not much has been made available for AI capabilities to steal from brains, so far.

My argument is that if, hypothetically, a research project were getting close to making a true WBE, then well before that point, they would have (intentionally or not) made dangerous brain elements much more available for expropriation. That is, they would have made it much easier for other people to take elements of the brain (whether they be actual PBEs, or methods for making PBEs, or brain algorithms, or other such information) and use those brain elements to significantly accelerate AI capabilities.

Indeed, apparently, there is already a company, funded to the tune of five hundred fucking million dollars, called Flourish. One of the neuroscientists there says they "want to do data collection across the nano, micro, and meso scales to support the discovery of the core algorithm" of the brain. If there were cheaper, easier, more accurate ways to get at some aspects of brain function, these people and others would jump at the chance to expropriate those methods, get brain elements, and incorporate those elements into AI systems to increase capabilities.

As a general rule, the AI capabilities research sector is better equipped to find useful, albeit supremely dangerous, applications of elements of intelligent cognition. This seems to have been somewhat borne out by the history of "AI safety" research. That's the job of people in that sector—finding ways to apply stuff to the task of making AI / AGI. Further, the technological ecosystem itself is already somewhat built out for that purpose. Thus, the AI capabilities research sector volatilizes elements of intelligent cognition, making them liable to react with other preexisting elements.

Just because Flourish is trying to do the bad thing, doesn't mean pursuing WBE research is fine or good. It can still hurt. Indeed, if you are especially insightful and determined, and then you attempt a moonshot that most others don't believe in, you might succeed surprisingly much (though probably still not all the way). ...And this is exactly the sort of research output that later gets expropriated and accelerates AI capabilities even though you didn't want to do that.

I don't have much faith in the idea of having good people do dangerous research. One could argue that it's fine, since bad people might do it anyway; and by being good, they won't use their ideas dangerously and their ideas won't get expropriated to accelerate capabilities. But that's not how it seems to have gone overall in the realm of "AI safety" / AI capabilities, and on reflection it doesn't seem like the world has to work that way, even if the world ought to work that way. Maybe there's some way to make it robustly workable, but I'm skeptical.

What to do instead of WBE research

• Do kinda-adjacent work that doesn't pose as much of a risk. For example, you could work on human intelligence amplification. There may be knowledge transfer from would-be WBE research backgrounds to BCIs, brain stem cell transplantation, brain drug design, etc.
• Don't fund WBE research.
• Stop Flourish, ban AGI, etc.