It takes 5 layers and 1000 artificial neurons to simulate a single biological neuron [Link]

This paper has at least two significant flaws when used to estimate relative complexity for useful purposes.  In the authors' defense such an estimate wasn't the main motivation of the paper, but the Quanta article is all about estimation and the paper doesn't mention the flaws.

Flaw one: no reversed control
Say we have two parameterized model classes $A_n$ and $B_n$, and ask  what ns are necessary for $A_n$ to approximate $B_1$ and $B_n$ to approximate $A_1$.  It is trivial to construct model classes for which the n is large in both directions, just because $A_1$ is a much better algorithm to approximate $A_1$ than $B_1$ and vice versa.  I'm not sure how much this cuts off the 1000 estimate, but it could easily be 10x.

Brief Twitter thread about this: https://twitter.com/geoffreyirving/status/1433487270779174918

Flaw two: no scaling w.r.t. multiple neurons
I don't see any reason to believe the 1000 factor would remain constant as you add more neurons, so that we're approximating many real neurons with many (more) artificial neurons.  In particular, it's easy to construct model classes where the factor decays to 1 as you add more real neurons.  I don't know how strong this effect is, but again there is no discussion or estimation of it in the paper.

It's possible much of that supposed additional complexity isn't useful

Yup! That's where I'd put my money.

It's a forgone conclusion that a real-world system has tons of complexity that is not related to the useful functions that the system performs. Consider, for example, the silicon transistors that comprise digital chips—"the useful function that they perform" is a little story involving words like "ON" and "OFF", but "the real-world transistor" needs three equations involving 22 parameters, to a first approximation!

By the same token, my favorite paper on the algorithmic role of dendritic computation has them basically implementing a simple set of ANDs and ORs on incoming signals. It's quite likely that dendrites do other things too besides what's in that one paper, but I think that example is suggestive.

Caveat: I'm mainly thinking of the complexity of understanding the neuronal algorithms involved in "human intelligence" (e.g. common sense, science, language, etc.), which (I claim) are mainly in the cortex and thalamus. I think those algorithms need to be built out of really specific and legible operations, and such operations are unlikely to line up with the full complexity of the input-output behavior of neurons. I think the claim "the useful function that a neuron performs is simpler than the neuron itself" is always true, but it's very strongly true for "human intelligence" related algorithms, whereas it's less true in other contexts, including probably some brainstem circuits, and the neurons in microscopic worms. It seems to me that microscopic worms just don't have enough neurons to not squeeze out useful functionality from every squiggle in their neurons' input-output relations. And moreover here we're not talking about massive intricate beautifully-orchestrated learning algorithms, but rather things like "do this behavior a bit less often when the temperature is low" etc. See my post Building brain-inspired AGI is infinitely easier than understanding the brain for more discussion kinda related to this.

My own quick takeaway is that it takes 5-8 layers with about 1000 neurons in total in an artificial neural network to simulate a single biological neuron of a certain kind, and before taking this into account, we'd likely underestimate the computational power of animal brains relative to artificial neural networks, possibly up to about 1000x.

This does not seem right to me. I haven't read the paper yet, so maybe I'm totally misunderstanding things, but...

1. The bio anchors framework does not envision us achieving AGI/TAI/etc. by simulating the brain, or even by simulating neurons. Instead, it tries to guesstimate how many artificial neurons or parameters we'd need to achieve similar capabilities to the brain, by looking at how many biological neurons or synapses are used in the brain, and then adding a few orders of magnitude of error bars. See the Carlsmith report, especially the conclusion summary diagram. Obviously if we actually wanted to simulate the brain we'd need to do something more sophisticated than just use 1 artificial neuron per biological neuron. For a related post, see this. Anyhow, the point is, this paper seems almost completely irrelevant to the bio anchors framework, because we knew already (and other papers had shown) that if we wanted to simulate a neuron it would take more than just one artificial neuron.
2.  Assuming I'm wrong about point #1, I think the calculation would be more complex than just "1000 artificial neurons needed per biological neuron, so +3 OOMs to bio anchors framework." Most of the computation in the bio anchors calculation comes from synapses, not neurons. Here's an attempt at how the revised calculation might go:
   1. Currently Carlsmith's median estimate is 10^15 flop per second. Ajeya's report guesses that artificial stuff is crappier than bio stuff and so uses 10e16 flop per second as the median instead, IIRC.
   2. There are 10e11 neurons in the brain, and 10^14-10^15 synapses.
   3. If we assume each neuron requires an 8-layer convolutional DNN with 1000 neurons... how many parameters is that? Let's say it's 100,000, correct me if I'm wrong.
   4. So then that would be 100,000 flop per period of neuron-simulation.
   5. I can't access the paper itself but one of the diagrams says something about one ms of input. So that means maybe that the period length is 1 ms, which means 1000 periods a second, which means 100,000,000 flop per second of neuron-simulation.
   6. This would be a lot more than the cost of simulating the synapses, so we don't have to bother calculating that.
   7. So our total cost is 10^8 flop per second per neuron times 10^11 neurons = 10^17 flop per second to simulate the brain.
   8. So this means a loose upper bound for the bio anchors framework should be at 10^17, whereas currently Ajeya uses a median of 10^16 with a few OOMs of uncertainty on either side. It also means, insofar as you think my point #1 is wrong and that this paper is the last word on the subject, that the median should maybe be 10^17 as well, though that's less clear. (Plausibly we'll be able to find more efficient ways to simulate neurons than the dumb 8-layer NN they tried in this paper, shaving an OOM or so off the cost, bringing us back down to 10^16again...)
   9. It's unclear whether this would lengthen or shorten timelines, I'd like to see the calculations for that. My wild guess is that it would lower the probability of <15 year timelines and also lower the probability of >30 year timelines.

This makes a lot of sense to me given our limited progress on simulating even very simple animal brains so far, given the huge amount of compute we have nowadays. The only other viable hypothesis I can think of is that people aren't trying that hard, which doesn't seem right to me.