titotal's Quick takes

It's not really my specific area, but I had a quick look. (Frankly, this is mostly me just thinking out loud to see if I can come up with anything useful, and I don't promise that I succeed.)

Yahoo Finance has option prices with expirations in Dec 2026. We're mostly interested in upside potential rather than downside, so we look at call options, for which we see data up to strike prices of 280.[fn 1]

In principle I think the next step is to do something like invert Black-Scholes (perhaps (?) adjusting for the difference between European- and American-style options, assuming that these options are the latter), but that sounds hard, so let's see if I can figure out something simpler from first principles:

The 280 strike Dec 2026 call option is the right to buy Nvidia stock, on Dec 18th 2026, for a price of $280. Nvidia's current price is ~$124, so these options only have value if the stock more than doubles by then. They're currently trading at $14.50, while the 275 call trades at $15.

The value of a particular option is the integral of the option's payoff profile multiplied by the stock price's probability density. If we want something like "probability the stock is at least X on date Y", the ideal option payoff profile would be an indicator function with a step at X, but we can't exactly get that. Instead, by buying a call struck at A and selling a call struck at B, we get a zero function up to A, then a linear increase from A to B, then a constant function from B. Picking A and B close together seems like the best approximation. It means looking at prices for very low-volume options, but looking at the nearby prices including for higher-volume options, they look superficially in line, so I'll go with it.

More intuitively, if the stock was definitely going to be above both A and B, then the strike-A option would be B - A more valuable than the strike B option (that is, the right to buy a stock worth $10 for a price $1 is worth exactly $3 more than the right to do so for $4). If the stock was definitely going to be below both A and B, then both options would be worthless.

So the value of the two options differ by (B - A)P(the price is above B), plus some awkward term for when the price is between A and B, which you can hopefully make ignorable by making that interval small.

From this I hesitantly conclude that the options markets suggest that P(NVDA >= 280-ish) = 10%-ish?

[fn 1]: It looks like there are more strike prices than that, but all the ones after 280 I think aren't applicable: you can see a huge discontinuity in the prices from 280 to 290, and all the "last trade date" fields for the higher options are from before mid-June, so I think these options don't exist anymore and come from before the 10-to-1 stock split.

I list exactly 2 criticisms. One of them was proven correct, the other I believe to be correct also but am waiting on a response. 

I agree with the asymettry in the cost of waiting, but the other way. If these errors are corrected a week from now, after the debate week has wrapped up, then everybody will have stopped paying attention to the debate, and it will become much harder to correct any BS arising from the faulty tool. 

Do you truly not care that people are accidentally spreading misinformation here? 

Hey, I want to thank you for going through the post. I think you've done a good job, and I appreciate it. I'll try to go through and give a similar effort in the replies. Note that I don't want to pressure you to do a re-reply, although you can if you want. I just want to say my piece and defend myself, and I'm happy to let the readers decide from our duelling accounts. 

Actually, I will skip ahead first, because I think it illuminates the most where the disagreement lies. 

This is a random nitpick, but animal bodies are indeed internally held together by flesh instead of skeletons. The skeleton itself is not connected. Bones only provide local structural support against bending and breaking. If I removed your flesh your bones would mostly disconnect and fall into a heap on the ground. Your bones are generally not under much of any strain at any given moment in time, instead you are more like a tensegrity structure where the vast majority of your integrity comes from tension, which comes from your tendons and muscles.

Yes, if I removed your flesh, your bones would fall to the ground. But similarly, If I removed your skeleton, your flesh would also fall to the ground. Holding a body together is a partnership between bones, muscles and flesh, and if you remove any one, the rest break. 

This is kind of the whole of my original point here. Yes, it's perfectly fine to zoom out onto the tertiary structure of a protein, and discuss the make-up of the crosslinks there, if you are clear that's what you're doing. But without the primarily covalent backbone of the system, there is no tertiary structure. So, starting at the beginning:

This paragraph clearly shows you misunderstood Eliezer. Different proteins are held together almost exclusively by non-covalent forces.

I just disagree. Proteins are held together by a combination of covalent bonds and non-covalent forces. If you went in and removed all the covalent bonds, the protein would collapse into nothingness. If you removed all the non-covalent bonds, you would still have that covalent primary structure backbone, which would then snap back into place and reform all the other bonds, rebuilding the protein. (I mean, not every single time, because sometimes undoing denaturation has an energy penalty that is too high). In that sense, it really makes no sense to say that it's held together "almost exclusively by non-covalent forces". 

It is true that often non-covalent forces (typically hydrophobic interactions only sometimes Van der waals forces) are the dominant structural force of the 3D structure as a whole. Of course, other times covalent bonds are, as is the case in Keratin-type proteins. 

This is also evidently false, since like dozens of people I know have engaged with Drexlers and Eliezers thoughts on this space, many of which have a pretty deep understanding of chemistry, and would use similar (or the same) phrase. 

I spent a very, very long time investigating Drexlerian nanotech, and I definitely never saw anything like "covalently bonded equivalents of biology". I think that would be a pretty bad way to describe it, because, as has been established, biology uses covalent bonds at every level. I could see a case for "strictly covalently bonded" though. 

You seem to be invoking some expert consensus that doesn't exist. Indeed, multiple people with PhD level chemistry background have left comments saying they understood Eliezer's point here.

I don't want to discount the people who did agree with me or who didn't. I saw a chemist saying they agreed with me and getting downvoted, a protein chemist saying they thought the wording was wrong but they liked the "spaghetti" analogy, and an organic chemist also agreeing with me. Generally the consensus seems to be that the language was badly worded or wrong, but some people found the underlying point defensible or even good. I do agree that some experts are okay with the language. 

I think it's worth pointing out that the commentators here and on Lesswrong are disproportionately likely to be Eliezer fans, and be willing to give him the benefit of the doubt. This is not the case for a random person watching a TED talk.

This is also false. The point makes sense, many people with chemistry or biology background get it, as shown above. 

You are right that it is an overstatement, I will edit that. However, I maintain that many experts who encounter these badly worded claims will dismiss your argument as a result. 
 

Look, I appreciate the post about the errors in the quantum physics sequence, but you are again vastly overstating the expert consensus here. I have talked with literally 10+ physics PhDs about the quantum physics sequence. Of course there are selection effects, but most people liked it and thought it was great. Yes, it was actually important to add a renormalization term, as you said in your critique, but really none of the points brought up in the sequences depended on it at all.

I think you are underestimating the selection effects here. The physics phd's who thought it sucked are not on lesswrong, they got turned away by the overconfident mistakes. As a physics PHD myself... Eh, it's better than most pop-science stuff, but that's a very low bar. There's plenty more errors in there, and the underlying argument about MWI is pretty bad, but I'll save that for a future post. 

Like look, when people read your post without actually reading Eliezer's reply, they get the very strong sense that you are claiming that Eliezer is making an error at the level of high-school biology. That somehow he got so confused about chemistry that he didn't understand that a single protein of course is made out of covalent bonds. 

But this is really evidently false and kind of absurd. As you can see in a lot of Eliezer's writing, and also his comment level response, Eliezer did not at any point get confused that proteins are made out of covalent bonds. Indeed, to me and Eliezer it seemed so obvious that proteins internally are made out of covalent bonds that I did not consider the possibility that people could somehow interpret this as a claim about the atoms in proteins being held together by Van der Waals forces (how do you even understand what Van der Waals forces are, but don't understand that proteins are internally covalently bonded?). But that misinterpretation seems really what your post was about.  

 I don't think it really matters whether he did or did not truly know that the primary structure was covalent. The problem was that at no point, in all of the quotes I found of him discussing the matter, did he clarify that he was talking only about the tertiary structure, or "strictly covalent bonding", or crosslinks between protein folds.

 Intentionally or unintentionally, an uninformed listener would get the interpretation: "biology does not use this super duper strong force called "covalent bonds". 

Imagine reading those quotes from the perspective of someone who knows nothing about biology, and tell me that that is not the obvious implication of what he says. 

I'm happy to give the benefit of the doubt when chatting between friends, but he is using this terminology on podcasts and TED talks. Factual rigour matters. 

And, for the record, I don't think it's that unthinkable that he didn't know the  primary structure of protein was covalent, it's not that hard of a mistake to make. Unlike say quantum physics, organic chemistry was never a subject I've seen him delve deeply into, and the only source he ever cited on the subject was Drexler. 

Look, this is doubling down on a misinterpretation which at this point you really should have avoided. We are talking about what you call the tertiary structure here. At the level of tertiary structures, and the bonds between proteins, biology does almost solely stick to ionic bonds and proteins held together by Van der Waals forces. 

Just no. Wait a second, and actually re-read the statement I am responding to.  It's a flat statement that "humans utilise covalent bonds, but "biology doesn't". Obviously, biology does "utilise covalent bonds", in that it's made out of covalent bonds.  If he only wanted to talk about tertiary structure, he should have said "tertiary structure", and not make a flat statement about all of biology. 

Look man, I think you really know by know what Eliezer means by this. Eliezer is talking about alternatives to biology where most of the tertiary structure leverages covalent bonds.

If he means this, he should say that, instead of a different thing that makes no sense. If you say "covalently bonded bacteria", that's the same thing as a regular bacteria.

This is also doubling down on the same misunderstanding. The machinery and tertiary structure of bacteria do not use covalent bonds very much. This is quite different from most current nanomachine designs, which the relevant books hypothesize would be substantially more robust than present biological machinery due to leveraging mostly covalent bonds.

When you say "bacteria", you are talking about the entire bacteria, not just the machinery and tertiary structure. And the entire bacteria does include covalent bonds, or it would fall apart. 

Also, they do utilise tertiary covalent bonds for the parts of the cell that need to be very strong.  

The relevant books focus on the structural differences: diamondoid nanofactories were hypothesized to be very stable, so that atomically precise placement of molecules could enable new structures that were not available in biology. 

I don't understand what you are talking about here. Basically everything in biology is either made out of proteins or manufactured by proteins. If you can make proteins, you can basically make anything. Proteins are the way most things get done in a cell. The sentence above reads as confused as saying "a CPU is not a general purpose calculator. It does exactly one thing, and that is to read instructions and return the results". Yes, ribosomes read instructions and link together amino acids to form proteins, and that is how biological systems generally assemble things.

I see where you're coming from here. It may be a simple matter of bad phrasing on his part leading to a misleading statement. For example, the latter statement:

It should not be very hard for a superintelligence to repurpose ribosomes to build better, more strongly bonded, more energy-dense tiny things that can then have a quite easy time killing everyone

implies that it is the ribosome itself which would directly print out drexlerian nanotech, which of course is impossible. I get that he was probably trying to say the ribosomes will create proteins which in turn create drexlerian nanotech (probably also impossible, but I'll litigate that another day). The phrasing here overstates the ease of this process: uninformed readers will come away thinking "oh, ribosomes are general assemblers, so they can generally assemble nanotech", which is not true. 

This one is confused on multiple levels. The meaning of "X is held together by something" of course depends on what level of organization of X you are talking about. 

I accept this if you specify the level you are talking about, or it's obvious to both you and your audience which level you are talking about. Neither of these apply to Eliezer's statements. 

If you don't specify the level, then the statement is either meaningless, or it's talking about the entire structure. I'm sorry, I am just never going to accept that you can say "instead of covalent bonds", talking generally, when the backbone is covalent bonds. 

I think the main takeaway is that you should really avoid using the phrase "held together" altogether, unless it's paired with a more precise descriptor of what structure you are talking about, primarily for all the reasons explored in the post. 

Even with correction this is still inaccurate. It is correct that non-covalent bonds are the dominant forces for the structure of proteins. Yes, there are some exceptions, like lignin, and that matters, and as I said, I would have upvoted a post talking about that. Yes, it's structurally dependent. But if you aggregate across structures it's true and seems reasonably to describe as being the dominant force.

He never used the phrase "non-covalent bonds". He only ever said van der waals forces, which is wrong: as multiple people pointed out, if you are forced to state one thing primarily making the 3d structure, it would be hydrophobic bonds. 

I think the exceptions are very important, as they negate the whole argument he was making. If biology can make densely covalent bonded things, why doesn't it make all of it's structures covalently bonded? My guess is simply that it's extremely useful to utilize all of the forces available to you, rather than sticking to stiff, rigid, and unflexible structures. But I'll save this for a more detailed research post further down the line. 

As a conclusion, I'll just say this: I think that yudkowsky did not sufficiently scientifically vet his arguments before sending them out to the general public. As a result, he said things that gave his audience a misleading picture of the advantages and disadvantages of biology vs nanotech. They were scientifically badly worded, ending up with phrases that were at best misleading and at worst just incorrect. While this whole experience has been pretty exhausting, I hope that it will at least lead to improved scientific terminology in the future. 

I will eventually write a more detailed analysis of biology vs Drexler, and you can better believe that one will be extensively fact checked before it goes out. 

I will do another round of edits tomorrow, and I'll probably let that be it, there is only so much time in the world one can devote to posting. 

As someone in the ‘general public rather than chemistry/physics PhD’ group, which Eliezer is saying he’s targeting, I definitely thought he meant that