All of Ghost_of_Li_Wenliang's Comments + Replies

EA Forum Prize: Winners for May-July 2021

I will miss the Prize, it helps me identify which of the effortposts I should read.

COVID: How did we do? How can we know?

Agreed - and I agree that "naive" is the right word for it.

I think the main thing I would change about the post (besides the info from Chi which greatly weakens one of the 4 main planks) is better emphasising that it sketches an ideal ceiling which we should expect to sink as details are added. A relatively realistic ideal, but not strong warrant for righteous rage.

Signing off now; thanks to everyone.

COVID: How did we do? How can we know?

Not mostly happy, I think. China apparently needed a new factory, but other places didn't (to the tune of 3bn wasteful doses or ~12bn real ones). 

Also fast approval was only one prong of the fix, along with 2) an order of magnitude more investment, 3) invested much earlier, as pre-Phase I pre-purchases, 4) HCTs, and 5) pivoting away from 80%+ waste as soon as we realise we're doing that.

(HCTs are still relevant here because some of the vaccines have a shelf life < 6 months, and HCTs could thus allow May-June 2020 production to dampen the second or ... (read more)

COVID: How did we do? How can we know?

Datum about process over speed in the civil service and civil society (and also about OWID being far more savvy about optics than Cummings):

“Someone please ensure that they have the 530k within 24 hours from now and report back to me it’s been sent,” Cummings wrote to the chief executive of NHSX. “No procurement, no lawyers, no meetings, no delay please – just send immediately,” he continued. The funding request had the backing of the health secretary, Matt Hancock, who was copied in on the email chain at this point...

After a flurry of communication betwee

... (read more)
4Tom_Ash2moIt's interesting that (being from the Guardian), that article presents the story as being a scandal, with the implication that Cummings was being corrupt.
COVID: How did we do? How can we know?

August 28, 2020: "Production has started at a new plant in Beijing with an annual capacity of roughly 300 million doses. Sinovac has agreed to supply 40 million doses to Bio Farma, an Indonesian state-owned company, between November and March. Sinovac started building the factory in late March and finished the project in July."

So this is in fact a little piece of the happy timeline.

2Tsunayoshi4moBut shouldn't this update our priors towards mostly being on the happy timeline, in the West as well? Given that it took Sinovac/China one year from last March to this March to scale up, and that their vaccines are easier to manufacture than mRNA vaccines, and if we assume high investment from the start in China (so their timeline is close to optimal), it really starts to look like we could not have done much better on manufacturing (because the West does not differ strongly in available doses compared to China)? I.e. we could have approved a few months earlier, but even in December the UK and the US (I think?) were mostly bottlenecked by supply issues, so an earlier approval should not have changed much by this intuition.
2Ghost_of_Li_Wenliang4moDatum about process over speed in the civil service and civil society [] (and also about OWID being far more savvy about optics than Cummings):
COVID: How did we do? How can we know?

One last guess:

My ideology-of-all-public-officials guess is pretty weak compared to an obvious alternative: simple public-choice herding at the executive level. (200 units instead of a million.)

If governments were each minimising their own reputation loss by (correctly) predicting that they wouldn't be punished for doing what everyone was doing, this could be enough to prevent ~all innovation. As much as you want safety in numbers, you doubly don't want to be the first to risk and lose. No entrainment needed, let alone intentional coordination.

(What could ... (read more)

6Linch4moI place >60% on the herding belief fwiw, especially if we limit to countries that have enough power to actually shake things (eg China, US, UK, Russia etc). An additional piece of evidence for this is the degree of correlated beliefs about things like HCTs and genetic enhancement.
COVID: How did we do? How can we know?

I am also very confused. The incentives for politicians to move as fast as they could were so vast.

Besides just vaguely accusing them of lacking courage: Another possibility is a profound entrainment of world elite opinion. One globalised and very narrow Overton window for public professionals. University is the obvious place for this to propagate, but I don't really know. What is its content? "Don't be hasty"? Could a philosophical accommodation really prevent every defection? 

(There were some - Hungary vs EU on vaccines, Israel. I actually just trie... (read more)

COVID: How did we do? How can we know?

You should go for a Connemara; so good natured, unlike some other breeds.

1DAV1D4moOo, I like that one.
COVID: How did we do? How can we know?

You seem to be mistaking this for a white paper, or a piece of legislation, or an itemized purchase order for one different timeline please. It is not that. It is instead a thing to measure our situation against, to short-circuit the useless shrugging described in the opening section.

It would be difficult for more seriousness, more money, more personal and institutional courage to not help. I struggle to understand why you are so sure it wouldn't, or, if you do, why you're pointing out that unexpected things happen, on occasion.

In fact the genome was relea... (read more)

COVID: How did we do? How can we know?

The estimate undersells long COVID because we don't know how many years of 3.2m QALYs to add, but yes that's roughly it. And yes, I only claim that it was a decent deal, particularly since the funding for it couldn't really have gone on something else.

I freely admit that it could be off by a large factor (see my final paragraph). I would love for someone to come and do a proper Bayesian interval version, which would foreground the uncertainty.

I continue to challenge calling it "unrealistic", on priors, just because it's very uncertain. Last January, a hist... (read more)

COVID: How did we do? How can we know?

Human challenge trials are a very old idea. Not doing them is the aberration.

Lockdown uncertainty seems moot. I'm not arguing that any lockdown policy should have been different (except that we might have lifted it a few months early if vaccinations were successfully time-shifted). Did anyone think that (realistic, non-Chinese, non-remote-island) lockdowns were an alternative to vaccines - back when we thought vaccines were coming in 2022 or 2027? The UK government seriously thought they could only do lockdown for a month or two. It doesn't add up.

But my r... (read more)

1Sjlver4moHmm... Here's how I understand your estimate. Is that a fair summary? * If all had gone according to a perfectly happy timeline where everyone makes the right decisions, we could have had enough vaccines in August. * This would be worth approximately 205 million QALYs. * It would also cost approximately 0.7 trillion dollars. * That's 3400 dollars per QALY. My concern (expressed in the comments above) is mainly that the happy timeline is unrealistic, so the estimate could be off by a large factor, similarly to how the time and cost estimates of our plans are often off by a large factor. Your estimate is probably still valuable, even if it is imprecise. We can use it to think about whether vaccine development is cost-effective; I reckon 3400$/QALY puts the cost-effectiveness an order of magnitude below effective charities and some orders of magnitude above many other public health interventions. Is that a fair conclusion? I'd like to take away more from your post than just the estimate, but am not sure at the moment what other recommendations I can take from it...
COVID: How did we do? How can we know?

Sure. Challenge trials polled well in the West, but you're right that Wuhan could have been scarifying. Test of this: how did they poll in Lombardia?

My contention is that (in the US and UK at least) bioethicists and policymakers overestimated the controversy, possibly projecting their own misgivings.

COVID: How did we do? How can we know?

Here's my source, "based on dozens of interviews with diplomats, Commission officials, pharma industry representatives and national government aides". Here's another, and another.

The EU's priorities are revealed in the result: 25% - 45% lower prices. They actually sort of brag about it: 

It seems to have been a mix of understandable coordination, show of force, price haggling, and liability haggling (which is just a kind of price haggling with extra politics). 

Gallina was soon called into the European Parliament, where she repeatedly promised that

... (read more)

Oh, there is not a shred of doubt that the EU delayed buying the vaccines in order to lower the price, and I agree that this was a disastrous decision that led to supply delays. This is however a separate question from approving the vaccine, which is what my objection was about.

COVID: How did we do? How can we know?

I don't recognise my post in this description. I openly acknowledge that there are bottlenecks., including unknown bottlenecks. I put a 150% interval on the key uncertainty. (I am protected somewhat from Hofstadter's law there by the reference point of the Braintree facility, with its almost known lead time.)

It's not unrealistic to pay weekend overtime  or make new weekend hires for regulators, in the biggest health crisis of the century. It's not unrealistic for a single Chinese scientist to just decide on his own to release the genome he already seq... (read more)

3DAV1D4moI’m with Sjlver. There’s a lot of hand waving past problems in the happy timeline: eg, how long does it take to train new regulators who are going to work weekends? How quickly do trained regulators working constant overtime burn out? What happens when the Chinese government denies that the genome released early is the right one? (Also, the invocation of “war time” as a model assumes that wars are run any more efficiently or even urgently. They’re not.)
4Sjlver4moHere's why the post reminds me of the planning fallacy: When people make flawed plans, these plans don't seem unrealistic. They often consist of detailed steps, each of which is quite likely to succeed. And yet, in most cases, the world takes a different turn and the planned project ends up late and more expensive. You describe a "happy timeline" that's analogous to such a plan. For it to work, we would have to make many good decisions; many unknown obstacles would have to be overcome; and many novel ideas (like human challenge trials) accepted. None of these is unrealistic when looked at individually. But collectively, it is very unlikely that all these factors could realistically come together to form your happy timeline. One example to illustrate this: Your post strongly favors vaccines and also attributes enormous costs to lockdowns. I think that this is realistic, but I can think so only in hindsight. In early 2020, it wasn't at all clear that sufficiently strong lockdowns wouldn't bring the pandemic to a manageable level or at least buy us the time we need for vaccine development. Remember the hammer and the dance [] ? Yet, around that same time in your happy timeline, decisions are made to approve and pre-purchase vaccines at high costs. With hindsight, it's easy to say that we should have paid these costs; but at the time, it wasn't obvious at all. The answer to this question wouldn't have been easier to find with better institutional decision-making, either. It was simply a difficult question with no clear answer at that time.
COVID: How did we do? How can we know?

1. The UK began 2020 with an unspoken, dubiously voluntary version of this strategy. As the IFR firmed up, the backlash against this was so large that they were forced to deny that they ever intended any such thing. 

The goalposts of this post are: "what is the best we could actually do, just changing the opinion of say a few hundred elites?" Challenge trials were popular; I strongly predict variolation is different, and that popularity matters even if 5% of the youngest and maddest volunteer.


2. You missed the absolutely critical, sign-flipping b... (read more)

1zxcv4moMm good points. If I were dictator, I would still have a variolation process at least tested and developed for each new pathogen, which could be deployed as an emergency backup. If by threshold you mean the percentage thing, I would say that at 25% conterfactual harm, variolation is a reasonable option for countries that couldn't manage vaccines, if nobody else was going to give it to them.
COVID: How did we do? How can we know?

Sounds undoable.

1) unlike challenge trials, which had surprising popular appeal, this looks terrifying.
2) unlike challenge trials, you need to isolate billions of live cultures. Given real world biosecurity this is a nightmare.
3) I don't get the impression that the Indian government could move so fast, even though there are amazing private actors like the Serum Institute.

I don't want to shift the goalposts; my post also relies on a few things being viewed differently. But mine just requires a few hundred elites to get out the way.

1zxcv4moRe 1: could it work as opt in? The brave go first? Re 2: Wait why do you need to do that? Maybe you can have a sick person sneeze into a napkin and immediately rub it on the next person or whatever, so there is no intermediate storage step? Wrt biosecurity, a high percentage of the population was going to get covid / it's an available everywhere already anyway ? Re 3: Have a private actor do it on govt contract? Competing firms ranked by effectiveness and safety etc Re goalposts: What are you viewing as the goal post? I think I misunderstood you somewhere. What things are you viewing differently? Yeah this might be getting out of scope for the discussion but a good variolation technique have no manufacturing, no patents, no buying / selling negotiation. Random people in poor countries with almost no govt could variolate each other. If world order or the science machine breaks down (new pandemic kills all the world leaders or something) then the technique can still be used. And to get more out of scope, what's the harm reduction of a vaccine? Let's generously say the cost and side effects of getting vaccinated are 1% that of getting the disease itself. (Ignore spread reduction benefits for now.) At what percent would you consider variolation worthwhile as an option: 30%, 10%, 5%, 1%?