Stuart Buck

Executive Director @ Good Science Project
1177 karmaJoined Dec 2021


I lead a small think tank dedicated to accelerating the pace of scientific advancement by improving the conditions of science funding. As well, I'm a senior advisor to the Social Science Research Council. Prior to these roles, I spent some 9 years at Arnold Ventures (formerly the Laura and John Arnold Foundation) as VP of Research. 

How I can help others

Science policy, reproducibility, and philanthropy. 


Is the consensus currently that the investment in Twitter has paid off or is ever likely to do so? 

I could imagine making that case, but what's the point of all the Givewell-style analysis of evidence, or all the detailed attempts to predict and value the future, if in the end, what would have been the single biggest allocation of EA funds for all time was being proposed based on vibes? 

I did think Harris could have been slightly more aggressive in his questioning (as in, some level above zero). E.g., why would MacAskill even suggest that SBF might have have been altruistic in his motivations, even though we now know about the profligate and indulgent lifestyle that SBF led? MacAskill had to have known about that behavior at the time (why didn't it make him suspicious?).

 And why was MacAskill trying to ingratiate himself with Elon Musk so that SBF could put several billion dollars (not even his in the first place) towards buying Twitter? Contributing towards Musk's purchase of Twitter was the best EA use of several billion dollars? That was going to save more lives than any other philanthropic opportunity? Based on what analysis?

This statement cracked me up for some reason: "At this point, we are not actively making grants to further investigate these questions. It is possible we may do so in the future, though, so if you plan to research any of these, please email us."

I.e., this isn't an RFP (request for proposals). Instead, it's more like a RFINP, BMITFWDEK? Request for Information Not Proposals, But Maybe In The Future (We Don't Even Know)? 

OK, mostly joking -- in all seriousness, I haven't seen wealthy philanthropies release lists of ideas that are hopefully funded elsewhere, but maybe that actually makes sense! No philanthropy can fund everything in the world that might be interesting/useful. So maybe all philanthropies should release lists of "things we're not funding but that we hope to see or learn from." 

Great journalists are getting laid off all the time these days. You could find any number of professional and highly accomplished journalists for a tiny fraction of $800k per year. 

$800k per year? For one person to do investigative journalism? What would all that money be spent on? 

"Completely remove efficacy testing requirements, making the FDA a non-binding consumer protection and labeling agency."

That seems to be how the supplement market works under DSHEA. Is there any evidence that stuff marketed as supplements is any better at curing disease, preventing disease, improving health, etc., than the FDA pathway? Indeed, is there any evidence that the supplement market works well at all? We've known for decades that antioxidants don't actually improve health, yet they are still a huge market.  

So let me put it this way: 

If there is a future bioterrorist attack involving, say, smallpox, we can disaggregate quite a few elements in the causal chain leading up to that: 

  1. The NIH published the entire genetic sequence of smallpox for the world to see.
  2. Google indexed that webpage and made it trivially easy to find.
  3. Thanks to electricity and internet providers, folks can use Google.
  4. They now need access to a laboratory and all the right equipment.
    1. Either they need to have enough resources to create their own laboratory from scratch, or else they need to access someone's lab (in which case they run a significant risk of being discovered).
  5. They need a huge amount of tacit knowledge in order to able to actually use the lab --knowledge that simply can't be captured in text or replicated from text (no matter how detailed). Someone has to give them a ton of hands-on training. 
  6.  An LLM could theoretically speed up the process by giving them a detailed step-by-step set of instructions.
  7. They are therefore able to actually engineer smallpox in the real world (not just generate a set of textual instructions). 

The question for me is: How much of the outcome here depends on 6 as the key element, without which the end outcome wouldn't occur? 

Maybe a future LLM would provide a useful step 6, but anyone other than a pre-existing expert would always fail at step 4 or 5. Alternatively, maybe all the other steps let someone let someone do this in reality, and an accurate and complete LLM (in the future) would just make it 1% faster. 

I don't think the current study sheds any light whatsoever on those questions (it has no control group, and it has no step at which subjects are asked to do anything in the real world). 

In a way, the sarin story confirms what I've been trying to say: a list of instructions, no matter how complete, does not mean that people can literally execute the instructions in the real world. Indeed, having tried to teach my kids to cook, even making something as simple as scrambled eggs requires lots of experience and tacit knowledge.

Load more