poppinfresh's Quick takes

One of the weaker parts of the Situational Awareness essay is Leopold's discussion of international AI governance.

He argues the notion of an international treaty on AI "fanciful", claiming that:

• It would be easy to "break out" of treaty restrictions
• There would be strong incentives to do so
• So the equilibrium is unstable

That's basically it - international cooperation gets about 140 words of analysis in the 160 page document.

I think this is seriously underargued. Right now it seems harmful to propagate a meme like "International AI cooperation is fanciful". 

This is just a quick take, but I think it's the case that:

• It might not be easy to break out of treaty restrictions. Of course it will be hard to monitor and enforce a treaty. But there's potential to make it possible through hardware mechanisms, cloud governance, inspections, and other mechanisms that we haven't even thought of yet. Lots of people are paying attention to this challenge and working on it.
• There might not be strong incentives to do so. Decisionmakers may take the risks seriously and calculate the downsides of an all-out race exceed the potential benefits of winning. Credible benefit-sharing and shared decision-making institutions may convince states they're better off cooperating than trying to win a race.
• International cooperation might not be all-or-nothing. Even if we can't (or shouldn't!) institute something like a global pause, cooperation on more narrow issues to mitigate threats from AI misuse and loss of control could be possible. Even in the midst of the Cold War, the US and USSR managed to agree on issues like arms control, non-proliferation, and technologies like anti-ballistic missile tech.

(I critiqued a critique of Aschenbrenner's take on international AI governance here, so I wanted to clarify that I actually do think his model is probably wrong here.)

I left this as a comment on a post which has since been deleted,  so wanted to repost it here (with a few edits).

Some, though not all, men in EA subscribe to an ask culture philosophy which endorses asking for things, including dates, even when the answer is probably no. On several occasions this has caused problems in EA organizations and at EA events. A small number of guys endorsing this philosophy can make multiple  women feel uncomfortable.

I think this is pretty bad. Some people point out that the alternative, guess culture, is more difficult for people who struggle to read social cues. But if there's a trade-off between making this community safe and comfortable for women and making it easier for guys to find dates, then, sorry, the former is the only reasonable choice.

Probably there are formal solutions, such as clearer rules and codes of conduct, which could help a bit. But I also think there's a cultural element, created and enforced through social norms, that will only change when EAs (especially men) choose to notice and push back on questionable behaviour from our acquaintances, colleagues, and friends.

Death by feedback

It's not unusual to see a small army of people thanked in the "Acknowledgements" section of a typical EA Forum post. But one should be careful not to get too much feedback. For one, the benefits of more feedback diminish quickly, while the community costs scale linearly. (You gain fewer additional insights from the fifth person who reads your draft than you do from the first, but it takes the fifth person just as long to read and comment.)

My biggest worry, though, is killing my own vision by trying to incorporate comments from too many other people. This is death by feedback. If you try to please everyone, you probably won't please anyone. 

There are lots of different ways one could write about a given topic. Imagine I'm writing an essay to convince EA Forum readers that the resplendent quetzal is really cool. There's lots I could talk about: I could talk about its brilliant green plumage and long pretty tail; I could talk about how it's the national animal of Guatemala, so beloved that the country's currency is called the quetzal; or I could talk about its role in Mesoamerican mythology. Different people will have different ideas about which tack I should take. Some framings will be more effective than others. But any given framing can be killed by writing a scattered, unfocused, inconsistent essay that tries to talk about everything at once.

Sure, go ahead and get feedback from a few people to catch blunders and oversights. It's pretty awesome that so many clever, busy people will read your Forum posts if you ask them to. But don't Frankenstein your essay by stitching together different visions to address all concerns. It's important to recognize that there's not a single, ideal form a piece can approach if the author keeps gathering feedback. "Design by committee" is a perjorative phrase for a reason.

Thanks to absolutely nobody for giving feedback on this post.

Many organizations I respect are very risk-averse when hiring, and for good reasons. Making a bad hiring decision is extremely costly, as it means running another hiring round, paying for work that isn't useful, and diverting organisational time and resources towards trouble-shooting and away from other projects. This leads many organisations to scale very slowly.

However, there may be an imbalance between false positives (bad hires) and false negatives (passing over great candidates). In hiring as in many other fields, reducing false positives often means raising false negatives. Many successful people have stories of being passed over early in their careers. The costs of a bad hire are obvious, while the costs of passing over a great hire are counterfactual and never observed.

I wonder  whether, in my past hiring decisions, I've properly balanced the risk of rejecting a potentially great hire against the risk of making a bad hire. One reason to think we may be too risk-averse, in addition to the salience of the costs, is that the benefits of a great hire could grow to be very large, while the costs of a bad hire are somewhat bounded, as they can eventually be let go.

I think it's possible there's too much promotion on the EA Forum these days. There are lots of posts announcing new organizations, hiring rounds, events, or opportunities. These are useful but not that informative, and they take up space on the frontpage. I'd rather see more posts about research, cause prioritization, critiques and redteams, and analysis. Perhaps promotional posts should be collected into a megathread, the way we do with hiring.

In general it feels like the signal-to-noise ration on the frontpage is lower now than it was a year ago, though I could be wrong. One metric might be number of comments - right now, 5/12 posts I see on the frontpage have 0 comments, and 11/12 have 10 comments or fewer.

The Against Malaria Foundation is described as about 5-23x more cost-effective than cash transfers in GiveWell's calculations, while Founders Pledge thinks StrongMinds is about 6x more cost-effective.

But this seems kind of weird. What are people buying with their cash transfers? If a bednet would be 20x more valuable, than why don't they just buy that instead? How can in-kind donations (goods like bednets or vitamins) be so much better than just giving poor people cash? 

I can think of four factors that might help explain this.

• Lack of information. Perhaps beneficiaries aren't aware of the level of malaria risk they face, or aren't aware of the later-life income benefits GiveWell models. Or perhaps they are aware but irrationally discount these benefits for whatever reason.
• Lack of markets. Cash transfer recipients usually live in poor, rural areas. Their communities don't have a lot of buying power. So perhaps it's not worth it for, e.g., bednet suppliers to reach their area. In other words, they would purchase the highly-valuable goods with their cash transfers, but they can't, so they have to buy less valuable goods instead.
• Economies of scale. Perhaps top-rated charities get a big discount on the goods they buy because they buy so many. This discount could be sufficiently large that, e.g., bednets are the most cost-effective option when they're bought at scale, but not the most cost-effective option for individual consumers.
• Externalities. Perhaps some of the benefits of in-kind goods flow to people who don't receive cash transfers, such as children. These could  account for their increased cost-effectiveness if cash transfer recipients don't account for externalities.

I think each of these probably plays a role. However, a 20x gap in cost-effectiveness is really big. I'm not that convinced that these factors are strong enough to fully explain the differential. And that makes me a little bit suspicious of the result.

I'd be curious to hear what others think. If others have written about this, I'd love to read it. I didn't see a relevant question in GiveWell's FAQs.

One of the most common lessons people said they learned from the FTX collapse is to pay more attention to the character of people with whom they're working or associating (e.g. Spencer Greenberg, Ben Todd, Leopold Aschenbrenner, etc.). I agree that some update in this direction makes sense. But it's easier to do this retrospectively than it is to think about how specifically it should affect your decisions going forward.

If you think this is an important update, too, then you might want to think more about how you're going to change your future behaviour (rather than how you would have changed your past behaviour). Who, exactly, are you now distancing yourself from going forward?

Remember that the challenge is knowing when to stay away basically because they seem suss, not because you have strong evidence of wrongdoing.