3358Joined Jun 2015



A limit on compute designed to constrain OpenAI, Anthropic, or Google from training a new model sounds like a very high bar. I don't understand why that could easily be got around?


A variant on your proposal could be a moratorium on training new large models (e.g. OpenAI would be forbidden from training GPT-5, for example).

  • That would be more enforceable, because you need lots of compute to train a new model. I don't know how we would stop an academic thinking up new ideas on how to structure AI models better, and even if we could, it would be hard to disentangle this from alignment research.
  • It would probably achieve most of what you want. For someone who's worried about short timelines, reducing the scope for the scaling hypothesis to apply is probably pretty powerful, at least in the short term

Thanks very much Saulius. 

In SoGive's 2023 plans document, we said 

"An investigation of No Means No Worldwide was suggested to us by a member of the EA London community, who was excited to have an EA-aligned recommendation for a charity which prevents sexual violence. We have mostly completed a review of this charity, and were asked not to publish it yet because it used a study which is not yet in the public domain."

That said, part of the reason I didn't allude to NMNW is that my vague memory of the average was older (presumably my vague memory was wrong).


I don't see how we could implement a moratorium on AGI research that does stop capabilities research but doesn't stop alignment research?


Cool. To be clear, I think if anyone was reading your piece with any level of care or attention, it would be clear that you were comparing normal and lognormal, and not making any stronger claims than that.

Someone pinged me a message on here asking about how to donate to tackle child sexual abuse. I'm copying my thoughts here.

I haven't done a careful review on this, but here's a few quick comments:

  • Overall, I don't know of any charity which does interventions tackling child sexual abuse, and which I know to have a robust evidence-and-impact mindset.
  • Overall, I have the impression that people who have suffered from child sexual abuse (hereafter CSA) can suffer greatly, and tackling this is intractable. My confidence on this is medium -- I've spoken with enough people to be confident that it's true at least some of the time, but I'm not clear on the academic evidence.
  • This seems to point in the direction of prevention instead.
  • There are interventions which aim to support children to avoid being abused. I haven't seen the evidence on this (and suspect that high quality evidence doesn't exist). If I were to guess, I would guess that the best interventions probably do have some impact, but that impact is limited.
    • To expand on this: my intuition says that the less able the child is to protect themselves, the more damage the CSA does. I.e. we could probably help a confident 15-year old avoid being abused, however that child might get different -- and, I suspect, on average less bad -- consequences than a 5 year old; but helping the 5 year old might be very intractable. 
  • This suggests that work to support the abuser may be more effective. 
    • It's likely also more neglected, since donors are typically more attracted to helping a victim than a perpetrator.
    • For at least some paedophiles, although they have sexual urges toward children, they also have a strong desire to avoid acting on them, so operating cooperatively with them could be somewhat more tractable.
  • Unfortunately, I don't know of any org which does work in this area, and which has a strong evidence culture. Here are some examples:
    • I considered volunteering with Circles many years ago. They offer circles of accountability and support to perpetrators of CSA who were recently released from prison -- essentially this is a handful of volunteer who befriend a CSA perpetrator. I opted not to volunteer because they didn't have a strong enough evidence culture.
    • The Dunkelfeld project (aka Troubled Desire) seems to be doing work which has promise, but again, the evidence appears to be lacking.
  • I'm unclear on the geographic dimension to this. I.e. maybe the fact that costs are lower in the developing world could make it more effective there, but I'm unclear on this. I doubt that tractability would be much better there, which is normally what drives most of the benefit of operating in the developing world.
  • I'm not very optimistic about this achieving the cost-effectiveness of GiveWell Top Charity / SoGive Gold Standard status, unless the moral weights component of the model weighted the extent of suffering associated with CSA as very high on average (which might be valid).

If anyone is interested in this topic and wants to put aside a substantial sum (high 5 digits or six digits) then the next steps would involve a number of conversations to gather more evidence and check whether existing interventions are as lacking in evidence as I suspect. If so, the next step would be work on creating a new charity. It's possible that Charity Entrepreneurship might be interested in this, but I haven't spoken with them about this and I don't know their appetite. I'd be happy to support you on this, mostly because I know that CSA can be utterly horrific (at least some of the time). 


If someone gives me a book, I feel like I have no deadline for reading it. Which sometimes means I never read it. If it's a loan, it's more likely I'll read it at all.

The flipside of this dynamic is that I'm unlikely to accept a book if I don't think I'm likely to read it, or if I'm interested in reading it, but know that I won't have time for a while.


I agree with your claim that lognormal distributions are a better choice than normal. However this doesn't explain whether another distribution might be better (especially in cases where data is scarce, such as the number of inhabitable planets).

For example, the power law distribution has some theoretical arguments in its favour and also has a significantly higher kurtosis, meaning there is a much fatter tail.


It looks like the arguments in favour of a boycott would look stronger if there were a coherent AI safety activist movement. (I mean "activist" in the sense of "recruiting other people to take part, and grassroots lobbying of decision-makers", not "activist" in the sense of "takes some form of action, such as doing AI alignment research")


I haven't thought hard about how good an idea this is, but those interested might like to compare and contrast with ClientEarth.

Load more