Researcher of causal models and human-aligned AI at FHI | https://twitter.com/ryancareyai


Launching a new resource: 'Effective Altruism: An Introduction'

Let's look at the three arguments for focusing more on shorttermist content:

1. The EA movement does important work in these two causes
I think this is basically a "this doesn't represent members views" argument: "When leaders change the message to focus on what they believe to be higher priorities, people complain that it doesn’t represent the views and interests of the movement". Clearly, to some extent, EA messaging has to cater to what current community-members think. But it is better to defer to reason, evidence, and expertise, to the greatest extent possible. For example, when:

  • people demanded EA Handbook 2.0 refocus away from longtermism, or
  • Bostrom's excellent talk on crucial considerations was removed from effectivealtruism.org

it would have been better to focus on the merits of the ideas, rather than follow  majoritarian/populist/democratic rule. Because the fundamental goal of the movement is to focus on the biggest problems, and everything else has to work around that, as the movement is not about us. As JFK once (almost) said: "ask not what EA will do for you but what together we can do for utility"! Personally, I think that by the time a prioritisation movement starts deciding its priorities by vote, it should have reached an existential crisis. Because it will take an awful lot of resources to convince the movement's members of those priorities, which immediately raises the question of whether the movement could be outperformed by one that ditched the voting, for reason and evidence. To avoid the crisis, we could train ourselves, so that when we hear "this doesn't represent members views", we  hear alarm bells ringing...

2. Many EAs still care about or work on these two causes, and would likely want more people to continue entering them / 3. People who get pointed to this feed and don't get interested in longtermism (or aren't a fit for careers in it) might think that the EA movement is not for them.

Right, but this is the same argument that would cause us to donate to a charity for guide dogs, because people want us to continue doing so. Just as with funds, there are limits to the audience's attention, so it's necessary to focus on attracting those who can do the most good in priority areas.

Resources on the expected value of founding a for-profit start-up?

I looked at some literature on this question, considering various reference classes back in 2014: YC founders, Stanford Entrepreneurs, VC-funded companies.

The essence of the problem in my view is 1) choosing (and averaging over) good reference classes, 2) understanding the heavy tails, and 3) understanding that startup founders are selected to be good at founding (a correlation vs causation issue).

First, consider the first two points:

1. Make very sure that your reference class consists mostly of startups, not less-ambitious family/lifestyle businesses.

2. The returns of startups are so heavy-tailed that you can make a fair estimate based on just the richest <1% of founders in the reference class (based on the public valuation and any dilution, or based on the likes of Forbes billionaire charts.).

For example, in YC, we see that Stripe and AirBnB are worth ~$100B each, and YC has maybe graduated ~2k founders, so each founder might make ~$100M on-expectation

I'd estimated $6M and $10M on-expectation for VC-funded founders and Stanford-founders respectively.

A more controversial reference class is "earn-to-give founders". Sam Bankman-Fried has made about $10B from FTX. If 50 people have pursued this path, the expected earnings are $200M.

The YC and "earn-to-give" founder classes are especially small. In aggregate, I think we can say that the expected earnings for a generic early-stage EA founder are in the range of $1-100M, depending on their reference class (including the degree of success and situation). Having said this, 60-90% of companies make nothing (or lose money). With such a failure rate, checking against one's tolerance for personal risk is important.

Then, we must augment the analysis by considering the third point:

3. Startup founders are selected to be good at founding (correlation vs causation)

If we intervene to create more EA founders, they'll perform less well than the EAs that already chose to found startups, because the latter are disproportionately suited to startups. How much worse is unclear - you could try to consider more and less selective classes of founders (i.e. make a forecast that conditions on / controls for features of the founders) but that analysis takes more work, and I'll leave it to others.

Some quick notes on "effective altruism"

EA popsci would be fun! 

§1. The past was totally fucked. 

§2. Bioweapons are fucked. 

§3. AI looks pretty fucked. 

§4. Are we fucked? 

§5. Unfuck the world!

RyanCarey's Shortform

Good point - this has changed my model of this particular issue a lot (it's actually not something I've spent much time thinking about).

I guess we should (by default) imagine that if at time T you recruit a person, that they'll do an activity that you would have valued, based on your beliefs at time T.

Some of us thought that recruitment was even better, in that  the recruited people will update their views over time. But in practice, they only update their views a little bit. So the uncertainty-bonus for recruitment is small. In particular, if you recruit people to a movement based on messaging in cause A, you should expect relatively few people to switch to cause B based on their group membership, and there may be a lot of within-movement tensions between those that do/don't.

There are also uncertainty-penalties for recruitment. While recruiting, you crystallise your own ideas. You give up time that you might've used for thinking, and for reducing your uncertainties.

On balance, recruitment now seems like a pretty bad way to deal with uncertainty.

RyanCarey's Shortform

How the Haste Consideration turned out to be wrong.

In The haste consideration, Matt Wage essentially argued that given exponential movement growth, recruiting someone is very important, and that in particular, it’s important to do it sooner rather than later. After the passage of nine years, noone in the EA movement seems to believes it anymore, but it feels useful to recap what I view as the three main reasons why:

  1. Exponential-looking movement growth will (almost certainly) level off eventually, once the ideas reach the susceptible population. So earlier outreach really only causes the movement to reach its full size at an earlier point. This has been learned from experience, as movement growth was north of 50% around 2010, but has since tapered to around 10% per year as of 2018-2020. And I’ve seen similar patterns in the AI safety field.
  2. When you recruit someone, they may do what you want initially. But over time, your ideas about how to act may change, and they may not update with you. This has been seen in practice in the EA movement, which was highly intellectual and designed around values, rather than particular actions. People were reminded that their role is to help answer a question, not imbibe a fixed ideology. Nonetheless, members’ habits and attitudes crystallised - severely - so that now, when leaders change the message to focus on what they believe to be higher priorities, people complain that it doesn’t represent the views and interests of the movement! The same thinking persists several years later. [Edit: this doesn't counter the haste consideration per se. It's just one way that recruitment is less good than one might hope-> See AGB's subthread].
  3. The returns from one person’s movement-building activities will often level off. Basically, it’s a lot easier to recruit your best friends, than the rest of your friends. Much easier to recruit your friends of friends, than their friends. Harder to recruit once you leave university as well. I saw this personally - the people who did the most good in the EA movement with me, and/or due to me were among my best couple of friends from high school, and some of my best friends from the local LessWrong group. These efforts at recruitment during my university days seem potentially much more impactful than my direct actions. However, more recent efforts at recruitment and persuasion have also made differences, but they have been more marginal, and seem less impactful than my own direct work.

Taking all of this together, I’ve sometimes recommended university students not spend too much time on recruitment. The advice especially applies to top students, who could become a distinguished academic or policymaker later on - as their time may be better spent preparing for that future. My very rough sense is that for some, the optimal amount of time to spend recruiting may be one full-time months. For others, a full-time year. And importantly, our best estimates may change over time!

RyanCarey's Shortform

A step that I think would be good to see even sooner is any professor at a top school getting in a habit of giving talks at gifted high-schools. At some point, it might be worth a few professors each giving dozens of talks per year, although it wouldn't have to start that way.

Edit: or maybe just people with "cool" jobs. Poker players? Athletes?

Some quick notes on "effective altruism"

What kinds of names do you think would convey the notion of prioritised action while being less self-aggrandising?

RyanCarey's Shortform

High impact teachers? (Teaching as Task Y)

The typical view, here, on high-school outreach seems to be that:

  1. High-school outreach has been somewhat effective, uncovering one highly capable do-gooder per 10-100 exceptional students.
  2. But people aren't treating it with the requisite degree of sensitivity: they don't think enough about what parents think, they talk about "converting people", and there have been bad events of unprofessional behaviour.

So I think high-school outreach should be done, but done differently. Involving some teachers would be useful step toward professionalisation (separating the outreach from the rationalist community would be another). 

But (1) also suggests that teaching at a school for gifted children could be a priority activity in itself. The argument is if a teacher can inspire a bright student to try to do good in their career, then the student might be manifold more effective than the teacher themselves would have been, if they had tried to work directly on the world's problems. And students at such schools are exceptional enough (Z>2) that this could happen many times throughout a teacher's career.

This does not mean that teaching is the best way to reach talented do-gooders. But it doesn't have to be, because it could attract some EAs who who wouldn't suit outreach paths. It leads to stable and respected employment, involves interpersonal contact that can be meaningful, and so on (at least, some interactions with teachers were quite meaningful to me, well before EA entered my picture).

I've said that teachers could help professionalise summer schools, and inspire students. I also think that a new high school for gifted altruists could be a high-priority. It could gather talented altruistic students together, so that they have more social support, and better meet their curricular needs (e.g. econ, programming, philosophy, research). I expect that such a school could attract great talent. It would be staffed with pretty talented at knowledgeable teachers. It would be advised by some professors at top schools. If necessary, by funding scholarships, it could grow its student based arbitrarily. Maybe a really promising project.

Some quick notes on "effective altruism"

A friend's "names guy" once suggested calling the EA movement "Unfuck the world"...

Load More