We haven’t done a full annual review of 2023 and the complete data isn’t in yet, so we haven't done a thorough assessment of the answer to your question yet. The answers to your question probably differ quite a bit programme to programme. But here are a few thoughts that seemed relevant to me:
On web:
Over the past couple of years, the biggest predictor of change in web engagement time appears to be changes in our marketing spending. In 2022 we substantially increased our marketing spend. In 2023 our marketing spend was not dramatically larger than in 2022. This is reflected in the web engagement time metrics. (We are actively investigating the cost-effectiveness of marginal marketing spending, and are not fundraising for marketing as part of this public fundraising round as it is already being covered by Open Philanthropy.)
We have also put more effort into driving off-site engagement time in 2023, e.g. via our AI video, improvements to our newsletter, etc. This is not included in the engagement time metrics in the chart, but we estimate that in 2023 we grew off-site engagement time notably more than we did on-site engagement time.
On podcast:
The drivers of engagement with the podcast are more mysterious to me, and I have trouble making accurate predictions of future engagement time with the podcast. Viewed on a quarterly basis, growth in the podcast appears to be healthy.
On advising:
In 2023 we focused more on growing and systematising headhunting, active outreach and systems, and relatively less on increasing call numbers.
We didn't make as many calls as we had hoped to, due in part to a manager on the team leaving.
We also put relatively more focus on improving call quality, for example by putting in place feedback systems. This was a focus because we grew the team in 2021 and 2022 and wanted more systems to keep everyone in sync and ensure continued quality.
On job board:
We've actually reduced our FTE input into the job board in 2023, but we are still seeing solid quarter-on-quarter growth.
Additional points:
Some of our staff growth came from hires to our internal systems team, which should strengthen our capacity over time but won’t result in direct improvements on these metrics.
We do expect some diminishing returns to staff growth over time. I'll address this in another comment on this thread.
(1) “we seem to be hitting diminishing returns in outreach encouraging more people to apply to advising...” (page 7 under current challenges)
and again
(2) “overall, we’d guess that 80,000 Hours continued to see diminishing returns to its impact per staff member per year.” (on page 10 under impact evaluation)
What is the strategy/argument for “expand the team” being the best intervention for increasing organizational outreach and subsequent impact? Is it really just a capacity issue or could it be a scope issue?
Thanks for the question. To be clear, we do think growing the team will significantly increase our impact in expectation.
We do see diminishing returns on several areas of investment, but having diminishing returns is consistent with significantly increasing impact.
Not all of our impact is captured in these metrics. For example, if we were to hire to increase the quality of our written advice even while maintaining the same number of website engagement hours, we’d expect our impact to increase (though this is of course hard to measure).
In our view, investments in 80k’s growth are still well above the cost-effectiveness bar for similar types of organisations and interventions in the problem areas we work on.
Hey John, unfortunately a lot of the data we use to assess our impact contains people’s personal details or comes from others’ analyses that we’re not able to share. As such, it is hard for me to give a sense of how many times more cost-effective we think our marginal spending is compared with the community funding bar.
But the original post includes various details about assessments of our impact, including the plan changes we’ve tracked, placements made, the EA survey, and the Open Philanthropy survey. We will be working on our annual review in spring 2024 and may have more details to share about the impact of our programmes then.
If you are interested in reading about our perspective on our historical cost-effectiveness from our 2019 annual review, you can do sohere.
On a related note to my other comment on this post:
A lot of organizations are acknowledging the impact of FTX on their work which is important but I would also like to see an EA organization try to evaluate the positive or negative impact switching to longtermism has had on their ability to attract talent, donors etc.
You say you want to diversify assets but Open Phil still holds a commanding 80% of your bottom line - and both orgs have become much more longtermist in recent years. If OP is going to just prop up 80k for the next several years because it serves their aims, why would I fund 80k when there is now a considerable gap in the landscape for a new career service org that caters to the other cause priorities of EA?
a new career service org that caters to the other cause priorities of EA?
I'm guessing you are familiar with Probably Good? They are doing almost exactly the thing that you describe here. They are also accepting donations, and if you want to support them you can do so here.
Thanks for engaging with this post! A few thoughts prompted by your comment in case they are helpful:
80k has been interested in longtermism-related causes for many years, including many years in which we’ve seen a lot of growth. We were interested in longtermism for several years before we received our first grant from Open Philanthropy.
We believe there’s still a lot of need for talent in the problems areas that we focus on, so we don’t think there’s a strong reason for us to shift our focus on that front — at least for the time being.
In evaluating our impact, you should consider whether the causes where we focus seem most pressing to you. If you think our focus areas are not that pressing, we think it’s reasonable to be less interested in donating to us.
We’re happy to see others offering alternatives to our career advice — this kind of competition is healthy and we are keen to encourage it in the ecosystem.
All that said, we do have a lot of advice to people who are not that interested in longtermism. For example, our job board features opportunities for people working on global health and animal issues, and our career guide offers advice that is widely applicable, including about how readers could approach thinking through the question of which problems are most pressing for themselves.
We’re happy to see others offering alternatives to our career advice — this kind of competition is healthy and we are keen to encourage it in the ecosystem.
I wanted to chime in and say that while a lot of people / organizations say things like this, in my experience, 80,000 really does mean it and follows through. When we were setting up Probably Good (and not only then) the amount of encouragement and help we received from Michelle, Niel and others there has been incredible.
Update: Our spin-out from Effective Ventures is now complete. We've established 80,000 Hours Limited and 80,000 Hours Foundation, each with their own board. See our recent blog post for details.
Here’s a quick update to say that we’ve closed this fundraising round.
If you’re still interested in donating to 80,000 Hours, we always welcome donations through our Giving What We Can donation page and are excited to hear from potential donors at [email protected].
We have also updated the donate page of our site — updating the metrics to cover the whole of 2023 and making an adjustment to account for cookie banner changes in 2023 which affects our growth over time metrics. Going forward, our page is a better source of up-to-date information than this forum post.
This is a crosspost from the new Animal Welfare Alignment Newsletter by Anima International. You can subscribe on Substack if you are interested in following these efforts. Audio reading also available on Substack.
The goals of this post are to:
1. Raise a question I see as crucially important to the goal of aligning AI to animal welfare...
Hello! I'm Justin Portela. I got hired by GWWC to make YouTube videos after AI in Context did such a kickass job.
My channel is using that same cinematic, high-production value beauty to talk about everything in the EA universe that isn't AI.
...
This is a linkpost for Request for Proposals: Research and Applied Work on Digital Minds.
I'm glad to announce a request for proposals for research and applied work on digital minds at Longview Ph...
80,000 Hours is fundraising. We’re currently seeking $1,200,000 to support our general activities (excluding marketing and grantmaking) until the middle of 2024.
Spin out
We are excited to share that 80,000 Hours has officially decided to spin out as a project from Effective Ventures Foundation UK and Effective Ventures US (known collectively as Effective Ventures) and establish an independent legal structure.
We’re incredibly grateful to the Effective Ventures leadership and team and the other orgs for all their support, particularly in the past year as our community faced a lot of challenges. They devoted countless hours and enormous effort to helping ensure that we and the other orgs could pursue our missions.
And we deeply appreciate their support in our spin-out. They recently announced that all of the other organisations will likewise become their own legal entities; we’re excited to continue to work alongside them to improve the world.
Back in May, we investigated whether it was the right time to spin out of our parent organisations. We’ve considered this option at various points in the last three years.
There have been many benefits to being part of a larger entity since our founding. But as 80,000 Hours and the other projects have grown, we concluded we can now best pursue our mission and goals independently. EV leadership approved the plan.
Becoming our own entity will allow us to:
Match our governing structure to our function and purpose
Design operations systems that best meet our staff’s needs
Reduce interdependence on other entities that raises financial, legal, and reputational risks
There’s a lot for us to do. We’re currently in the process of finding a new CEO to lead us in our next chapter. We’ll also need a new board to oversee our work and new staff for our internal systems team and other growing programmes.
Which brings us to our next item: we’re fundraising!
Fundraising
We’re currently seeking $1,200,000 to support our general activities (excluding marketing and grantmaking) until the middle of 2024.
This post has more information about us, our track record, and our current fundraising round. You can donate directly or view our fundraising page for additional information and ways to contact us.
About us
At 80,000 Hours, we provide research and support to help students, recent graduates, and others have high-impact careers.
Our goal is to get talented people working on the world’s most pressing problems. We focus on problems that threaten the long-term future, including risks from artificial intelligence, catastrophic pandemics, and nuclear war.
To achieve our goal, we:
Reach people who might be interested through marketing, engaging and user-friendly content, and word-of-mouth.
Introduce people to information, frameworks, and ideas which are useful for having a high-impact career and help them get excited about contributing to solving pressing global problems.
Support people in transitioning to careers that contribute to solving pressing global problems.
We’re currently running a selection process for a long-term CEO; we have three internal candidates and an open advertisement for external candidates. We expect to know the outcome of this selection process in the next month or two.
Our track record
We think that donating to 80,000 Hours is a promising way for people with similar priorities as us to have an impact.
We have a strong track record of encouraging people to change their career plans to focus on pressing problems, helping them find impactful positions, and introducing them to the effective altruism community.
Here, we’ll go through a few recent sources of evidence about our impact. You may also want to explore our repository of organisation evaluations, which cover our historical impact and progress, and you can reach out to [email protected] if you have any questions.
Plan changes
We have tracked hundreds of cases where people have made a major decision to pursue a higher-impact path – such as deciding to go to grad school, switching causes, or taking a particular job – which they say they might not have done without us.
We call these plan changes. Because careers are so long, we think an average plan change influences how someone spends tens of thousands of hours. We estimate that this is equivalent to hundreds of thousands of dollars worth of value to the world over the course of each person’s lifetime.
As of the end of 2022, we had tracked 604 plan changes. We estimate that the true number of plan changes for this period was closer to 2,000, because we only expect to find out about a fraction of all plan changes.
Here are four examples of plan changes (click 'read more' to see any of the links within the images):
Top organisations working on global catastrophic risk reduction regularly tell us that we’re their best source of referrals.
The table below shows some of the roles people have found on our job board — though we expect this to underestimate the true number of placements attributable to us.
We also know of about 27 times where our headhunting service introduced an organisation to a candidate who they hadn’t already been considering and then that person was hired for the role.[2]
Open Philanthropy’s 2020 survey
Open Philanthropy recognised 80,000 Hours as one of seven organisations and individuals that “seemed to have been most impactful on our respondents robustly, across multiple metrics,” highlighting the influence of all of our programmes.
According to Open Philanthropy’s overall impact-weighted measure[3] of how influential different factors have been:
Engagement with 80,000 Hours was tied as the most mentioned factor in the free text questions about what increased respondents’ positive impact on the world. When respondents were asked to name the top four of these factors that influenced them, 80,000 Hours was the 4th most mentioned factor.
When respondents were prompted to assign influence to a list of factors, 80,000 Hours was the 2nd most influential factor.
The 2023 Open Philanthropy survey has not yet been published. But they have shared with us that over the years 2020-2022, we have positively influenced the careers of a similar proportion of survey respondents, and, they think, a greater absolute number of people.
2022 EA survey
When survey respondents were asked which factors were “important for getting involved in EA,” 58% mentioned 80,000 Hours, making it the most commonly mentioned factor.
Some of our donors have asked whether 80,000 Hours is a strong donation opportunity from the perspective of AI safety. This section outlines some of our work in this area.
We’ve sent 10 research newsletters about AI. Our newsletter has more than 350,000 subscribers.
We’ve covered news topics like new governance initiatives and promoted our AI-focused podcast and website releases.
We’ve carried out 48 headhunts for AI-focused roles at 20 different orgs.
We had advising calls with many people excited about working on AI, helping them to think through their uncertainties, test out their fit, and build their network in AI.
This kind of work means that we have a history of helping people on their path to working on AI safety. Neel is one example of this.
If you’d like more information about our work on AI or another particular cause area, please contact [email protected].
What we’re fundraising for
Briefly: we’re fundraising for the final 20% of nine months’ worth of our general budget, including hiring but excluding marketing and grantmaking – coming to $1,200,000.
In more detail, we’re fundraising for:
Our general budget — we are seeking to cover the costs of our general budget, which supports our staff and products.
It does not include our paid marketing and grantmaking, which we have already received funding for. If you’d like to know more about those budgets, please contact [email protected].
Nine months of our general budget — in this fundraising round, we’re aiming to cover our general budget for the period Q4 2023 to Q2 2024.
As we’re currently running a selection process for a long-term CEO, we think this shorter-term maintenance approach is appropriate so that the long-term CEO can contribute to our longer-term planning and next fundraising round.
We currently aim to keep 12 months of “keeping the lights on” runway in reserves, because it’s difficult to change our income and expenses quickly. Through some underspend and reduced expansion in 2023, our existing reserves already cover the runway for the 12 months after Q2 2024, so we’re not fundraising for this now. Our reserves also slightly reduce the amount we’re asking for from our budget — which you can see in the tables below.
20% of our general budget for that period — we’ve already fundraised for 80% of our general budget from Open Philanthropy, so we’re fundraising for the final 20%. We value having diverse funding sources and are excited to grow our network of donors.
Hiring — we will continue to grow our team so that we can overcome capacity constraints and increase the impact of our existing programmes. In particular, we are currently hiring for people to join our one-on-one team in advising, headhunting, and systems capacities, and we expect to grow our internal systems and web teams in the near future.
This is our general budget for Q4 2023 – Q2 2024, compared to our actuals in the preceding nine months.
This is how we calculated our fundraising ask:
As staff costs take up a large proportion of our budget, here is a visualisation of the full-time-equivalent staff working on 80,000 Hours and our forecast for 2024:
*Includes long term contractors and 80,000 Hours’ share of Effective Ventures’ operations staff.
To contextualise how this staff growth affects our output and impact, here are our annual key lead metrics[4] for the same period:
As of December 21st, we've updated this claim because we spotted a mistake. It previously said we'd had "over 4.5 million readers on our site", but this is the metric for site visits (which we've now amended it to). We've had around 2 million unique visitors on the site this year.
Note that most (>70%) of these placements are attributable to our headhunting efforts between 2018 and 2020. In 2023, we began re-investing in headhunting, but don’t yet have much placement feedback for these due to hiring lead times.
Open Philanthropy reported both weighted and unweighted rankings for each of the different influence factors. The rankings reported above are based on the weighted rankings, which include Open Philanthropy’s quantitative assessment of the value of the respondents career. Here’s a breakdown of how 80,000 Hours’ ranking differ when using the unweighted rankings:
We’ve used our key lead metrics here. While the information in our track record gives a better picture of our historical impact, our lead metrics show the outputs of our growing team. We don’t think our track record captures this growth because we estimate that there’s a 1–4 year lag between the primary year that people engage with our service and the time of their plan change.
74
More posts like this
417
Stepping down from GWWC: So long, and thanks for all the shrimp
I'll ask the obvious awkward question:
Staff numbers are up ~35% this year but the only one of your key metrics that has shown significant movement is "Job Vacancy Clickthroughs".
What do you think explains this? Delayed impact, impact not caught by metrics, impact not scaling with staff - or something else?
Hey George —thanks for the question!
We haven’t done a full annual review of 2023 and the complete data isn’t in yet, so we haven't done a thorough assessment of the answer to your question yet. The answers to your question probably differ quite a bit programme to programme. But here are a few thoughts that seemed relevant to me:
On web:
On podcast:
On advising:
On job board:
Additional points:
Equally curious about the push to grow the team if not seeing significant increase in impact, especially given the $2M marketing push this past year.
In 80K’s 2021-2022 Review it mentioned:
(1) “we seem to be hitting diminishing returns in outreach encouraging more people to apply to advising...” (page 7 under current challenges)
and again
(2) “overall, we’d guess that 80,000 Hours continued to see diminishing returns to its impact per staff member per year.” (on page 10 under impact evaluation)
What is the strategy/argument for “expand the team” being the best intervention for increasing organizational outreach and subsequent impact? Is it really just a capacity issue or could it be a scope issue?
Thanks for the question. To be clear, we do think growing the team will significantly increase our impact in expectation.
I think this comment would be more persuasive if it shared some evidence or reasoning as to why its claims are likely true
Hey John, unfortunately a lot of the data we use to assess our impact contains people’s personal details or comes from others’ analyses that we’re not able to share. As such, it is hard for me to give a sense of how many times more cost-effective we think our marginal spending is compared with the community funding bar.
But the original post includes various details about assessments of our impact, including the plan changes we’ve tracked, placements made, the EA survey, and the Open Philanthropy survey. We will be working on our annual review in spring 2024 and may have more details to share about the impact of our programmes then.
If you are interested in reading about our perspective on our historical cost-effectiveness from our 2019 annual review, you can do so here.