Career choiceCareer choice
In-depth career profiles, specific job opportunities, and overall career guidance
Posts tagged Career choice
Quick takes
Career choice (in effective altruism) is about the ethical and practical considerations relevant to finding a high-impact career.
Although people spend more time on their jobs than on any other activity besides sleep—about 80,000 hours over the course of a typical life[1]—career choice often gets little attention in discussions on how to live ethically, compared to considerations like lifestyle changes (e.g. switching to local produce) or volunteering. But choosing an impactful career may be one of the most promising options for someone who wants to improve the world.[2]
One of the earliest publications on the subject was a youth essay by Karl Marx, who held that we should choose the career "which offers us the widest scope to work for mankind". Such a career, Marx claimed, would be best not only altruistically but also for the person pursuing it, since "experience acclaims as happiest the man who has made the greatest number of people happy."[3] In contemporary philosophy, there is a small literature on whether it is permissible to pursue a non-altruistic rather than an altruistic career, but almost no discussion of how ethical different careers are, or how to approach this question.[4]
The organization 80,000 Hours specializes in research about and support for impact-oriented career choice and has developed a framework for comparing the social impact of different careers.
Holden Karnofsky has identified four alternative frameworks for making career choice decisions: paths to particular roles working on particular causes; aptitudes a candidate can build in a variety of roles and causes and apply to a variety of jobs; causes to which a candidate can contribute with their career; and heuristics, such as "Do work you can be great at" or "Do work that builds your career capital and gives you more options."[5] 80,000 Hours uses the "paths" framework, while Karnofsky has focused mostly on the aptitude framework, though he advises candidates to consider multiple frameworks.
Aimed at an audience newer to EA/AI safety
It seems like many people have been confused lately about why 1) AI safety orgs are in desperate need of good hires and 2) org hiring rates are very low. My understanding is that there are many reasons for this, but a 2021 post states it pretty well:
"A lot of EA content is subtle and detailed, and easy to accidentally compress into something misleading. For example, 80k might write a nuanced article saying 'You should focus on talent gaps, not funding gaps', but this gets translated into 'EA is talent constrained', and then people repeat that phrase without linking to the article, and then many people form an inaccurate belief that EA needs 'pretty talented people', rather than 'EA needs very specific talents that are missing.'"
The best career advice seems to be to focus on developing a rare and valuable skill that orgs currently need. In AI safety, this skill is a combination of specific context, agency, mission-alignment, and strategic vision. You can develop it by reading a lot, writing out your thoughts, talking to people at events, and completing real projects.
It's hard to do really impactful work. You need a plausible explanation for why your efforts are the best way your salary can be spent. This takes time, effort, and careful strategizing. Most of the world has high schools, colleges, and internships to train people for jobs, but a similar system is only recently becoming legible for AI safety and EA. The rare skills are rare for a reason. Still, I think the broader EA community could do a better job of communicating what it means for these "talent-gaps" to exist.
I've had some interesting conversations this week with hiring managers for my SPAR project of improving hiring processes for new (smaller org <7 f/t employees).
I couple of best practices are coming out of it, on why AI Safety orgs are struggling with hiring ,and especially for important specialized roles:
The Founder's core job is to: Set Strategy, Raise funds, and Hire. Organizations need to source candidates widely. Not relying upon just inside referrals.
MATS Residency applications close this Friday, Oct 31. The Residency is built for experienced researchers with an established track record, ready to do independent, full-time AI safety work.
Residents get:
• $155k–$285k compensation
• Abundant compute
• Support from leading AI safety researchers
If you know someone with a strong research background who's been thinking about AI safety, please send this their way. Apply: matsprogram.org/residency
I'm continuing the work on my SPAR project, and I met with a hiring manager at an AI Safety org this week. Here's some interesting things I've learned:
- They use a multiple of tools and processes - a mixture of HR tools, AI and human review to score and review applications and work tests. They stated the process felt "disjointed".
- Work tests are developed for multiple roles are broken down into 2-3 types of tests:
- General - a foundational test that each applicant receives at this stage to map to cultural fit
- Role specific - work problem (based upon junior or senior role type) that is reviewed by human reviewers.
- Work tests are developed by an internal team (not just the hiring manager) and they use archives of past (and high scoring responses) + team to create tests. Scoring (initial screening is done by AI with human review).
- Their biggest frictions in the hiring process:
- Not being able to review in a timely manner completed work tests, and therefore their concern is they are losing high quality candidates to other orgs.
- When the potential tidal increase of more candidates coming, due to expected IPO's, then their own hiring strategy may be to only hire from referrals.
This conversation makes me realize that the hiring process is not only being hindered by processes (disjointed or non-existent) but also that most orgs (even with processes or tools) are not ready for the influx of people that could potentially look to AI Safety orgs as how they can contribute.
Therefore, my two may crux's that I need to explore more (with more discussions with AI Safety hiring managers) are the following:
- Is the hiring process inefficient due to the process (or lack of it) and a Playbook for hiring managers would help?
- How are work tests being written, how are the being scored and is the "reviewer" knowledgeable about the specific role that they can determine if the applicant has not only done this work in the past (and knows what they are talking about) but also is it good work (and not just a basic understanding)?
My ask for help: I'm looking for AI Safety founders or hiring managers who would be willing to conduct a short interview (15 mins) with me to help me understand the issues that you are facing. All information that you share with me will be only shared with my SPAR mentor, and my final published report will only include generialiations (and not specifics to any individual or org) in regards to your internal processes. I can be reached at: achartersmith@gmail.com
With the large expected inflow of philanthropic capital, there has been a surge in interest in entrepreneurship among those who otherwise might not have pursued it. As we pursue promising new ideas, many ventures will fail. Some of these failures will be because the idea was not as good as it seemed, but many will fail because the entrepreneurs executed it poorly.
I'm concerned we will abandon good ideas due to poor execution.
Founders new and old should strive to make their successes and failures publicly visible (though these stories should be read critically given founder biases). More importantly, we should be skeptical of claims that an idea is bad based solely on the failure of one attempt, and we should be willing to pursue ideas more than once if we have a strong account for why the previous attempt(s) failed.
AI safety needs people everywhere but quickly stated, current talent bottlenecks to me look like:
-- Founders
-- Grantmakers
-- (technical) Research leads
-- Policy entrepreneurs and implementors (which includes a lot of technical work)
-- bets in international coordination and/or cooperation
-- All manner of supporting talent -- program leads, ops proper, public outreach, content creators, comms
Most sought-after qualities for talent are:
-- context, mission alignment, domain understanding, sophisticated views on AI strategy and threat modelling etc.
-- "good judgement", "sound epistemics", "reasoning transparency" and other similar ideas/meta-skills from the EA/rationalist cannon
-- a willingness to get shit done/bias for action (rather than be in learning mode, or people who need a lot of management and oversight, or folks with too many preferences/constraints)
-- low ego, similar to above
-- ambitious folks, since they would be really trying to be their own managers, take on bigger projects, grow themselves and their teams etc.
Finally, even having these, it's not enough to just claim to have these; job-seekers mainly trip up in being able to demonstrate and be legible about having them.
The "Desperate" AI Safety Talent Bottleneck is deeply misleading. Please.stop.
I worked at Google.
Google does not tell applicants that there is a talent bottleneck and they are desperately hiring. They say "we're cool, join us!"
I didn't get into Harvard.
Harvard does not tell applicants that they are desperately seeking students. I am not misled.
I've gotten rejected from countless AI Safety organizations.
AI Safety tells applicants - we are in desperate need of talent (operators/generalists)! Please join us!
I've coached 70+ aspiring career pivoters on navigating the ecosystem. A common theme is the dejection & disappointment from all this rejection. Because of this false marketing, I am the one picking up the pieces to calibrate professionals on how difficult, competitive, and picky organizations are, how to build context, and how to endure the marathon, which is like any other job hunt.
Please.stop.
Some alternatives:
Relevant posts:
This is a linkpost for https://heresjonny.substack.com/p/paul-christianos-barista.
When I left my previous job in April of last year, I took a long break from thinking about AI safety. This was nice. Thinking about catastrophic risks all the time wears you down. Imagining oneself as Cassandra, baffled that nobody around you is taking this sufficiently seriously, is not good for the soul[1]. I was angry at the community that had got me interested in AI risk in the first place. I felt powerless to help, and that I had earned more selfishness anyway.
Of course, avoiding AI altogether wasn’t an option. I was worried that coding would soon be automated, and my most marketable skill would disappear. I wanted to broaden my skillset and position myself well in case I couldn’t go back to a regular software engineering job. I started writing posts about how I was going to found a company, after having been heavily influenced by reading The Intelligence Curse[2]. I got really into thinking about the intersection of AI and psychiatry. I started talking about the importance of defensive acceleration and how effective altruists systematically underrate comparative advantage. This was motivated reasoning; I wanted to work on something other than AI safety and not feel guilty about it[3]. I figured that all these super smart, status-y types had it under control already.
The EMH is trivially false across all domains
In April of this year, I had various conversations that led to me feeling excited about being part of the EA community again. I went to EAG London and primarily talked about things other than AI safety[4]. I started doing some contract work for an EA-flavoured global health and development charity. I was managing to feel like I was part of the community without getting sucked into AI safety’s gravity well.
I ended up applying for a job at an EA organisation. I managed to get through to the work trial stage, which involved giving people advice. It was noteworthy that diverse groups of people often seemed to need to hear relatively similar advice. I kept repeating something along the lines of:
And then of course something deeply upsetting happened. I realised that I was not following my own advice.
So I went back to basics. Claude and I went back-and-forth over my litany of questions and objections to the idea that advanced AI systems could pose an existential threat. I wrote a post where I attempted to reason through what I thought was likely to happen over the next two years. It started to seem likely that, at the very least, things are going to get weird, soon.
Somewhere during the process it dawned on me that vanishingly few people do this kind of thing; most were LARPing, just like me. When considering something complex, at some point, it’s all too easy to let rigour lapse and hand-wave something away. Thinking rigorously is hard, but in a particularly subtle way that makes this kind of hand-waving easily done without being consciously noticeable.
I felt a sense of clarity. It might be that this clarity is entirely fake and I’m totally wrong, but still, it persists. I was ready to say it with roon:
I’ve long felt like I’m trapped in a midwit meme, painfully aware of the importance of AI safety but simply lacking the intelligence or skills to actually do anything about it. When I read Scott Alexander’s Parable of the Talents, I imagined myself as Paul Christiano’s personal barista. But I don’t think any of it is true. I’ve sufficient genius. What I was lacking was the courage to actually do the hard work of thinking seriously about things for long enough to reach a conclusion, even when that conclusion is an unpleasant one.
Having sat down and made a good-faith effort to think it through, I do not think we are on track to develop AGI in a safe way. I think I have enough power and resources that there’s some chance I could make a difference. And so I’m going to try.
Although Michael Burry seems to be making it work for him
I feel weird about this. In some sense, it is true; I am the sole director of a company. The word “found” has various implications that aren’t accurate in my case, but sometimes it is useful to pretend that they are. Sometimes “founding” is quite high-status, and so there’s a temptation to play it up. I think that’s pretty clearly what was happening in this post. The whole thing seems naïve too – although I think that finding success with a startup probably does require some amount of naïveté.
Despite being motivated, I do think there’s some legitimacy to the claim regardless.
It was my sixth EAG, and the other five had all been dominated by the topic.