Effective Altruism Forum
EA Forum

Hide table of contents
Comment Permalink
tcelferact1
0
0

Hi Yonatan, I actually got some 1:1 career advice from 80k recently, they were great! I'm also friends with someone in AI who's local to Montréal and who's trying to help me out. He works at MILA which has ties to a few universities in the city (that's kind of what inspired the speculative master's application). Thanks in advance for the referrals! 

Reply
See in context
by tcelferact
2 min read4 answers 0

30

Career choiceAI safetyPhilosophyAI alignment
Frontpage

Hi forum,

I'm a philosophy graduate from Oxford who's been working as a programmer for five years. I'm missing some of the required mathematical background right now, but I think I could be a good fit for AI safety research. I'm trying to figure out the next steps I should take.

Done:

  • Andrew Ng's machine learning course (https://www.coursera.org/learn/machine-learning).
  • Decent understanding of single variable derivations.
  • Some basic statistics (probability distributions, Bayesian inference, confidence intervals, linear/logistic regression).
  • Python (not an expert but have used it full time for ~1 yr).
  • Applied to MLAB bootcamp, rejected (I'm a bootcamp graduate so the timed-algorithm-based application process was always a long shot).
  • Applied to a local computer science masters, rejected due to 'insufficient training' (trying to follow up and get more data).
  • 80K 1:1 career advising. It was a really helpful experience and they suggested I take a closer look at this path.
  • Some local networking. I'm in Montréal and have a friend who works at MILA, he has been pointing me towards a few events and putting me in touch with some folks in the ML space.
  • Reading LessWrong, trying to have some of my own ideas on alignment, however simple/flawed. I've had the 'narrow melt-all-GPUs AI' thought summarized here: https://astralcodexten.substack.com/p/practically-a-book-review-yudkowsky. I've thought a bit about how strongly typed systems might help, ekmett's livestreams from when he was working with MIRI are on my to-view list: https://www.twitch.tv/ekmett.

Todo:

  • Multivariable calculus, chain rule, backpropagation.
  • More statistics, e.g. enough to understand all of this ARC paper.
  • Aforementioned ekmett livestreams.
  • [...]?

I'm interested in this because my impression from struggling with papers in the space (like the ARC one above) is that the work to be done requires mathematical and statistical foundations but actually leans more towards something like philosophy in terms of methodology. I think I can pick the mathematics and statistics up, even if I do it quite a bit more slowly than a lot of the readers on here. And I enjoy the methodology of/could be pretty good at philosophy (1st class degree from Oxford).

I am looking for advice on what I should do next (and why). Some hypothetical candidate actions: "quit your job, read and implement papers x, y, z, apply to these openings" (a Daniel Ziegler style opportunity https://80000hours.org/podcast/episodes/olsson-and-ziegler-ml-engineering-and-safety/), "take these online courses", "apply to this course that I suspect will consider applicants with your background", "try and get a job in problem space x". "I actually think you have misconceptions about the space and wouldn't be a great fit" would be disappointing but also valuable feedback if applicable.

I'm in my late 20s and can relocate, retrain, etc. If your suggestion is extreme, please suggest it anyway and I can decide if I'm willing/able. Thanks all!

30

0
0

Reactions

0
0
New Answer
New Comment


4 Answers sorted by
Top

Yonatan Cale

12
0
0

Hey!

  1. 80k wrote about similar things: https://forum.effectivealtruism.org/posts/gbPthwLw3NovHAJdp/new-80-000-hours-career-review-software-engineering#Working_in_AI_safety
  2. I encourage you to talk to them since it's something they specialize in
  3. I hope someone doing AI safety will reply here.
    1. My prior from people trying to enter domain that I know something about - is that they could save a ton of time if they'd get insider advice. But I don't have the answers for AI Safety specifically
  4. I'll refer some friends to this post

+1 for asking publicly!

frances_lorenz9
0
0

This is great advice :) Already mentioned below; however, for people in similar positions, please do consider booking a coaching call with AI Safety Support: https://www.aisafetysupport.org/. We have experience helping people navigate the AI Safety field and can also connect you to others.

Reply
4
Yonatan Cale
Ah wow! Would you recommend contacting you over reading this? https://forum.effectivealtruism.org/posts/pbiGHk6AjRxdBPoD8/ai-safety-starter-pack (Please tell me who you would or wouldn't like me to send your way, I ongoingly talk to lots of software developers, and sometimes they want to do AI safety)
frances_lorenz11
0
0

Yeah, absolutely! Happy to go through posts offering career advice, how one might implement the advice, if there are any other perspectives to consider, etc.

I would really encourage having a low-bar for sending people our way, very happy to talk to anyone! But generally, we offer coaching to those trying to get into the AI Safety field (ex. undergrads looking for research positions, software engineers or research scientists looking for work in the field, independent researchers or community-builders interested in applying for funding). Also happy to talk people through AI Safety career-related decisions (ex. whether or not to go to graduate school, choosing between positions, etc.)

Reply
tcelferact1
0
0

I'm going to add some of this to my 'done' column, thanks for pointing it out.

Reply
tcelferact1
0
0

Hi Yonatan, I actually got some 1:1 career advice from 80k recently, they were great! I'm also friends with someone in AI who's local to Montréal and who's trying to help me out. He works at MILA which has ties to a few universities in the city (that's kind of what inspired the speculative master's application). Thanks in advance for the referrals! 

Reply

Bogdan Ionut Cirstea

7
0
0

Consider applying for https://www.eacambridge.org/agi-safety-fundamentals

tcelferact2
0
0

Thanks, I'm now on their mailing list!

Reply

mic

5
0
0

Richard Ngo recently wrote a post on careers in AI safety.

I think you could divide AI safety careers into six categories. I've written some quick tentative thoughts on how you could get started, but I'm not an expert in this for sure.

  • Software engineering: infrastructure, building environments, etc.
    • Do LeetCode/NeetCode and other interview prep and get referrals to try to get a really good entry-level software engineering job. Work in software engineering for a few years, try to get really good at engineering (e.g., being able to dive into a large, unfamiliar codebase and submit a significant pull request within a few weeks). Maybe learn in-demand skills like parallel computing, data engineering, information security, etc. Then, try to get into a software engineering role at Anthropic, Redwood Research, etc. Anthropic is generally looking for fairly experienced engineers, as they aren't able to provide enough mentorship at this stage for new engineers.
  • ML implementation: converting a research idea into a working model.
    • Take an ML course (you can apply for a grant from the Long-Term Future Fund if necessary), especially in deep natural language processing or reinforcement learning, reproduce some ML papers, maybe do a master's in ML if you want, apply for ML jobs at Redwood or Anthropic.
  • ML research direction: coming up with good ideas, designing experiments.
    • Maybe do a PhD in machine learning, apply to CHAI or DeepMind or OpenAI? But I've heard that a PhD takes way too long and many AI safety orgs aren't that credentialist. I have no idea what I'm talking about here.
  • Theory research: building good abstractions, mathematical reasoning.
    • Go through the AGI Safety Fundamentals technical alignment program or dive deep into alignment research that seems interesting to you. Think about the Eliciting Latent Knowledge problem and Richard Ngo's Alignment research exercises, and maybe apply for a grant from the Long-Term Future Fund to do independent research.
  • AI policy
    • I'm not that familiar with this, but I think you could start with the AGI Safety Fundamentals governance program
  • Non-technical roles in AI safety orgs such as Redwood Research. I'm also personally excited about AI safety field-building at top universities, something like EA movement-building at universities, based on the experience of EA at Georgia Tech, OxAI Safety Hub, EA NYU, and AI Safety @ MIT this semester.

Again, check out Richard Ngo's post on careers in AI safety, and apply for relevant internships/residencies. AI jobs that aren't related to safety can still be helpful for gaining experience so you can transition to safety work.

Jay Bailey

5
0
0

AI Safety Support (https://www.aisafetysupport.org/) would be an excellent place to reach out to in my opinion. (I am not in AI Safety myself, but recently spoke to them about getting into it and they offered some excellent advice) They offer help for people looking to get started in AI Safety, and can offer that insider perspective that Yonatan refers to in Point 3 of his answer.

tcelferact2
0
0

Done, thank you!

Reply
More from tcelferact
93
Selective truth-telling: concerns about EA leadership communication.
tcelferact
· · 6m read
45
45
View more
Curated and popular this week
 ·  · 5m read
 · 
[Cross-posted from my Substack here] If you spend time with people trying to change the world, you’ll come to an interesting conundrum: Various advocacy groups reference previous successful social movements as to why their chosen strategy is the most important one. Yet, these groups often follow wildly different strategies from each other to achieve social change. So, which one of them is right? The answer is all of them and none of them. This is because many people use research and historical movements to justify their pre-existing beliefs about how social change happens. Simply, you can find a case study to fit most plausible theories of how social change happens. For example, the groups might say: * Repeated nonviolent disruption is the key to social change, citing the Freedom Riders from the civil rights Movement or Act Up! from the gay rights movement. * Technological progress is what drives improvements in the human condition if you consider the development of the contraceptive pill funded by Katharine McCormick. * Organising and base-building is how change happens, as inspired by Ella Baker, the NAACP or Cesar Chavez from the United Workers Movement. * Insider advocacy is the real secret of social movements – look no further than how influential the Leadership Conference on Civil Rights was in passing the Civil Rights Acts of 1960 & 1964. * Democratic participation is the backbone of social change – just look at how Ireland lifted a ban on abortion via a Citizen’s Assembly. * And so on… To paint this picture, we can see this in action below: Source: Just Stop Oil which focuses on…civil resistance and disruption Source: The Civic Power Fund which focuses on… local organising What do we take away from all this? In my mind, a few key things: 1. Many different approaches have worked in changing the world so we should be humble and not assume we are doing The Most Important Thing 2. The case studies we focus on are likely confirmation bias, where
 ·  · 2m read
 · 
I speak to many entrepreneurial people trying to do a large amount of good by starting a nonprofit organisation. I think this is often an error for four main reasons. 1. Scalability 2. Capital counterfactuals 3. Standards 4. Learning potential 5. Earning to give potential These arguments are most applicable to starting high-growth organisations, such as startups.[1] Scalability There is a lot of capital available for startups, and established mechanisms exist to continue raising funds if the ROI appears high. It seems extremely difficult to operate a nonprofit with a budget of more than $30M per year (e.g., with approximately 150 people), but this is not particularly unusual for for-profit organisations. Capital Counterfactuals I generally believe that value-aligned funders are spending their money reasonably well, while for-profit investors are spending theirs extremely poorly (on altruistic grounds). If you can redirect that funding towards high-altruism value work, you could potentially create a much larger delta between your use of funding and the counterfactual of someone else receiving those funds. You also won’t be reliant on constantly convincing donors to give you money, once you’re generating revenue. Standards Nonprofits have significantly weaker feedback mechanisms compared to for-profits. They are often difficult to evaluate and lack a natural kill function. Few people are going to complain that you provided bad service when it didn’t cost them anything. Most nonprofits are not very ambitious, despite having large moral ambitions. It’s challenging to find talented people willing to accept a substantial pay cut to work with you. For-profits are considerably more likely to create something that people actually want. Learning Potential Most people should be trying to put themselves in a better position to do useful work later on. People often report learning a great deal from working at high-growth companies, building interesting connection
 ·  · 31m read
 · 
James Özden and Sam Glover at Social Change Lab wrote a literature review on protest outcomes[1] as part of a broader investigation[2] on protest effectiveness. The report covers multiple lines of evidence and addresses many relevant questions, but does not say much about the methodological quality of the research. So that's what I'm going to do today. I reviewed the evidence on protest outcomes, focusing only on the highest-quality research, to answer two questions: 1. Do protests work? 2. Are Social Change Lab's conclusions consistent with the highest-quality evidence? Here's what I found: Do protests work? Highly likely (credence: 90%) in certain contexts, although it's unclear how well the results generalize. [More] Are Social Change Lab's conclusions consistent with the highest-quality evidence? Yes—the report's core claims are well-supported, although it overstates the strength of some of the evidence. [More] Cross-posted from my website. Introduction This article serves two purposes: First, it analyzes the evidence on protest outcomes. Second, it critically reviews the Social Change Lab literature review. Social Change Lab is not the only group that has reviewed protest effectiveness. I was able to find four literature reviews: 1. Animal Charity Evaluators (2018), Protest Intervention Report. 2. Orazani et al. (2021), Social movement strategy (nonviolent vs. violent) and the garnering of third-party support: A meta-analysis. 3. Social Change Lab – Ozden & Glover (2022), Literature Review: Protest Outcomes. 4. Shuman et al. (2024), When Are Social Protests Effective? The Animal Charity Evaluators review did not include many studies, and did not cite any natural experiments (only one had been published as of 2018). Orazani et al. (2021)[3] is a nice meta-analysis—it finds that when you show people news articles about nonviolent protests, they are more likely to express support for the protesters' cause. But what people say in a lab setting mig
Recent opportunities in AI safety
24
Last week to apply for the Futurekind AI Fellowship! (deadline: April 1)
Jay Luong
· · 1m read
0
0
121
AI Moral Alignment: The Most Important Goal of Our Generation
Ronen Bar
· · 10m read
31
31
54
Stripe Economics of AI Fellowship
basil.halperin
· · 3m read
0
0
View more