Effective Altruism Forum
EA Forum

Hide table of contents
Comment Permalink
Vael Gates4
0
0

Suggestion for a project from Jonathan Yan: 

Given the Future Fund's recent announcement of the AI Worldview Prize, I think it would be a good idea if someone could create an online group of participants. Such a group can help participants share resources, study together, co-work, challenge each other's views, etc. After the AI Worldview Prize competition ends, if results are good, a global AI safety community can be built and grown from there.

Reply
See in context
by Vael Gates
7 min read 6

126

AI safetyBuilding effective altruismCareer choiceBuilding the field of AI safetyField-building
Frontpage

Tldr: If you’re interested in working on an AI safety field building project similar to those listed below (e.g. researcher outreach): please fill out this application form or nominate someone ($200 bounty)! Hiring is ongoing indefinitely. If you’re an EA org that has an AI safety field building project, please submit your project idea, and if our priorities and people align sufficiently we’ll try to get it done! 

[Crossposted to LessWrong]

Update from 10/31/22: I'm still working on similar projects and looking for people, and encourage you to apply!

Update from 11/12/22: I'm currently not hiring new people, due to the new funding situation. You're still welcome to fill out the application form, but unfortunately I'm focusing on existing projects at the moment.

Update from 2/1/23: AISFB has ended! See the Retrospective here.

When individual EAs are interested in working on AI safety research, they can apply to many programs designed to give mentorship, funding, and project ideas (SERI MATS, MLAB, AI Safety Camp, SERI / CERI / CHERI summer programs, etc). This is amazing and speaks to our growth as a community. 

However, I think there’s a noticeable lack of such structure in field building. For EAs interested in AI safety field building, how do they know which projects are most promising? Do they have to make them up from scratch? Where do they get collaborators, how do they get access to mentorship, how do they show a good track record to apply for funding? 

To fill this gap, I’m starting the “AI Safety Field Building Hub”, a new effort to provide projects, mentorship, and funding to individuals interested in working in AI safety field building. If you’re an EA looking for a full-time or part-time project, I’m hiring project-specific contractors. I have a list of projects I’m interested in working on, have been sent more shovel-ready projects by others, and hope that other organizations will continue to send me great project ideas they don’t have capacity to manage. In the future, this “hub”-like functionality may spin into its own organization; at the moment, I’m looking to hire people to work for me. 

Of note, my projects tend to be aimed more at outreach and at older populations– AI researchers, academia and industry, rather than university populations or high schoolers— which I think is relatively neglected right now. I also think there are potentially more downside risks for doing outreach to researchers poorly than in reaching out to university students, and the culture is substantially different, so I’m hoping my experience in this space will be helpful to people who are interested in these directions. (About me: I’m currently a postdoc at Stanford whose most recent project was interviewing AI researchers about AI safety.)

I’m aiming to hire agentic, independent, or small-team people who are excited about closely coordinating with me and other EAs. From that point, there are several options:

  • For many of these projects, I’ll want people who are excited about closely coordinating in the beginning of the project and during key decision points, but then go on to truly own the project without further involvement from me. (I can help you with securing your own funding once you’re up and running.) Note that, however, several of the top projects listed here may have significant downside risks, so I’m going to be restrictive with hiring and training for this reason.
  • I’m also interested in hiring people who work part-time on a single project under me indefinitely. This is especially true for many of the top-listed projects which could have significant downside risks.
  • Finally, I’m hoping to hire an advanced executive assistant-type person who will closely work with me across all of my projects (pay will be higher for this role)

I’ll be offering mentorship like you’d find in a research program, and project-specific funding for you to work on the project (and all associated project costs) full-time or part-time. Some of the projects are more operations-oriented, some people-facing, some writing-oriented, some light-technical (such as updating websites): what I consider the usual spread of what field-building jobs involve. 

Initial positions by default last 2 months before reevaluation, and can either be renewed (I continue paying you, mentorship continues, etc.) or if you’re now in charge of the project and there’s more to be done, I can help you secure additional funding. These positions will be remote.

Pay will be around $60k-80k/year ($30-40/hour), depending on level of skill, commitment level, and degree of project ownership. If you’d work on one of these projects but want to be paid substantially more, talk to me and we can likely make it happen. 

The qualifications I’m looking for depend on the project, but across the board I’ll be looking for high generalist competence, AI safety knowledge (for example, I want you to have read / have a plan for reading all of the main readings in the AGISF Technical Curriculum), and strong follow-through on projects that’ll last for months or years. I’m going to be pretty selective on hiring, since I’d like the individual projects to get done well and relatively quickly, and I want to use my own time well between mentoring and object-level work. I’m probably going to want at least 10h/week for 8 weeks, though it does depend on the person. (However, I encourage any interested person to apply; imposter syndrome is a thing in this community and you might be one of the best candidates!)

Projects I’m currently prioritizing (will change over time)

  • 1-on-1 AI Researcher Outreach:
    • I talked to 97 researchers one-on-one about AI safety. There are several reasons to expect that I am uniquely suited to doing this outreach, but I want to test the hypothesis that if non-postdoc-level academics are trained and situated under a good narrative umbrella, more positive expected value interactions can be had with researchers.
    • Additional qualifications: friendly, politic with respect to status hierarchies, excellent knowledge of AI safety space, and academic or industry status markers strongly preferred.
    • This is the “highest-risk” project here, whose success depends a lot on who applies, and I would not be surprised if I quickly decide this is a bad idea. I do think it’s worth trying, however, and I’m also interested in the next pivot I’m interested in trialing, which is seeing if there’s interest in pairing alignment researchers with targeted other researchers for one-on-one talks.
  • Helping with analysis and display of my transcript data
    • This is an ongoing project I’m not finished with yet, that I’d like help with. You’d be closely collaborating with me. Could involve preparing this work for academic publication. One of the subquestions is assembling information on the question: What are common objections and misunderstandings around AGI development being a risk (see my work, and EA version)?
  • Helping with a project encouraging expert engagement and critique of existing AI arguments via sending AI researchers arguments to review
    • This is an ongoing project that I’d like help with. You’d be closely collaborating with me.
  • Several projects developed by the Center for AI Safety, including:
    • Helping make AI Safety Unconferences happen for the major AI conferences
    • Running ML Safety workshops at the major AI conferences
    • Translating various safety-related machine learning papers into Chinese
    • Gaining a bird’s eye view of China’s AI Field
  • Related to the last two points, if you have project ideas for AI Safety in China you’d like to implement, I’m happy to discuss with you since I’m very interested in more work in the area.
  • I’m interested in helping with any internal OpenAI / DeepMind field-building efforts – perhaps logistics-related outsourceable things? I don’t currently know what these are; interested if anyone from those organizations wants to reach out to me.
  • If you have connections that will help make an AI safety-oriented film be made, I’m interested and have thoughts about how this could be done hopefully well. I’m also interested in ideas like providing support to pair graduate students and AI researchers within institutions.
  • Note: my ultimate theory of impact is to get more very talented people in AI alignment research, where it seems to me that this is neglected at the PhD-level and above. For this population, I expect we’ll need higher acceptance of AI alignment in the AI research community overall (changing the overall views of the field of AI to be more interested in the safety arguments, thus increasing norms / incentives / prestige markers / career advancement opportunities towards safety research within the field), since researchers are less willing to change careers without a lot of infrastructure compared to college students. A lot of my intuitive feel for what’s “impactful” draws from these intuitions.
  • (Alternative orgs) Here are some initiatives / organizations which are doing similar work to mine. If you're interested in AI safety field building but not in my specific projects, these might be good places to also apply to.

Action items!

  • If you are an EA organization with AI safety field-building projects that are plausibly transferable to a third party like me, please send project ideas, and I’ll see if we can get them implemented! Note that by default, it’ll probably be better to pass me projects with minimal downside risk and no special need for privacy, but you can specify your needs on the form.
  • If you’re interested in working on one of the above AI safety field building projects with me: please fill out this application form or nominate someone ($200 bounty). Hiring will be ongoing until I make an edit to this post saying otherwise. After submitting a form, I plan to do a Zoom interview and send a work-trial task to people who progress to subsequent stages.

Thanks everyone, and looking forward to it! 

126

0
0

Reactions

0
0
Mentioned in
Load more (5/10)

More posts like this

Comments6


Sorted by
New & upvoted
Click to highlight new comments since:
Vael Gates4
0
0

Suggestion for a project from Jonathan Yan: 

Given the Future Fund's recent announcement of the AI Worldview Prize, I think it would be a good idea if someone could create an online group of participants. Such a group can help participants share resources, study together, co-work, challenge each other's views, etc. After the AI Worldview Prize competition ends, if results are good, a global AI safety community can be built and grown from there.

Reply
Alexander Saeri3
0
0

Great to see this initiative, Vael. I can think of several project ideas that could use this dataset of interviews with researchers.

Edit:

  1. Analyse transcripts to identify behaviours that researchers and you describe as safe or unsafe, and identify influences on those behaviours (this would need validation with follow up work). Outcome would be an initial answer to the concrete question "who needs to do what differently to improve AI safety in research, and how"

  2. Use the actors identified in the interviews to create a system/actor map to help understand flows of influence and information. Outcome: a better understanding of power dynamics of the system and opportunities for influence.

  3. With information about the researchers themselves (especially of there are 90+), could begin to create a typology / segmentation to try and understand which types are more open / closed to discussions of safety, and why. Outcome: a strategy for outreach or further work to change decision making and behaviour towards AI safety.

Reply
Vael Gates1
0
0

I think my data has insights about 3, and not about 1 and 2! You can take a look at https://www.lesswrong.com/posts/LfHWhcfK92qh2nwku/transcripts-of-interviews-with-ai-researchers to see what 11 interviews look like; I think it'd have to be designed differently to get info on 1 or 2. 

Reply
sawyer🔸3
0
0

I'm excited to see this happening and I think you're one of the better people to be launching it. I think there's probably some helpful overlap with BERI's world here, so please reach out if you'd like to talk about anything.

Reply
Vael Gates2
0
0

Sounds great; thanks Sawyer! "Reaching out to BERI" was definitely listed in my planning docs for this post; if there's anything that seems obvious to communicate about, happy to take a call, otherwise I'll reach out if anything seems overlapping.

Reply
Vael Gates1
0
0

Keeping a running list of field-building posts I personally want to keep track of:

Project ideas:
- Akash's: https://forum.effectivealtruism.org/posts/yoP2PN5zdi4EAxdGA/ai-safety-field-building-projects-i-d-like-to-see
- Ryan's: https://www.lesswrong.com/posts/v5z6rDuFPKM5dLpz8/probably-good-projects-for-the-ai-safety-ecosystem

Survey analysis: 
- Ash's: https://forum.effectivealtruism.org/posts/SuvMZgc4M8FziSvur/analysis-of-ai-safety-surveys-for-field-building-insights
- Daniel Filan's: https://www.lesswrong.com/posts/rXSBvSKvKdaNkhLeJ/takeaways-from-a-survey-on-ai-alignment-resources
- https://arxiv.org/abs/2208.12852 (Bowman NLP Survey)
- AI Impacts surveys generally, 2022 and 2016, also GovAI has some
 


 Presentation of ideas: (though this should also be updated in https://www.lesswrong.com/posts/gdyfJE3noRFSs373q/resources-i-send-to-ai-researchers-about-ai-safety )
- Marius's: https://forum.effectivealtruism.org/posts/8JazqnCNrkJtK2Bx4/why-eas-are-skeptical-about-ai-safety
- Lukas's: https://forum.effectivealtruism.org/posts/8JazqnCNrkJtK2Bx4/why-eas-are-skeptical-about-ai-safety
- Mine: https://forum.effectivealtruism.org/posts/q49obZkQujkYmnFWY/vael-gates-risks-from-advanced-ai-june-2022 // https://www.lesswrong.com/posts/LfHWhcfK92qh2nwku/transcripts-of-interviews-with-ai-researchers

Reply
More from Vael Gates
View more
Curated and popular this week
 ·  · 20m read
 · 
Advanced AI could unlock an era of enlightened and competent government action. But without smart, active investment, we’ll squander that opportunity and barrel blindly into danger. Executive summary See also a summary on Twitter / X. The US federal government is falling behind the private sector on AI adoption. As AI improves, a growing gap would leave the government unable to effectively respond to AI-driven existential challenges and threaten the legitimacy of its democratic institutions. A dual imperative → Government adoption of AI can’t wait. Making steady progress is critical to: * Boost the government’s capacity to effectively respond to AI-driven existential challenges * Help democratic oversight keep up with the technological power of other groups * Defuse the risk of rushed AI adoption in a crisis → But hasty AI adoption could backfire. Without care, integration of AI could: * Be exploited, subverting independent government action * Lead to unsafe deployment of AI systems * Accelerate arms races or compress safety research timelines Summary of the recommendations 1. Work with the US federal government to help it effectively adopt AI Simplistic “pro-security” or “pro-speed” attitudes miss the point. Both are important — and many interventions would help with both. We should: * Invest in win-win measures that both facilitate adoption and reduce the risks involved, e.g.: * Build technical expertise within government (invest in AI and technical talent, ensure NIST is well resourced) * Streamline procurement processes for AI products and related tech (like cloud services) * Modernize the government’s digital infrastructure and data management practices * Prioritize high-leverage interventions that have strong adoption-boosting benefits with minor security costs or vice versa, e.g.: * On the security side: investing in cyber security, pre-deployment testing of AI in high-stakes areas, and advancing research on mitigating the ris
 ·  · 15m read
 · 
In our recent strategy retreat, the GWWC Leadership Team recognised that by spreading our limited resources across too many projects, we are unable to deliver the level of excellence and impact that our mission demands. True to our value of being mission accountable, we've therefore made the difficult but necessary decision to discontinue a total of 10 initiatives. By focusing our energy on fewer, more strategically aligned initiatives, we think we’ll be more likely to ultimately achieve our Big Hairy Audacious Goal of 1 million pledgers donating $3B USD to high-impact charities annually. (See our 2025 strategy.) We’d like to be transparent about the choices we made, both to hold ourselves accountable and so other organisations can take the gaps we leave into account when planning their work. As such, this post aims to: * Inform the broader EA community about changes to projects & highlight opportunities to carry these projects forward * Provide timelines for project transitions * Explain our rationale for discontinuing certain initiatives What’s changing  We've identified 10 initiatives[1] to wind down or transition. These are: * GWWC Canada * Effective Altruism Australia funding partnership * GWWC Groups * Giving Games * Charity Elections * Effective Giving Meta evaluation and grantmaking * The Donor Lottery * Translations * Hosted Funds * New licensing of the GWWC brand  Each of these is detailed in the sections below, with timelines and transition plans where applicable. How this is relevant to you  We still believe in the impact potential of many of these projects. Our decision doesn’t necessarily reflect their lack of value, but rather our need to focus at this juncture of GWWC's development.  Thus, we are actively looking for organisations and individuals interested in taking on some of these projects. If that’s you, please do reach out: see each project's section for specific contact details. Thank you for your continued support as we
 ·  · 3m read
 · 
We are excited to share a summary of our 2025 strategy, which builds on our work in 2024 and provides a vision through 2027 and beyond! Background Giving What We Can (GWWC) is working towards a world without preventable suffering or existential risk, where everyone is able to flourish. We do this by making giving effectively and significantly a cultural norm. Focus on pledges Based on our last impact evaluation[1], we have made our pledges –  and in particular the 🔸10% Pledge – the core focus of GWWC’s work.[2] We know the 🔸10% Pledge is a powerful institution, as we’ve seen almost 10,000 people take it and give nearly $50M USD to high-impact charities annually. We believe it could become a norm among at least the richest 1% — and likely a much wider segment of the population — which would cumulatively direct an enormous quantity of financial resources towards tackling the world’s most pressing problems.  We initiated this focus on pledges in early 2024, and are doubling down on it in 2025. In line with this, we are retiring various other initiatives we were previously running and which are not consistent with our new strategy. Introducing our BHAG We are setting ourselves a long-term Big Hairy Audacious Goal (BHAG) of 1 million pledgers donating $3B USD to high-impact charities annually, which we will start working towards in 2025. 1 million pledgers donating $3B USD to high-impact charities annually would be roughly equivalent to ~100x GWWC’s current scale, and could be achieved by 1% of the world’s richest 1% pledging and giving effectively. Achieving this would imply the equivalent of nearly 1 million lives being saved[3] every year. See the BHAG FAQ for more info. Working towards our BHAG Over the coming years, we expect to test various growth pathways and interventions that could get us to our BHAG, including digital marketing, partnerships with aligned organisations, community advocacy, media/PR, and direct outreach to potential pledgers. We thin
Recent opportunities in AI safety
49
AIS Netherlands is looking for a Founding Executive Director (EOI form)
7 authors
· · 5m read
4
4
24
Last week to apply for the Futurekind AI Fellowship! (deadline: April 1)
Jay Luong
· · 1m read
0
0
3
Apply to become a Futurekind AI Facilitator or Mentor (deadline: April 10)
Jay Luong
· · 2m read
0
0
View more