Why did I create both a post and a wiki article?
Posts are best for making sure people see the initial version of the post, whilst Wiki articles are best for long-term maintenance. Posting an article that links to a Wiki page provides the best of both worlds.
For the most up-to-date version, see the Wiki page.
Articles:
There should be more AI safety organisations
Why does the AI Safety Community need help founding projects?
AI Assurance Tech Report
AI Safety as a YC Startup
Alignment can be the ‘clean energy’ of AI
AI Tools for Existential Security
Incubation Programs:
Def/acc at Entrepreneur First - this is a new program, which focuses on "defensive" tech, which will produce a wide variety of startups, some of which may help with reducing x-risk, but others may include AI applications like autonomous weapons or diagnostic medical software, which, while they generally won't increase x-risk, may not reduce it either.
Catalyze - AI Safety Incubation
Seldon Accelerator: “We take on AGI risk with optimism”
AE Studio has expressed interest in helping seed neglected approaches, incl. ideas that take the form of business ideas
AIS Friendly General Program:
Founding to Give: General EA incubator
Entrepreneur First - They run def/acc, so this could be a decent alternative
Halycon Futures: “We’re an entrepreneurial nonprofit and VC fund dedicated to making AI safe, secure and good for humanity.” They do new project incubation, grants, investments, ect.
Communities:
Entrepreneurship Channel EA Anyway
AI Safety Founders: Website, Discord, LinkedIn
VC:
AI Safety Seed Funding Network
Lionheart Ventures - “We invest in the wise development of transformative technologies.”
Juniper Ventures - Invests in companies “working to make AI safe, secure and beneficial for humanity”. Published the AI Assurance Tech Report
Polaris Ventures - “We support projects and people aiming to build a future guided by wisdom and compassion for all”
Mythos Ventures - “an early-stage venture capital firm investing in prosocial technologies for the transformative AI era.”
Metaplanet - “We support novel and evidence-based innovation that could produce an outsized return for the benefit of humankind.”
Anthology Fund - Backed by Anthropic. One of the five key areas is “trust and safety tooling”
Menlo Ventures - Lead the GoodFire round (article)
Safe AI Fund
Organisational Support:
Ashgro - fiscal sponsorship
Rethink Priorities Special Projects - provides fiscal sponsorship or incubation
Other:
Hackathon for AI Safety Startups - Ran once by Apart Research, may run again
Constellation Residency - Year-long position
Nonlinear - Free coaching for people who are running an AI safety startup or who are considering starting one
Dustbin - Things that previously existed
Longtermist Entrepreneurship Project - Retrospective
