Hide table of contents

What do I mean by “homegrown”? These projects are:

  • Local: Creators have a good track record in the EA or AI Safety community
  • Modest: The amount requested is not large; $5k would be meaningful
  • Overlooked: Not already backed by large institutional funders like OpenPhil

If you’re a small donor or earn to give, consider giving to projects like these:

1. Feature-length documentary on SB 1047

By Michael Trazzi — $16k raised of $55k

 

Michael has already recorded interviews with the main characters of SB 1047: sponsors like Scott Wiener and Dan Hendrycks, proponents like Zvi Mowshowitz and Holly Elmore, and opponents like Dean Ball and Timothy B Lee. Now he needs the funding to turn it into a 1-hour feature documentary. This is a rare chance to sponsor a high-quality video narrative, and share it beyond our existing ecosystem. I’ve personally donated $10k towards this, and expect Michael to be able to effectively use much more.

More info & donate here: https://manifund.org/projects/finishing-the-sb-1047-documentary-in-6-weeks

2. Overview of AI Safety in 2024

By Gavin Leech — $8k of $17.6k raised

Gavin Leech is a forecaster, researcher and founder of Arb; he’s proposing to re-rerun a 2023 survey of AI Safety. The landscape shifts pretty quickly, so I’d love to see what’s changed since last year.

As I was writing this, regrantor Neel Nanda funded it to the minimum $8k ask! Neel adds:

I think collections like this add significant value to newcomers to the field, mostly by being a list of all areas worth maybe thinking about, and key links (rather than eg by providing a lot of takes on which areas are more or less important, unless the author has excellent taste). Gavin has convinced me that the previous post gets enough traffic for it be valuable to be kept up to date.

More info & donate here: https://manifund.org/projects/shallow-review-of-ai-safety-2024

3. Podcast series on Effective Altruism’s values

By Elizabeth Van Nostrand — $1.3k raised of $2.6k

Elizabeth & Timothy’s initial podcast was very well received, drawing extensive, thoughtful comments from a variety of folks. I’d be excited to see them continue this series, especially if they bring in folks involved with steering the EA community (like Sarah Cheng, who has extensively engaged with their points)

More info & donate here: https://manifund.org/projects/elizabeth-and-timothy-podcast-on-values-in-effective-altruism

4. Sentinel, a foresight and emergency response team

By Nuno Sempere — $16k raised of $90k

Nuno has long been one of our community’s most outspoken forecasters; now he’s working with Rai Sur to spin up an emergency response team (think: Army Reserve Corps, but for responding to existential risks). They’re already putting out a useful weekly report on biosecurity, geopolitics and other such topics.

More info & donate here: https://manifund.org/projects/fund-sentinel-for-q4-2024

5. Research on co-occurence of sparse autoencoder latents

By Matthew A. Clarke — $0 raised of $6.4k

TBH, I don’t know much about the merits for or against this line of research; I’m highlighting this grant because it’s overseen by Joseph Bloom, a past Manifund grantee who I and others have been very impressed with. If mechanistic interpretability is your jam, check this one out!

More info & donate here: https://manifund.org/projects/salaries-for-sae-co-occurrence-project

 

What else have we been up to?

It’s been a quiet couple of months, but here’s what Rachel and I have been busy with:

 

Cheers,

Austin

 

PS: want to make a larger grant? Manifund can facilitate donations via donor advised funds, crypto, and bank transfers.

Comments1


Sorted by Click to highlight new comments since:

I work at Open Philanthropy, and I recently let Gavin know that Open Phil is planning to recommend a grant of $5k to Arb for the second project on your list: Overview of AI Safety in 2024 (they had already raised ~$10k by the time we came across it). Thanks for writing this post Austin — it brought the funding opportunity to our attention.

Like other commenters on Manifund, I believe this kind of overview is a valuable reference for the field, especially for newcomers. 

I wanted to flag that this project would have been eligible for our RFP for work that builds capacity to address risks from transformative AI. I worry that not all potential applicants are aware of the RFP or its scope, so I’ll take this opportunity to mention that this RFP’s scope is quite broad, including funding for: 

  • Training and mentorship programs
  • Events
  • Groups
  • Resources, media, and communications
  • Almost any other type of project that builds capacity for advanced AI risks (in the sense of increasing the number of careers devoted to these problems, supporting people doing this work, and sharing knowledge related to this work). 

More details at the link above. People might also find this page helpful, which lists all currently open application programs at Open Phil. 

Curated and popular this week
 ·  · 12m read
 · 
Economic growth is a unique field, because it is relevant to both the global development side of EA and the AI side of EA. Global development policy can be informed by models that offer helpful diagnostics into the drivers of growth, while growth models can also inform us about how AI progress will affect society. My friend asked me to create a growth theory reading list for an average EA who is interested in applying growth theory to EA concerns. This is my list. (It's shorter and more balanced between AI/GHD than this list) I hope it helps anyone who wants to dig into growth questions themselves. These papers require a fair amount of mathematical maturity. If you don't feel confident about your math, I encourage you to start with Jones 2016 to get a really strong grounding in the facts of growth, with some explanations in words for how growth economists think about fitting them into theories. Basics of growth These two papers cover the foundations of growth theory. They aren't strictly essential for understanding the other papers, but they're helpful and likely where you should start if you have no background in growth. Jones 2016 Sociologically, growth theory is all about finding facts that beg to be explained. For half a century, growth theory was almost singularly oriented around explaining the "Kaldor facts" of growth. These facts organize what theories are entertained, even though they cannot actually validate a theory – after all, a totally incorrect theory could arrive at the right answer by chance. In this way, growth theorists are engaged in detective work; they try to piece together the stories that make sense given the facts, making leaps when they have to. This places the facts of growth squarely in the center of theorizing, and Jones 2016 is the most comprehensive treatment of those facts, with accessible descriptions of how growth models try to represent those facts. You will notice that I recommend more than a few papers by Chad Jones in this
LintzA
 ·  · 15m read
 · 
Introduction Several developments over the past few months should cause you to re-evaluate what you are doing. These include: 1. Updates toward short timelines 2. The Trump presidency 3. The o1 (inference-time compute scaling) paradigm 4. Deepseek 5. Stargate/AI datacenter spending 6. Increased internal deployment 7. Absence of AI x-risk/safety considerations in mainstream AI discourse Taken together, these are enough to render many existing AI governance strategies obsolete (and probably some technical safety strategies too). There's a good chance we're entering crunch time and that should absolutely affect your theory of change and what you plan to work on. In this piece I try to give a quick summary of these developments and think through the broader implications these have for AI safety. At the end of the piece I give some quick initial thoughts on how these developments affect what safety-concerned folks should be prioritizing. These are early days and I expect many of my takes will shift, look forward to discussing in the comments!  Implications of recent developments Updates toward short timelines There’s general agreement that timelines are likely to be far shorter than most expected. Both Sam Altman and Dario Amodei have recently said they expect AGI within the next 3 years. Anecdotally, nearly everyone I know or have heard of who was expecting longer timelines has updated significantly toward short timelines (<5 years). E.g. Ajeya’s median estimate is that 99% of fully-remote jobs will be automatable in roughly 6-8 years, 5+ years earlier than her 2023 estimate. On a quick look, prediction markets seem to have shifted to short timelines (e.g. Metaculus[1] & Manifold appear to have roughly 2030 median timelines to AGI, though haven’t moved dramatically in recent months). We’ve consistently seen performance on benchmarks far exceed what most predicted. Most recently, Epoch was surprised to see OpenAI’s o3 model achieve 25% on its Frontier Math
Omnizoid
 ·  · 5m read
 · 
Edit 1/29: Funding is back, baby!  Crossposted from my blog.   (This could end up being the most important thing I’ve ever written. Please like and restack it—if you have a big blog, please write about it). A mother holds her sick baby to her chest. She knows he doesn’t have long to live. She hears him coughing—those body-wracking coughs—that expel mucus and phlegm, leaving him desperately gasping for air. He is just a few months old. And yet that’s how old he will be when he dies. The aforementioned scene is likely to become increasingly common in the coming years. Fortunately, there is still hope. Trump recently signed an executive order shutting off almost all foreign aid. Most terrifyingly, this included shutting off the PEPFAR program—the single most successful foreign aid program in my lifetime. PEPFAR provides treatment and prevention of HIV and AIDS—it has saved about 25 million people since its implementation in 2001, despite only taking less than 0.1% of the federal budget. Every single day that it is operative, PEPFAR supports: > * More than 222,000 people on treatment in the program collecting ARVs to stay healthy; > * More than 224,000 HIV tests, newly diagnosing 4,374 people with HIV – 10% of whom are pregnant women attending antenatal clinic visits; > * Services for 17,695 orphans and vulnerable children impacted by HIV; > * 7,163 cervical cancer screenings, newly diagnosing 363 women with cervical cancer or pre-cancerous lesions, and treating 324 women with positive cervical cancer results; > * Care and support for 3,618 women experiencing gender-based violence, including 779 women who experienced sexual violence. The most important thing PEPFAR does is provide life-saving anti-retroviral treatments to millions of victims of HIV. More than 20 million people living with HIV globally depend on daily anti-retrovirals, including over half a million children. These children, facing a deadly illness in desperately poor countries, are now going