Quick takes
Significant fractions of Magnifica Humanitas, the papal encylical on safeguarding the human person in the time of artificial intelligence, is written significantly by AI, most likely Claude. I currently believe Pope Leo himself was not personally responsible (encyclicals tend to be group projects), however the AI usage is likely substantial enough that it's not the result of minor brushups or AI translation:
https://linch.substack.com/p/claude-author-of-the-humanitas
Key claims:
- Significant fractions of the recent papal encyclical are written by AI. I provide
Today’s the day!!! 80,000 Hours (the book) launches in the US.
This is possibly one of the biggest opportunities for EA-per-se growth that we’ve seen in a while.
I’m feeling excited, but also very nervous! We’ve done our best but I don’t know how it’ll be received. I hope people like it 🙂
Shares of our launch post (especially quote tweets) and reviews on Amazon or Goodreads are really really appreciated. Fingers crossed…!!!
The Frontier in 2025 (data), by Gavin Leech, Lauren Gilbert, and Ulkar Aghayeva, rated 202 of the biggest breakthroughs of last year. Some favorites, mainly public health- and society-related:
- Diagnostics on a phone with no doctor needed (source)
- Murder rates worldwide have fallen 25% since 2000 (source) ("On average! Potentially some confounding from improved trauma emergency care converting murders into attempted murders")
- 5 factors explain most of the genetic variance in common mental illnesses (source)
- Large effect for 5-MeO-DMT for treatment-resistant dep
AI systems that match or exceed human intelligence could very plausibly arrive within the next decade, and raise some significant challenges as they do. Alongside the well-known issues surrounding AI safety, there are many other potential problems that we don’t yet seem prepared for, but that could affect society on a huge scale. Forethought has published lots of research on this recently, and we wanted to cover some of the key points for people who might be interested in exploring some of these challenges with their career.
A few of the issues th...
HEY I just realized I've never posted publicly about something I think is pretty important, that I try to tell all my loved ones.
Unfortunately I think tattoos might give you cancer. I don't think we know for certain whether it's causal, but I think there's a pretty good chance it is.
Big study on it:
https://pubmed.ncbi.nlm.nih.gov/38827888/
I'm extremely grateful that my surgeon alerted me to this study before I got a second tattoo. I'm pretty bummed about not being able to get the huge tower crane chest piece / left sleeve that I wanted, but I'm even more h...
You should volunteer at your first EAG! (Especially if you are a student or early career)
- If you don’t have a network in EA, EAG’s can be overwhelming. Volunteering gives you a ready-made, organic network.
- Volunteering is pretty chill - a lot of the shifts aren’t that hard.
- At your first EAG, it’s unlikely that you are using your time so efficiently that a few hours of volunteering would cut into the value of your conference.
I liked GiveDirectly's recent update via GWWC's email newsletter:
...
Donations from Giving What We Can community members were delivered to Masauli, Chirtera, and Mtembo villages in Chiradzulu district in Malawi. Together, we funded transfers for 954 Malawians in poverty across all three villages.
How did families spend their transfers? Here’s what follow-up surveys show:
Hear directly from Emily and David, who are just a few of the people in Masauli village who received transfers from you and other GWWC members:
Emily and her husband, Evance
“My husband and I
My three most recent posts on Substack are relevant to effective altruism:
- Shouldn’t we spend money on AGI safety, just in case?
- The sad decline of effective altruism
- The pseudo-religious origins of the AI bubble
I can’t discuss them on the EA Forum, but I’m happy to do so on Substack.
I empathize with your experience on the EA Forum. The top comment on this post is a clear example where someone just fundamentally misunderstood the point you were trying to make, and responded in an unhelpful and kinda snarky way. Then when you clarified your point to them, they just repeated their original point again. Really frustrating. Sometimes it seems like people don’t have the patience to deeply engage, yet they do have the patience to comment (often rudely). Which is not a good headspace for discussion.
It could potentially be nice to have alterna...
David Manheim's If AI is normal technology, history is not reassuring is a good read (emphasis mine):
...There’s a truism that technology is good - even if it creates winners and losers, it improves the world. Toby Ord argues that the conclusions about the benefits of technology is sensitive to the end of humanity - but this jumps over the transitions by starting from the assumption[1] that “long-term progress in science, technology, and values have tended to make people’s lives longer, freer, and more prosperous.” That is, looking back historically
Broadly in agreement but I don't think the examples are actually the pure version of what he is saying. It sounds like he is classifying goods based on the ratio of private benefit/social effect ish, so yes things with a very high private benefit and plausibly low externalities are definitely good but I feel less confident than him we could say that about refrigerators for instance.
It seems pretty bad to allow people to react to their own comments/posts beyond the initial karma bump of an upvote. It allows users to artificially create the appearance of positive engagement with their posts, which skews initial perceptions and detracts from the truth-seeking goals of the Forum.
Maybe my biggest medium-term worry about transformative AI, other than the takeover stuff, is a constellation of concerns I sometimes abbreviate to "political economy." Right now a large fraction of humans in democracies can live and support their families as a direct result of voluntarily exchanging their labor. It'd take active acts of violence to break from this (pretty good, all things considered) status quo. As a peacetime norm, this is unusually good relative to the history of human civilization.
At some point in the future (in the "good" future...
If people have a higher quality of life overall, and specifically if they have access to more goods and services than they did before while literally not working at all, that actually seems extremely good.
Democracy is just one governance technique among many in our existing liberal-democratic countries, and isn't essential to good governance.
Over the past 12 years, I almost always avoided applying for any jobs in effective altruism – though they did often seem like dream jobs – because:
- I was afraid I might not be the best candidate, and if, by chance, I replace the best candidate, my work would not only be a waste but outright harmful. I'm the sort of person who's afraid to drive a car for fear of hurting someone, and the funding allocation can affect the lives of billions or trillions of beings, so any mistakes I could make would vastly outstrip any harm I could do with a car if I tried.
- That
Some notes on OpenAI disproving the Erdős unit distance conjecture (from a non-mathematician):
- First, this is big. A notorious math conjecture being disproved by AI would be sci-fi 10 years ago. In my layman's read, this is plausibly the most prominent math result in the last 12 months -- AI, centaur, human or whatever.
- Second, it rebutted an Erdős conjecture, and I found it curious that the first clear math breakthrough goes against consensus. There are a few potential reads to this: a) this seems to go against the LLMs-are-sycophantic-machine claims; b) ev
To add color to your 5th bullet, Terry Tao's AI contributions to Erdős problems GitHub page gives you a sense of the accelerating level of AI math proving activity, including partial and incorrect proof claims as well as correct ones. Problem #90 is definitely the most notable so far.
Re: your guess
My best sense right now is that they probably ran a search across various problems, had internal employees [that include very bright mathematicians] to verify what was most promising, then got Gowers/other prominent mathematicians involved to double-check.
I...
I now think principles-first EA is more important than I previously thought because it helps prevent effectiveness drift. My anecdotal evidence from AI Safety and especially biosecurity gives me the impression that without constant anchoring to EA and especially comparisons to the clear ToCs and tractability for e.g. AMF, it is easy to lose focus on the high demands of choosing x-risk as a cause area over others. I previously placed less value on having strong links between EA and the various cause areas but now think I should update to thinking strong, co...
I sometimes worry that I'm still super persuadable/impressionable when people with e.g. upper-class British accents or who speak very well or who're affectively agreeable say things vs e.g. American Southerners or more abrasive debaters but... I found this BBC discussion between Lady Hale and Archbishop Williams to be such a heartwarming/encouraging example of how people can disagree with each other on a very contentious/important topic in the most gentle/humble/polite way.
The Current Landscape: The current State-of-the-Art (SOTA) in function-based screening relies heavily on sophisticated machine learning models, such as Transformers and Sparse Autoencoders (SAEs), trained on massive genomic databases. These tools excel at analyzing sequence text to identify familiar structural homologies or dangerous functional motifs. By screening digital intent at the order stage, these models provide a highly effective defense against known biological threats and their immediate variants.
The Frontier Challenge: However, as synthesis cap...
Here's an idea on how funders in AI safety and governance could help applicants improve their applications and projects: Share statistics! (What follows is a text I'm also sending directly to a funder.)
I understand you aren't really able to give individualized feedback. Though, as applicants, it would be really helpful to have some more clarity. I think you'd like to give more feedback, if you were able. In thinking about this, here's an idea I had.
You could create a score chart, where for every application you keep numerical or binary scores on the reaso...