Quick takes

At my particular place, I think that improving AI safety community epistemics (writing thorough and careful arguments against people both in the community and outside of it who make silly mistakes) is, by most reasonable estimates, probably less impactful than helping to run MAIA, which is what I currently spend most of my time doing. I don’t think I’m the only person in this situation, which is quite scary to me, as I think there are a lot of things are epistemically iffy in the AI safety community. For example: * A lot of the most viral pro-AIS people (especially the ones that are relatively new to the field) seem to not be reasoning especially carefully * The field’s hiring process is quite insular * Not enough time is spent engaging with a lot of good faith disagreements (imo) * Fieldbuilding strategy doesn’t optimize for careful reasoning amongst those it recruits * People receive lots of social/material benefits for continuing to believe that AI risk is the most important thing and work on it * Even under some of the shortest timelines and most speculative capabilities that superintelligence might have, I really don’t extinction within the next year should be assigned any notable probability mass, due to the need for data centers to be replenished I also see a lot of disagreements that I think are poorly reasoned. For example:  * Poor application of/thinking about EV when conceptualizing risks * Psychoanalysis that doesn’t explain the safety community’s behaviors well * Comments significantly overstating the case for something * Arguments without the slightest amount of research into what the other side actually believes Tbh though, most of these critiques apply to both sides. I think this primarily arises because the people working in AI safety who are the most careful reasoners also have the highest opportunity costs, because they tend to be good at other things. This means it’s less valuable for them to write out their thoughts online carefu

As a new person here, i am curious to learn more. I think that Longtermism strongly implies growth-oriented policies, and that might even be the existential risk-minimizing strategy. I did an entire substack post on my thoughts, and would be interested to hear where you might think i am wrong (I probably am, you may have thought a lot more on it than I have, or just be smarter). Also what is the general longtermist take on AI-regulation, and the larger statement made here - We Must Act Now. It seems to me that carte blanche calls to action are not really helpful, but would be open to changing my mind. :-) https://open.substack.com/pub/ulrikahm/p/longtermism-and-the-moral-necessity?r=4cnrou&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

In Bruce Friedrich's new book, he writes, "Sometimes when I talk about cultivated meat someone will bring up the handful of states that have banned it. I'm mostly unconcerned. Cultivated meat companies won't be able to supply all 50 US states anytime soon anyway. Once there are multiple companies selling their products in...the majority of cities all across the country, the states that banned will-- I predict-- quietly repeal their laws" (p. 191). It's hard to know how literally to interpret this apparently sanguine attitude, as the book is designed to generate enthusiasm for alternative proteins. But, still it seems raise an important question about the cost-effectiveness of repealing existing bans or preventing further ones. Initial thoughts:   * My guess is that he'd still view preventing additional bans as important, at least in key regions expected to be first-adopters. Maybe Florida and Texas would be laggards in adoption even if cultivated meat were legal. * It'd be interesting to do an outside view analysis to see how quickly bans on other novel products have been undone once they've achieved a certain level of popularity elsewhere. * He's writing as if the industry can definitely succeed in spite of the bans. But, even if the bans spread no further, they already apply to >140M potential consumers across the US and Europe. That, combined with uncertainty about the prospect of additional bans, may chill the sort of public and private investment necessary for industry success.  

A bird's eye view on why donating to relieve the earthquake's damage to Venezuela is one of the best causes to donate to. I have seen the photos, tens of complete buildings shattered to pieces, more than 150+ reported dead, and a lot more buried without clue to whether they're dead or alive, and on top of that, a poor, government in crisis country has to handle that. When a natural catastrophe happens to a country like that, your money goes a long way in saving lives. 

Bettering social media platforms could be a top-tier systemic intervention. My understanding is that EA puts most systemic change / solutions S-rated in terms of impact but F-rated in terms of neglectedness and tractability. Is this the case for social media? Is it super hard and attempted to make the platforms, either internally or through policy, to not maximize for engagement? What about popularizing (and building if necessary) platforms that reward for what would be "useful" for people (or society at large)? Because I think it's the most impactful systemic leverage I can think of. Unlike reforms to traditional media, entertainment, education and even politics, changes to them would reach and make at least small changes to most most of the demographics of most countries. And we need to add to this that they are not just replacing those sources of influence to individuals (except education I guess?), but also replacing the last bastion of influence: social circles.

  I wanted to make this poll to see how the community views the speed/x-risk tradeoff. I'm personally 99% x-risk and 1% speed, so I would hard agree. My prediction is most people will agree, maybe a 70/30 split, but I'm curious to see.

At what level of compute spending will AI Safety research be cut off from being considered effective altruism (if any)? Of course, saving humanity from misaligned AI could be argued to be close to priceless. But how many experiments have a direct theory of change (ToC) of how it's going to mitigate existential risk?  Perhaps a general one is fine at low compute ("it only costs $10 and 'control research' is generally thought to be a good research agenda").   But what about $5,000? What about $10,000? These numbers start to compare to or surpass what organizations like Giving What We Can receive from someone who donates for a whole year. It also starts to compete with saving a human life via programmes like those in GiveWell's top charities.  What about $20,000? $30,000? $50,000?  Over what time frame are we comfortable spending that much money on compute and still considering that money well (effectively) spent? A year? A month? A single experiment?  What kind of discovery is worth $50,000 in AIS research? Should we expect a clear ToC?  I'm very pro AI Safety, but I'm worried about some of the numbers I'm hearing for compute budgets being thrown around (compared to the information gained). I'm wondering - is anyone else is worried about a movement being (famously) concerned with cost effectiveness continuing on this path? Should we encourage more accountability?   

The 85 million children we cannot count New wars are starting before the old ones have ended. Humanitarian budgets are being cut with a chainsaw. And in this time of ultra-prioritisation, even more than before, we are asked to prove that every euro or dollar is spent on saving lives. I have been working in this sector for 15 years. I have seen its inefficiencies up close. I have also seen what it holds together. For the last few years, I have been exploring Effective Altruism and asking whether its principles can be brought into mainstream humanitarian aid. Whether that is even possible. The global aid cuts are now forcing that question into the open. I find that both necessary and deeply unsettling. Necessary, because the push toward cost-effectiveness is overdue. The tools are strong. Metrics like the Disability-Adjusted Life Year and the Wellbeing-Adjusted Life Year have made trade-offs clearer. Work by GiveWell and Rethink Priorities has improved how we compare and prioritise interventions. Unsettling, because the version of effectiveness thinking now leaking into institutional aid is the narrowest version available. EA itself has internal language for working under uncertainty. Hits-based giving, cluster thinking, and work under deep uncertainty are part of the framework. Funders like Open Philanthropy¹ regularly support areas with long causal chains and incomplete evidence when the potential upside is large. None of that nuance is what is showing up in the rooms where humanitarian budgets are being cut. What is showing up is the most legible version of cost-effectiveness, deployed as a universal filter. There is also a division of labour problem. Effective Altruism began as a framework for philanthropic choice: how should private donors direct marginal giving if they want to do the most good? Official development assistance was meant to do something different. Public funding is supposed to hold up systems, sustain services, and maintain the protective