If you believe that: - ASI might come fairly soon - ASI will either fix most of the easy problems quickly, or wipe us out - You have no plausible way of robustly shaping the outcome of the arrival of ASI for the better does it follow that you should spend a lot more on near-term cause areas now? Are people doing this? I see some people argue for increasing consumption now, but surely this would apply even more so to donations to near-term cause areas?

Some very harsh criticism of Leopold Aschenbrenner's recent AGI forecasts in the recent comments on this Metaculus question. People who are following stuff more closely than me will be able to say whether or not they are reasonable: 

Peter Thiel said a few years ago: "Courage is in shorter supply than genius". I find myself thinking a lot about that recently. 

When AI Safety people are also vegetarians, vegans or reducetarian, I am pleasantly surprised, as this is one (of many possible) signals to me they're "in it" to prevent harm, rather than because it is interesting.

Sharing a piece of advice I've given to a few people about applying for (EA) funding. I've heard various people working on early-stage projects express hesitancy about applying for EA funding because their plan isn't "complete" enough. They don't feel confident enough in their proposal, or think what they're asking for is too small. They seem to assume that EA funders only want to look at proposals with a long time-horizons from applicants who will work full-time who are confident their plan will work. In my experience (I've done various bits of grantmaking and regularly talk to EA funders), grantmakers in EA spaces are generally happy to receive applications that don't have these qualities. It's okay to apply if you just want to test a project out for a few months; maybe you won't be full-time, maybe you aren't confident in some part of the theory of change, maybe it's just a few months. You should apply and just explain your thinking, including all of your uncertainties. Funders are uncertain too, and often prefer to fund tests for a few months than commit to multi-year projects with full-time staff because tests give them useful information about you and the theory of change. Ideally, funders eventually support long-term projects too. I'm not super confident in this take, but I ran it past a few EA funders and they agreed. Note that I think this probably doesn't apply outside of EA; I understand many grant applications require detailed plans.

For those voting in the EU election and general elections in Belgium, here's an overview of the party positions when it comes to animal welfare: (For more details, click this link) ✅ means more in favor    ❌ means more against Federal election (Flanders): policy proposal PVDA 🔴 GROEN ❇️ VOORUIT 🔺 Open-VLD 🔵 CD&V 🔶 N-VA 🔆 VB ⬛️ VAT rate reduction on veterinary care and pet food✅✅❌❌❌❌✅A ban on traditional fireworks✅✅✅❌❌✅✅ Federal election (Walloon): policy proposal PTB 🔴 ECOLO ❇️ PS 🔺 LE 🐬 Défi 🌸 MR 🔵 VAT rate reduction on veterinary care and pet food✅❌✅❌❌✅A ban on traditional fireworks✅❌✅❌❌✅ Flanders election: policy proposal PVDA 🔴 GROEN ❇️ VOORUIT 🔺 Open-VLD 🔵 CD&V 🔶 N-VA 🔆 VB ⬛️ Better living conditions for broiler chickens in Flanders✅✅✅❌❌✅✅A ban on live cooking and cutting lobsters in half✅✅❌❌❌❌❌A phasing out plan of Boudewijn Seapark✅✅✅❌❌✅✅A ban on the painful surgical castration of piglets✅✅✅❌❌✅❌A ban on chick killing✅✅✅✅❌✅✅Stricter legislation around the dog and cat trade✅✅✅❌❌✅✅A duty of care for horses, dogs, cats and rabbits✅✅❌❌❌✅✅The development of cultured meat in Flanders✅✅✅✅❌✅❌Animal testing: for an animal-free strategy in Flanders✅✅✅✅❌✅✅A Flemish ban on the sale of products that harm animal welfare✅✅✅❌❌✅❌Animal welfare as a criterion in environmental permit procedure✅✅❌❌❌✅❌A punishment of animal abuse through GAS fines ❌✅❌❌❌✅✅total score11/1212/128/123/120/1211/127/12Highest score ✅      Walloon election:   PTB 🔴 ECOLO ❇️ PS 🔺 LE 🐬 Défi 🌸 MR 🔵 Total score12/1311/138/1310/136/135/13Highest score✅      EU election (Flanders):   PVDA 🔴 GROEN ❇️ VOORUIT 🔺 Open-VLD 🔵 CD&V 🔶 N-VA 🔆 VB ⬛️ Total score9/1010/1010/108/100/1010/100/10Highest score ✅✅  ✅  EU election (Walloon):   PTB 🔴 ECOLO ❇️ PS 🔺 LE 🐬 Défi 🌸 MR 🔵 Total score9/109/107/109/106/107/10Highest score✅✅ ✅   Brussels election:   PVDA 🔴 ECOLO ❇️ GROEN ❇️ PS 🔺 VOORUIT 🔺 LE 🐬 Défi 🌸 MR 🔵 O-VLD 🔵 CD&V 🔶 N-VA 🔆 VB ⬛️ Score5/65/66/64/66/65/65/64/63/60/66/65/6Highest score  ✅ ✅     ✅  Highest score Federal election (Flanders):  PVDA, GROEN, VB Highest score Federal election (Walloon):   PTB, PS, MS Highest score Flanders election:                   GROEN Highest score Walloon election:                    PTB Highest score EU election (Flanders):         GROEN, Vooruit, N-VA Highest score EU election (Walloon):          PTB, Ecolo, LE Highest score Brussels election:                  GROEN, Vooruit, N-VA TLDR: Just like my post on the topic pointed out, the leftwing parties tend to be best for animal welfare, but the far-right can often be better than the center-right

I was thinking about to what extent NDAs (either non-disclosure or non-disparagement agreements) played a role in the 2018 blowup at Alameda Research (since if there were a lot, that could be a throughline between messiness at Alameda and messiness at Open AI recently). Here's what I've collected from public records: * Not mentioned as far as I can tell in Going Infinite * Ben West: "I don’t want to speak for this person, but my own experience was pretty different. For example: Sam was fine with me telling prospective AR employees why I thought they shouldn’t join (and in fact I did do this),[4] and my severance agreement didn’t have any sort of non-disparagement clause. This comment says that none of the people who left had a non-disparagement clause, which seems like an obvious thing a person would do if they wanted to use force to prevent disparagement.[5]" From here * Kerry Vaughn: "Information about pre-2018 Alameda is difficult to obtain because the majority of those directly involved signed NDAs before their departure in exchange for severance payments. I am aware of only one employee who did not. The other people who can spreak freely on the topic are early investors in Alameda and members of the EA community who heard about Alameda from those directly involved before they signed their NDAs". From here. * ftxthrowaway: "Lastly, my severance agreement didn't have a non-disparagement clause, and I'm pretty sure no one's did. I assume that you are not hearing from staff because they are worried about the looming shitstorm over FTX now, not some agreement from four years ago." From here (it's a response to the previous) * nbouscal: “I'm the person that Kerry was quoting here, and am at least one of the reasons he believed the others had signed agreements with non-disparagement clauses. I didn't sign a severance agreement for a few reasons: I wanted to retain the ability to sue, I believed there was a non-disparagement clause, and I didn't want to sign away rights to the ownership stake that I had been verbally told I would receive. Given that I didn't actually sign it, I could believe that the non-disparagement clauses were removed and I didn't know about it, and people have just been quiet for other reasons (of which there are certainly plenty).” From here (it's a response to the previous) * Later says "I do think I was probably just remembering incorrectly about this to be honest, I looked back through things from then and it looks like there was a lot of back-and-forth about the inclusion of an NDA (among other clauses), so it seems very plausible that it was just removed entirely during that negotiation (aside from the one in the IP agreement)." Link here. * arthrowaway: "Also no non-disparagement clause in my agreement. FWIW I was one of the people who negotiated the severance stuff after the 2018 blowup, and I feel fairly confident that that holds for everyone. (But my memory is crappy, so that's mostly because I trust the FB post about what was negotiated more than you do.)" From here (it's in the same thread as the above)   Overall this tells a story where NDAs weren't a big part of the Alameda story (since I think Ben West and nbouscal at least left during the 2018 blowup, but folks should correct me if I'm wrong). This is a bit interesting to me.  Interested in if others have different takeaways.

For me, perhaps the biggest takeaway from Aschenbrenner's manifesto is that even if we solve alignment, we still have an incredibly thorny coordination problem between the US and China, in which each is massively incentivized to race ahead and develop military power using superintelligence, putting them both and the rest of the world at immense risk. And I wonder if, after seeing this in advance, we can sit down and solve this coordination problem in ways that lead to a better outcome with a higher chance than the "race ahead" strategy and don't risk encountering a short period of incredibly volatile geopolitical instability in which both nations develop and possibly use never-seen-before weapons of mass destruction. Edit: although I can see how attempts at intervening in any way and raising the salience of the issue risk making the situation worse.

Reflections 🤔 on EA & EAG following EAG London (2024): * I really liked the location this year. The venue itself was easy to get to on public transport, seemed sleek and clean, and having lots of natural light on the various floors made for a nice environment. We even got some decent sun (for London) on Saturday and Sunday. Thanks to all the organisers and volunteers involved, I know it’s lot of work setting up an event like this us and making it run smoothly. * It was good to meet people in person who I previous had only met or recognised from online interaction. I won’t single out individual 1-on-1s I had, but it was great to be able to put faces to names, and hearing peoples stories and visions in person was hugely inspiring. I talked to people involved in all sorts of cause areas and projects, and that combination of diversity, compassion, and moral seriousness is one of the best things about EA. * Listening to the two speakers from the Hibakusha Project at the closing talk was very moving, and clear case of how knowing something intellectually is not the same thing as hearing personal (and in-person) testimony. I think it would’ve been one of my conference highlights in the feedback form if we hadn’t already been asked to fill it out a few minutes beforehand! * I was going to make a point about a ‘lack of EA leadership’ turning up apart from Zach Robinson, but when I double-checked the event attendee list I think I was just wrong on this. Sure, a couple of big names didn’t turn up, and it may depend on what list of ‘EA leaders’ you’re using as a reference, but I want to admit I was directionally wrong here. * I thought Zach gave a good opening speech, but many people noted on the apparent dissonance between saying that CEA wanted to focus on ‘principles-first’ approach to EA, but that they also expected AI to be their area of most focus/highest priority and that they don't expect that to change in the near future. * Finally, while I’m sure the people I spoke to (and those who wanted to speak to me) is strongly affected by selection-effects, and my own opinions on this are fairly strong, it did feel that there was consensus on there being a lack of trust/deference/shared beliefs from ‘Bay-Area EA’:[1] * Many people think that working on AI Safety and Governance is important and valuable, but not 'overwhelmingly important' or 'the most important thing human has done/will ever do'. This included some fairly well-known names from those who attended, and basically nobody there (as far as I could tell) I interacted with held extremely 'doomer' beliefs about AI. * There was a lot of uncomfortable feeling at the community-building funding being directed to ‘longtermism’ and AI Safety in particular. This is definitely a topic I'm want to investigate more post-EAG, as I'm not sure what the truth of the matter is, but I'd certainly find it problematic if some of the anecdotes I heard were a fair representation of reality. * In any case, I think it's clear that AI Safety is no longer 'neglected' within EA, and possibly outside of it.[2] (Retracted this as, while it's not true, commenters have pointed out that it's not really the relevant metric to be tracking here)  * On a personal level, it felt a bit odd to me that the LessOnline conference was held at exactly the same time as EAG. Feels like it could be a coincidence, but on the other hand this is not a coincidence because nothing is ever a coincidence. It feeds into my impression that the Bay is not very interested in what the rest of EA has to say. * One point which I didn't get any clear answers to was 'what are the feedback mechanisms in the community to push back on this', and do such feedback mechanisms even exist? * In summary: It feels like, from my perspective, that the Bay Area/Exclusively Longtermist/AI Safety Maximalist version of EA has 'lost of the mandate of heaven', but nonetheless at the moment controls a lot of the community's money and power. This, again, is a theme I want to explicitly explore in future posts. * I am old (over 30yo) and can’t party like the young EAs anymore 😔 1. ^ I'm not sure I have a good name for this, or concrete dividing lines. But in discussions people seemed to understand what it was meant to capture. 2. ^ To me it does seem like the case for the overwhelming importance of AI has been under-argued for and under-scrutinised.

FAW#3 An interesting idea (no evidence that it would work) - just putting it here for preservation more than anything else: Insects are haraam to eat. This is obviously good news in that it means at least 20% of the global population is unlikely to contribute to the demand for insects as food. However it doesn't automatically rule-out that muslims will contribute to the demand for insects through the consumption of farmed-animals who we might use insects to feed - e.g. Chickens and Fish. It might be worth finding out if muslims would care if their chicken or fish was unnecessarily fead exclusively haraam food instead of plant-based feed. My experience as a muslim makes me feel like a lot of people would much rather prefer the animals they consume to not be fead on things which they themselves wouldn't consider halaal.

Pretty wild discussion in this podcast about how aggressively the USSR cut corners on safety in their space program in order to stay ahead of the US. In the author's telling of the history, this was in large part because Khrushchev wanted to rack up as many "firsts" (e.g., first satellite, first woman in space) as possible. This seems like it was most proximately for prestige and propaganda rather than any immediate strategic or technological benefit (though of course the space program did eventually produce such bigger benefits). Evidence of the following claim for AI: people may not need a reason to cut corners on safety because the material benefits are so high. They may do so just because of the prestige and glory of being first. https://www.lawfaremedia.org/article/chatter--the-harrowing-history-of-the-soviet-space-program-with-john-strausbaugh

Let’s research some impactful interventions! Would you come to an intervention evaluation 101 learning-together event in London? I want to run an event where we get together and do some quick-and-dirty intervention evaluation research, to learn more about how it works. I know nothing about this so we’ll be learning together! Where: (central?) London When: a mutually-agreed weekend day What: I’ll come up with a structure loosely based on (some stages of?) the AIM/Charity Entrepreneurship research process. We’ll research and compare different interventions addressing the same broad problem or cause area. For example, we might start by quickly ranking or whittling down a long list in the morning, and then do some deeper dives in the afternoon.  We’ll alternate between doing independent research and discussing that research in pairs or small groups. If you're interested in coming, please DM me: I’ll use a WhatsApp chat to coordinate. No need to firmly commit at this stage! Why? I hope to: -Better understand how charity evaluators and incubators such as GiveWell and AIM form their recommendations, so I feel more empowered to engage with their research and can identify personal cruxes -Learn how to assess interventions in areas that I think are promising, but that haven’t been discussed or researched extensively by EAs -Just learn more about the world? The event could also be useful for people who want to test their fit for EA research careers, though that’s not my own motivation. What cause area? We’d vote on a cause area beforehand. My vision here is something like ‘an area in global health and development that seems very important, but that hasn’t been discussed, or has been discussed relatively minimally, by EAs’. What next? If this goes well, we could hold these events regularly and/or collaborate and co-work on more substantive research projects. Is there a point to this?  AIM’s process takes ~1300 hours and is undertaken by skilled professional researchers; obviously we’re not going to produce recommendations of anywhere near similar quality. My motivation is to become personally better-informed and better-engaged with the nitty gritty of EA/being impactful in the world, rather than to reinvent the GiveWell wheel.  That said, we’re stronger together: if 50 people worked on assessing a cause area together, they’d only have to spend 26 hours each to collectively equal the AIM process. 26 hours isn’t trivial (and nor is 50 people), but it’s not crazy implausible either. If collectives of EAs are putting AIM-level amounts of hours into intervention evaluation in their spare time, seems like a win?

Prompted by a different forum: > ...as a small case study, the Effective Altruism forum has been impoverished over the last few years by not being lenient with valuable contributors when they had a bad day. > > In a few cases, I later learnt that some longstanding user had a mental health breakdown/psychotic break/bipolar something or other. To some extent this is an arbitrary category, and you can interpret going outside normality through the lens of mental health, or through the lens of "this person chose to behave inappropriately". Still, my sense is that leniency would have been a better move when people go off the rails. > > In particular, the best move seems to me a combination of: > > * In the short term, when a valued member is behaving uncharacteristically badly, stop them from posting > * Followup a week or a few weeks later to see how the person is doing > > Two factors here are: > > * There is going to be some overlap in that people with propensity for some mental health disorders might be more creative, better able to see things from weird angles, better able to make conceptual connections. > * In a longstanding online community, people grow to care about others. If a friend goes of the rails, there is the question of how to stop them from causing harm to others, but there is also the question of how to help them be ok, and the second one can just dominate sometimes.

I am not under any non-disparagement obligations to OpenAI. It is important to me that people know this, so that they can trust any future policy analysis or opinions I offer. I have no further comments at this time.

EA organizations frequently ask for people to run criticism by them ahead of time. I’ve been wary of the push for this norm. My big concerns were that orgs wouldn’t comment until a post was nearly done, and that it would take a lot of time. My recent post  mentioned a lot of people and organizations, so it seemed like useful data. I reached out to 12 email addresses, plus one person in FB DMs and one open call for information on a particular topic.  This doesn’t quite match what you see in the post because some people/orgs were used more than once, and other mentions were cut. The post was in a fairly crude state when I sent it out. Of those 14: 10 had replied by the start of next day. More than half of those replied within a few hours. I expect this was faster than usual because no one had more than a few paragraphs relevant to them or their org, but is still impressive. It’s hard to say how sending an early draft changed things. One person got some extra anxiety because their paragraph was full of TODOs (because it was positive and I hadn’t worked as hard fleshing out the positive mentions ahead of time). I could maybe have saved myself one stressful interaction if I’d realized I was going to cut an example ahead of time Only 80,000 Hours, Anima International, and GiveDirectly failed to respond before publication (7 days after I emailed them). Of those, only 80k's mention was negative. I didn’t keep as close track of changes, but at a minimum replies led to 2 examples being removed entirely, 2 clarifications and some additional information that made the post better. So overall I'm very glad I solicited comments, and found the process easier than expected. 

This is a cold take that’s probably been said before, but I thought it bears repeating occasionally, if only for the reminder: The longtermist viewpoint has gotten a lot of criticism for prioritizing “vast hypothetical future populations” over the needs of "real people," alive today. The mistake, so the critique goes, is the result of replacing ethics with math, or utilitarianism, or something cold and rigid like that. And so it’s flawed because it lacks the love or duty or "ethics of care" or concern for justice that lead people to alternatives like mutual aid and political activism. My go-to reaction to this critique has become something like “well you don’t need to prioritize vast abstract future generations to care about pandemics or nuclear war, those are very real things that could, with non-trivial probability, face us in our lifetimes.” I think this response has taken hold in general among people who talk about X-risk. This probably makes sense for pragmatic reasons. It’s a very good rebuttal to the “cold and heartless utilitarianism/pascal's mugging” critique. But I think it unfortunately neglects the critical point that longtermism, when taken really seriously — at least the sort of longtermism that MacAskill writes about in WWOTF, or Joe Carlsmith writes about in his essays — is full of care and love and duty. Reading the thought experiment that opens the book about living every human life in sequential order reminded me of this. I wish there were more people responding to the “longtermism is cold and heartless” critique by making the case that no, longtermism at face value is worth preserving because it's the polar opposite of heartless. Caring about the world we leave for the real people, with emotions and needs and experiences as real as our own, who very well may inherit our world but who we’ll never meet, is an extraordinary act of empathy and compassion — one that’s way harder to access than the empathy and warmth we might feel for our neighbors by default. It’s the ultimate act of care. And it’s definitely concerned with justice. (I mean, you can also find longtermism worthy because of something something math and cold utilitarianism. That’s not out of the question. I just don’t think it’s the only way to reach that conclusion.)

Do we know if @Paul_Christiano or other ex-lab people working on AI policy have non-disparagement agreements with OpenAI or other AI companies? I know Cullen doesn't, but I don't know about anybody else. I know NIST isn't a regulatory body, but it still seems like standards-setting should be done by people who have no unusual legal obligations. And of course, some other people are or will be working at regulatory bodies, which may have more teeth in the future. To be clear, I want to differentiate between Non-Disclosure Agreements, which are perfectly sane and reasonable in at least a limited form as a way to prevent leaking trade secrets, and non-disparagement agreements, which prevents you from saying bad things about past employers. The latter seems clearly bad to have for anybody in a position to affect policy. Doubly so if the existence of the non-disparagement agreement itself is secretive.

Having a baby and becoming a parent has had an incredible impact on me. Now more than ever, I feel more connected and concerned about the wellbeing of others. I feel as though my heart has literally grown. I wanted to share this as I expect there are many others who are questioning whether to have children -- perhaps due to concerns about it limiting their positive impact, among many others. But I'm just here to say it's been beautiful, and amazing, and I look forward to the day I get to talk with my son about giving back in a meaningful way.  

In this "quick take", I want to summarize some my idiosyncratic views on AI risk.  My goal here is to list just a few ideas that cause me to approach the subject differently from how I perceive most other EAs view the topic. These ideas largely push me in the direction of making me more optimistic about AI, and less likely to support heavy regulations on AI. (Note that I won't spend a lot of time justifying each of these views here. I'm mostly stating these points without lengthy justifications, in case anyone is curious. These ideas can perhaps inform why I spend significant amounts of my time pushing back against AI risk arguments. Not all of these ideas are rare, and some of them may indeed be popular among EAs.) 1. Skepticism of the treacherous turn: The treacherous turn is the idea that (1) at some point there will be a very smart unaligned AI, (2) when weak, this AI will pretend to be nice, but (3) when sufficiently strong, this AI will turn on humanity by taking over the world by surprise, and then (4) optimize the universe without constraint, which would be very bad for humans. By comparison, I find it more likely that no individual AI will ever be strong enough to take over the world, in the sense of overthrowing the world's existing institutions and governments by surprise. Instead, I broadly expect unaligned AIs will integrate into society and try to accomplish their goals by advocating for their legal rights, rather than trying to overthrow our institutions by force. Upon attaining legal personhood, unaligned AIs can utilize their legal rights to achieve their objectives, for example by getting a job and trading their labor for property, within the already-existing institutions. Because the world is not zero sum, and there are economic benefits to scale and specialization, this argument implies that unaligned AIs may well have a net-positive effect on humans, as they could trade with us, producing value in exchange for our own property and services. Note that my claim here is not that AIs will never become smarter than humans. One way of seeing how these two claims are distinguished is to compare my scenario to the case of genetically engineered humans. By assumption, if we genetically engineered humans, they would presumably eventually surpass ordinary humans in intelligence (along with social persuasion ability, and ability to deceive etc.). However, by itself, the fact that genetically engineered humans will become smarter than non-engineered humans does not imply that genetically engineered humans would try to overthrow the government. Instead, as in the case of AIs, I expect genetically engineered humans would largely try to work within existing institutions, rather than violently overthrow them. 2. AI alignment will probably be somewhat easy: The most direct and strongest current empirical evidence we have about the difficulty of AI alignment, in my view, comes from existing frontier LLMs, such as GPT-4. Having spent dozens of hours testing GPT-4's abilities and moral reasoning, I think the system is already substantially more law-abiding, thoughtful and ethical than a large fraction of humans. Most importantly, this ethical reasoning extends (in my experience) to highly unusual thought experiments that almost certainly did not appear in its training data, demonstrating a fair degree of ethical generalization, beyond mere memorization. It is conceivable that GPT-4's apparently ethical nature is fake. Perhaps GPT-4 is lying about its motives to me and in fact desires something completely different than what it professes to care about. Maybe GPT-4 merely "understands" or "predicts" human morality without actually "caring" about human morality. But while these scenarios are logically possible, they seem less plausible to me than the simple alternative explanation that alignment—like many other properties of ML models—generalizes well, in the natural way that you might similarly expect from a human. Of course, the fact that GPT-4 is easily alignable does not immediately imply that smarter-than-human AIs will be easy to align. However, I think this current evidence is still significant, and aligns well with prior theoretical arguments that alignment would be easy. In particular, I am persuaded by the argument that, because evaluation is usually easier than generation, it should be feasible to accurately evaluate whether a slightly-smarter-than-human AI is taking bad actions, allowing us to shape its rewards during training accordingly. After we've aligned a model that's merely slightly smarter than humans, we can use it to help us align even smarter AIs, and so on, plausibly implying that alignment will scale to indefinitely higher levels of intelligence, without necessarily breaking down at any physically realistic point. 3. The default social response to AI will likely be strong: One reason to support heavy regulations on AI right now is if you think the natural "default" social response to AI will lean too heavily on the side of laissez faire than optimal, i.e., by default, we will have too little regulation rather than too much. In this case, you could believe that, by advocating for regulations now, you're making it more likely that we regulate AI a bit more than we otherwise would have, pushing us closer to the optimal level of regulation. I'm quite skeptical of this argument because I think that the default response to AI (in the absence of intervention from the EA community) will already be quite strong. My view here is informed by the base rate of technologies being overregulated, which I think is quite high. In fact, it is difficult for me to name even a single technology that I think is currently clearly underregulated by society. By pushing for more regulation on AI, I think it's likely that we will overshoot and over-constrain AI relative to the optimal level. In other words, my personal bias is towards thinking that society will regulate technologies too heavily, rather than too loosely. And I don't see a strong reason to think that AI will be any different from this general historical pattern. This makes me hesitant to push for more regulation on AI, since on my view, the marginal impact of my advocacy would likely be to push us even further in the direction of "too much regulation", overshooting the optimal level by even more than what I'd expect in the absence of my advocacy. 4. I view unaligned AIs as having comparable moral value to humans: This idea was explored in one of my most recent posts. The basic idea is that, under various physicalist views of consciousness, you should expect AIs to be conscious, even if they do not share human preferences. Moreover, it seems likely that AIs — even ones that don't share human preferences — will be pretrained on human data, and therefore largely share our social and moral concepts. Since unaligned AIs will likely be both conscious and share human social and moral concepts, I don't see much reason to think of them as less "deserving" of life and liberty, from a cosmopolitan moral perspective. They will likely think similarly to the way we do across a variety of relevant axes, even if their neural structures are quite different from our own. As a consequence, I am pretty happy to incorporate unaligned AIs into the legal system and grant them some control of the future, just as I'd be happy to grant some control of the future to human children, even if they don't share my exact values. Put another way, I view (what I perceive as) the EA attempt to privilege "human values" over "AI values" as being largely arbitrary and baseless, from an impartial moral perspective. There are many humans whose values I vehemently disagree with, but I nonetheless respect their autonomy, and do not wish to deny these humans their legal rights. Likewise, even if I strongly disagreed with the values of an advanced AI, I would still see value in their preferences being satisfied for their own sake, and I would try to respect the AI's autonomy and legal rights. I don't have a lot of faith in the inherent kindness of human nature relative to a "default unaligned" AI alternative. 5. I'm not fully committed to longtermism: I think AI has an enormous potential to benefit the lives of people who currently exist. I predict that AIs can eventually substitute for human researchers, and thereby accelerate technological progress, including in medicine. In combination with my other beliefs (such as my belief that AI alignment will probably be somewhat easy), this view leads me to think that AI development will likely be net-positive for people who exist at the time of alignment. In other words, if we allow AI development, it is likely that we can use AI to reduce human mortality, and dramatically raise human well-being for the people who already exist. I think these benefits are large and important, and commensurate with the downside potential of existential risks. While a fully committed strong longtermist might scoff at the idea that curing aging might be important — as it would largely only have short-term effects, rather than long-term effects that reverberate for billions of years — by contrast, I think it's really important to try to improve the lives of people who currently exist. Many people view this perspective as a form of moral partiality that we should discard for being arbitrary. However, I think morality is itself arbitrary: it can be anything we want it to be. And I choose to value currently existing humans, to a substantial (though not overwhelming) degree. This doesn't mean I'm a fully committed near-termist. I sympathize with many of the intuitions behind longtermism. For example, if curing aging required raising the probability of human extinction by 40 percentage points, or something like that, I don't think I'd do it. But in more realistic scenarios that we are likely to actually encounter, I think it's plausibly a lot better to accelerate AI, rather than delay AI, on current margins. This view simply makes sense to me given the enormously positive effects I expect AI will likely have on the people I currently know and love, if we allow development to continue.

Please people, do not treat Richard Hannania as some sort of worthy figure who is a friend of EA. He was a Nazi, and whilst he claims he moderated his views, he is still very racist as far as I can tell. Hannania called for trying to get rid of all non-white immigrants in the US, and the sterilization of everyone with an IQ under 90 indulged in antisemitic attacks on the allegedly Jewish elite, and even post his reform was writing about the need for the state to harass and imprison Black people specifically ('a revolution in our culture or form of government. We need more policing, incarceration, and surveillance of black people' https://en.wikipedia.org/wiki/Richard_Hanania).  Yet in the face of this, and after he made an incredibly grudging apology about his most extreme stuff (after journalists dug it up), he's been invited to Manifiold's events and put on Richard Yetter Chappel's blogroll.  DO NOT DO THIS. If you want people to distinguish benign transhumanism (which I agree is a real thing*) from the racist history of eugenics, do not fail to shun actual racists and Nazis. Likewise, if you want to promote "decoupling" factual beliefs from policy recommendations, which can be useful, do not duck and dive around the fact that virtually every major promoter of scientific racism ever, including allegedly mainstream figures like Jensen, worked with or published with actual literal Nazis (https://www.splcenter.org/fighting-hate/extremist-files/individual/arthur-jensen).  I love most of the people I have met through EA, and I know that-despite what some people say on twitter- we are not actually a secret crypto-fascist movement (nor is longtermism specifically, which whether you like it or not, is mostly about what its EA proponents say it is about.) But there is in my view a disturbing degree of tolerance for this stuff in the community, mostly centered around the Bay specifically. And to be clear I am complaining about tolerance for people with far-right and fascist ("reactionary" or whatever) political views, not people with any particular personal opinion on the genetics of intelligence. A desire for authoritarian government enforcing the "natural" racial hierarchy does not become okay, just because you met the person with the desire at a house party and they seemed kind of normal and chill or super-smart and nerdy.  I usually take a way more measured tone on the forum than this, but here I think real information is given by getting shouty.  *Anyone who thinks it is automatically far-right to think about any kind of genetic enhancement at all should go read some Culture novels, and note the implied politics (or indeed, look up the author's actual die-hard libertarian socialist views.) I am not claiming that far-left politics is innocent, just that it is not racist. 

Animal Justice Appreciation Note Animal Justice et al. v A.G of Ontario 2024 was recently decided and struck down large portions of Ontario's ag-gag law. A blog post is here. The suit was partially funded by ACE, which presumably means that many of the people reading this deserve partial credit for donating to support it. Thanks to Animal Justice (Andrea Gonsalves, Fredrick Schumann, Kaitlyn Mitchell, Scott Tinney), co-applicants Jessica Scott-Reid and Louise Jorgensen, and everyone who supported this work!

Why are April Fools jokes still on the front page? On April 1st, you expect to see April Fools' posts and know you have to be extra cautious when reading strange things online. However, April 1st was 13 days ago and there are still two posts that are April Fools posts on the front page. I think it should be clarified that they are April Fools jokes so people can differentiate EA weird stuff from EA weird stuff that's a joke more easily. Sure, if you check the details you'll see that things don't add up, but we all know most people just read the title or first few paragraphs.

Marcus Daniell appreciation note @Marcus Daniell, cofounder of High Impact Athletes, came back from knee surgery and is donating half of his prize money this year. He projects raising $100,000. Through a partnership with Momentum, people can pledge to donate for each point he gets; he has raised $28,000 through this so far. It's cool to see this, and I'm wishing him luck for his final year of professional play!