I'm fairly disappointed with how much discussion I've seen recently that either doesn't bother to engage with ways in which the poster might be wrong, or only engages with weak versions. It's possible that the "debate" format of the last week has made this worse, though not all of the things I've seen were directly part of that. I think that not engaging at all, and merely presenting one side while saying that's what you're doing, seems better than presenting and responding to counterarguments (but only the weak ones), which still seems better than strawmanning arguments that someone else has presented.

I mentioned a few months ago that I was planning to resign from the board of EV UK: I’ve now officially done so. Since last November, I’ve been recused from the board on all matters associated with FTX and related topics, which has ended up being a large proportion of board business. (This is because the recusal affected not just decisions that were directly related to the collapse of FTX, but also many other decisions for which the way EV UK has been affected by the collapse of FTX was important context.) I know I initially said that I’d wait for there to be more capacity, but trustee recruitment has moved more slowly than I’d anticipated, and with the ongoing recusal I didn’t expect to add much capacity for the foreseeable future, so it felt like a natural time to step down.   It’s been quite a ride over the last eleven years. Effective Ventures has grown to a size far beyond what I expected, and I’ve felt privileged to help it on that journey. I deeply respect the rest of the board, and the leadership teams at EV, and I’m glad they’re at the helm. Some people have asked me what I’m currently working on, and what my plans are. This year has been quite spread over a number of different things, including fundraising, helping out other EA-adjacent public figures, support for GPI, CEA and 80,000 Hours, writing additions to What We Owe The Future and helping with the print textbook version of utilitarianism.net that’s coming out next year. It’s also personally been the toughest year of my life; my mental health has been at its worst in over a decade, and I’ve been trying to deal with that, too. At the moment, I’m doing three main things: - Some public engagement, in particular around the WWOTF paperback and foreign language book launches and at EAGxBerlin. This has been and will be lower-key than the media around WWOTF last year, and more focused on in-person events; I’m also more focused on fundraising than I was before.  - Research into "trajectory changes”: in particular, ways of increasing the wellbeing of future generations other than 'standard' existential risk mitigation strategies, in particular on issues that arise even if we solve AI alignment, like digital sentience and the long reflection. I’m also doing some learning to try to get to grips on how to update properly on the latest developments in AI, in particular with respect to the probability of an intelligence explosion in the next decade, and on how hard we should expect AI alignment to be. - Gathering information for what I should focus on next. In the medium term, I still plan to be a public proponent of EA-as-an-idea, which I think plays to my comparative advantage, and because I’m worried about people neglecting “EA qua EA”. If anything, all the crises faced by EA and by the world in the last year has reminded me of just how deeply I believe in EA as a project, and how the message of taking a thoughtful, humble, and scientific approach to doing good is more important than ever. The precise options I’m considering are still quite wide-ranging, including: a podcast and/or YouTube show and/or substack; a book on effective giving; a book on evidence-based living; or deeper research into the ethics and governance questions that arise even if we solve AI alignment. I hope to decide on that by the end of the year.

(Clarification about my views in the context of the AI pause debate) I'm finding it hard to communicate my views on AI risk. I feel like some people are responding to the general vibe they think I'm giving off rather than the actual content. Other times, it seems like people will focus on a narrow snippet of my comments/post and respond to it without recognizing the context. For example, one person interpreted me as saying that I'm against literally any AI safety regulation. I'm not. For a full disclosure, my views on AI risk can be loosely summarized as follows: * I think AI will probably be very beneficial for humanity. * Nonetheless, I think that there are credible, foreseeable risks from AI that could do vast harm, and we should invest heavily to ensure these outcomes don't happen. * I also don't think technology is uniformly harmless. Plenty of technologies have caused net harm. Factory farming is a giant net harm that might have even made our entire industrial civilization a mistake! * I'm not blindly against regulation. I think all laws can and should be viewed as forms of regulations, and I don't think it's feasible for society to exist without laws. * That said, I'm also not blindly in favor of regulation, even for AI risk. You have to show me that the benefits outweigh the harm * I am generally in favor of thoughtful, targeted AI regulations that align incentives well, and reduce downside risks without completely stifling innovation. * I'm open to extreme regulations and policies if or when an AI catastrophe seems imminent, but I don't think we're in such a world right now. I'm not persuaded by the arguments that people have given for this thesis, such as Eliezer Yudkowsky's AGI ruin post.

(COI note: I work at OpenAI. These are my personal views, though.) My quick take on the "AI pause debate", framed in terms of two scenarios for how the AI safety community might evolve over the coming years: 1. AI safety becomes the single community that's the most knowledgeable about cutting-edge ML systems. The smartest up-and-coming ML researchers find themselves constantly coming to AI safety spaces, because that's the place to go if you want to nerd out about the models. It feels like the early days of hacker culture. There's a constant flow of ideas and brainstorming in those spaces; the core alignment ideas are standard background knowledge for everyone there. There are hackathons where people build fun demos, and people figuring out ways of using AI to augment their research. Constant interactions with the models allows people to gain really good hands-on intuitions about how they work, which they leverage into doing great research that helps us actually understand them better. When the public ends up demanding regulation, there's a large pool of competent people who are broadly reasonable about the risks, and can slot into the relevant institutions and make them work well. 2. AI safety becomes much more similar to the environmentalist movement. It has broader reach, but alienates a lot of the most competent people in the relevant fields. ML researchers who find themselves in AI safety spaces are told they're "worse than Hitler" (which happened to a friend of mine, actually). People get deontological about AI progress; some hesitate to pay for ChatGPT because it feels like they're contributing to the problem (another true story); others overemphasize the risks of existing models in order to whip up popular support. People are sucked into psychological doom spirals similar to how many environmentalists think about climate change: if you're not depressed then you obviously don't take it seriously enough. Just like environmentalists often block some of the most valuable work on fixing climate change (e.g. nuclear energy, geoengineering, land use reform), safety advocates block some of the most valuable work on alignment (e.g. scalable oversight, interpretability, adversarial training) due to acceleration or misuse concerns. Of course, nobody will say they want to dramatically slow down alignment research, but there will be such high barriers to researchers getting and studying the relevant models that it has similar effects. The regulations that end up being implemented are messy and full of holes, because the movement is more focused on making a big statement than figuring out the details. Obviously I've exaggerated and caricatured these scenarios, but I think there's an important point here. One really good thing about the AI safety movement, until recently, is that the focus on the problem of technical alignment has nudged it away from the second scenario (although it wasn't particularly close to the first scenario either, because the "nerding out" was typically more about decision theory or agent foundations than ML itself). That's changed a bit lately, in part because a bunch of people seem to think that making technical progress on alignment is hopeless. I think this is just not an epistemically reasonable position to take: history is full of cases where people dramatically underestimated the growth of scientific knowledge, and its ability to solve big problems. Either way, I do think public advocacy for strong governance measures can be valuable, but I also think that "pause AI" advocacy runs the risk of pushing us towards scenario 2. Even if you think that's a cost worth paying, I'd urge you to think about ways to get the benefits of the advocacy while reducing that cost and keeping the door open for scenario 1.

Some lawyers claim that there may be significant (though not at all ideal) whistleblowing protection for individuals at AI companies that don't fully comply with the Voluntary Commitments: https://katzbanks.com/wp-content/uploads/KBK-Law360-Despite-Regulation-Lag-AI-Whistleblowers-Have-Protections.pdf

Have there ever been any efforts to try to set up EA-oriented funding organisations that focus on investing donations in such a way as to fund high-utility projects in very suitable states of the world? They could be pure investment vehicles that have high expected utility, but that lose all their money by some point in time in the modal case. The idea would be something like this: For a certain amount of dollars, to maximise utility, to first order, one has to decide how much to spend on which causes and how to distribute the spending over time.  However, with some effort, one could find investments that pay off conditionally on states of the world where specific interventions might have very high utility. Some super naive examples would be a long-dated option structure that pays off if the price for wheat explodes, or a CDS that pays off if JP Morgan collapses. This would then allow organisations to intervene through targeted measures, for example, food donations. This is similar to the concept of a “tail hedge” - an investment that pays off massively when other investments do poorly, that is when the marginal utility of owning an additional dollar is very high. Usually, one would expect such investments to carry negatively, that is, to be costly over time possibly even with negative unconditional expected returns. However, if an EA utility function is sufficiently different from a typical market participant, this need not be the case, even in dollar terms (?). Clearly, the arguments here would have to be made a lot more rigorous and quantitative to see whether this might be attractive at all. I’d be interested in any references etc.

I believed for a while that public exposés are often a bad idea in EA, and the current Nonlinear drama certainly appears to be confirmatory evidence. I'm pretty confused about why other people's conclusions appears to be different from mine; this all seems extremely obvious to me.

Would newer people find it valuable to have some kind of 80,000 hours career chatbot that had access to the career guide, podcast notes, EA forum posts, job postings, etc, and then answered career questions? I’m curious if it could be designed to be better than just a raw read of the career guide or at least a useful add-on to the career guide. Potential features: * It could collect your conversation and convert most of it into an application for a (human) 1-on-1 meeting. * You could have a speech-to-text option to ramble all the things you’ve been thinking of. * ??? If anyone from 80k is reading this, I’d be happy to build this as a paid project.

I was watching this video yesterday https://www.youtube.com/watch?v=C25qzDhGLx8  It's a video about ageing and death, and how society has come to accept death as positive thing that gives life meaning. CGP grey goes on to explain that this is not true, which I agree with - I would still have a lot of meaning in my life if I stopped ageing. The impact of people living longer on society and politics is more uncertain, but I don't see it being a catastrophe - society has adapted to 'worse'.  The thing is that ageing can be seen as a disease that affects 100% of the world population, causing untold suffering at a scale that completely dwarfs any other disease in human history.  Curing ageing = infinite QALYs. It definitely seems like something worth considering to me. 

Should We Push For An AI Pause Might Be The Wrong Question A quick thought on the recent discussion on whether pushing for a pause on frontier AI models is a good idea or not. It seems obvious to me that within the next 3 years the top AI labs will be producing AI that causes large swaths of the public to push for a pause.  Is it therefore more prudent to ask the following question: when much of the public wants a pause, what should our (the EA community) response be?

It seems prima facie plausible to me that interventions that save human lives do not increase utility on net, due to the animal suffering caused by saving human life. Has anyone in the broader EA community looked into this? I'm not strongly committed to this, but I'd be interested in seeing what people have reasoned about this.

Wanted to give a shoutout to Ajeya Cotra (from OpenPhil), for her great work explaining AI stuff on a recent Freakonomics podcast series. Her explanations about both her work on the development of AI, and her easy to understand predictions of how AI might progress from here were great, she was my favourite expert on the series. People have been looking for more high quality public communicators to get EA/AI safety stuff out there, perhaps Ajeya could be a candidate if she's keen?  

Against "the burden of proof is on X" Instead, I recommend: "My prior is [something], here's why". I'm even more against "the burden of proof for [some policy] is on X" - I mean, what does "burden of proof" even mean in the context of policy? but hold that thought.   An example that I'm against: "The burden of proof for vaccines helping should be on people who want to vaccinate, because it's unusual to put something in your body" I'm against it because  1. It implicitly assumes that vaccines should be judged as part of the group "putting something in your body" 2. It's a conversation stopper. It claims one of the sides of the conversation has nothing to do.   I prefer: "my prior for vaccines is that they're bad, because my prior for putting things in my body is bad (but I'm open to changing my mind from evidence, and I'm open to maybe using a different prior if you have a better idea)"   I also like: "my prior is that governments should not force people to do things, and so I'm against forcing people to be vaccinated" or "my prior is that governments are allowed to force people to do things that, by scientific consensus, protect them". I like that we're discussing explicitly "which priors should we use to decide which policy to accept and which not to"   What got me to write a about this now: I don't like the discussion about who has the "burden of proof" to decide we should or shouldn't have an AI pause. I would prefer discussing which prior to use for it. For example, * Should our prior be "should we pause any new technology", and so AI is "just" another new technology? * Should our prior be that an AI is an extinction risk like a meteor in "don't look up", and so should be paused unless we have further evidence showing reasons to not-pause it? * Should our priors be based on expert polls (do experts recommend a pause), and should we require evidence in order to change our mind from those polls? My opinion: we should explicitly discuss which priors to use (which isn't an easy question), and not just assume that one "side" has the "burden of proof"   </rant>

Why is it that I must return from 100% of EAGs with either covid or a cold? Perhaps my immune system just sucks or it's impossible to avoid due to asymptomatic cases, but in case it's not: If you get a cold before an EAG(x), stay home! For those who do this already, thank you! 

I was going to post something for careers week but it was delayed for various reasons (including the mandatory last minute rewrite). I plan to post it in the next couple of weeks.

I just came across this old comment by Wei Dai which has aged well, for unfortunate reasons. > I think a healthy dose of moral uncertainty (and normative uncertainty in > general) is really important to have, because it seems pretty easy for any > ethical/social movement to become fanatical or to incur a radical element, and > end up doing damage to itself, its members, or society at large. (“The road to > hell is paved with good intentions” and all that.)

A couple of times I've probably been too defensive about people saying things behind my back. That's not how I want to behave. I'm sorry.

My overall impression is that the CEA community health team (CHT from now on) are well intentioned but sometimes understaffed and other times downright incompetent. It's hard to me to be impartial here, and I understand that their failures are more salient to me than their successes. Yet I endorse the need for change, at the very least including 1) removing people from the CHT that serve as a advisors to any EA funds or have other conflict of interest positions, 2) hiring HR and mental health specialists with credentials, 3) publicly clarifying their role and mandate.  My impression is that the most valuable function that the CHT provides is as support of community building teams across the world, from advising community builders to preventing problematic community builders from receiving support. If this is the case, I think it would be best to rebrand the CHT as a CEA HR department, and for CEA to properly hire the community builders who are now supported as grantees, which one could argue is an employee misclassification. I would not be comfortable discussing these issues openly out of concern for the people affected, but here are some horror stories: 1. A CHT staff pressured a community builder to put through with and include a community member with whom they weren't comfortable interacting. 2. A CHT staff pressured a community builder to not press charges against a community member who they felt harassed by. 3. After a restraining order was set by the police in place in this last case, the CHT refused to liaison with the EA Global team to deny access to the person restrained, even knowing that the affected community builder would be attending the event. 4. My overall sense is that CHT is not very mindful of the needs of community builders in other contexts. Two very promising professionals I've mentored have dissociated from EA, and rejected a grant, in large part because of how they were treated by the CHT. 5. My impression is that the CHT staff undermines the legitimacy of local communities to make their own decisions. CEA is often perceived as a source of authority, and the CHT has a lot of sway in funding decisions. This makes it so that it is really hard for local groups to go against the wishes of the CHT, who is the main intermediary with groups. I wish this relation was more transparent, so they could be hold accountable for it. To be clear, I think that these stories have a lot of nuance to them and are in each cases the result of the CHT making what they thought were the best decisions they could make with the tools they had, but in each of them I noticed that I ended up disagreeing with the decisions made and feeling very uncomfortable with how the whole community structure was set up.

The OECD are currently hiring for a few potentially high-impact roles in the tax policy space: The Centre for Tax Policy and Administration (CTPA) * Executive Assistant to the Director and Office Manager (closes 6th October) * Senior programme officer (closes 28th September) * Head of Division - Tax Administration and VAT (closes 5th October) * Head of Division - Tax Policy and Statistics (closes 5th October) * Head of Division - Cross-Border and International Tax (closes 5th October) * Team Leader - Tax Inspectors Without Borders (closes 28th September)  I know less about the impact of these other areas but these look good: Trade and Agriculture Directorate (TAD) * Head of Section, Codes and Schemes - Trade and Agriculture Directorate (closes 25th September) * Programme Co-ordinator (closes 25th September) International Energy Agency (IEA) * Clean Energy Technology Analysts (closes 24th September) * Modeller and Analyst – Clean Shipping & Aviation (closes 24th September) * Analyst & Modeller – Clean Energy Technology Trade (closes 24th September) * Data Analyst - Temporary (closes 28-09-2023) Financial Action Task Force  * Policy Analyst(s), Anti-Money Laundering & Combatting Terrorist Financing

Julia Nefsky is giving a research seminar in the Institute for Futures Studies titled "Expected utility, the pond analogy and imperfect duties", which sounds interesting for the community. It will be on September 27 at 10:00-11:45 (CEST) and can be attended for free in person or online (via zoom). You can find the abstract here and register here. I don't know Julia or her work and I'm not philosopher, so I cannot directly assess the expected quality of the seminar, but I've seen several seminars from the Institute for Futures Studies that where very good (eg. from Olle Häggström --and in Sep 20 Anders Sandberg gives one as well). I hope this is useful information.

Being able to agree and disagreevote on posts feels like it might be great. Props to the forum team.

I increasingly realize just how emotionally inaccessible the concept of ethical offsetting is to most people. With regard to climate, the figure I remember from William MacAskill is that he estimates one dollar donated to Clean Air Taskforce to save 1 ton of CO2. If you take this seriously, then you basically don't need to bother with any efforts to have less waste in your personal life. (With the possible exception being your image, but only if you're a public figure.) 

Politico just published a fairly negative article about EA and UK politics. Previously they’ve published similar articles about EA and Brussels. I think EA tends to focus on the inside game, or narrow EA, and I believe this increases the likelihood of articles such as this. I worry articles such as this will make people in positions of influence less likely to want to be associated with EA, and that this in the long run will undermine efforts to bring about the policy changes we desire. Still, of course, this focus on the inside game is also pretty cost-effective (for the short term, at least). Is it worth the trade-off? What do people think? My gut feeling is that, putting to one side the question of which is the most effective strategy for reducing x-risk etc., the 'narrow EA' strategy is a mistake because there's a good chance it is wrong to try to guide society without broader societal participation. In other words, if MacAskill argues here we should get our shit together first and then either a) collectively decide on a way forward or b) allow for everyone to make their own way forward, I think it's also important that 'the getting our shit together' has broad societal participation.

Within Epoch there is a months-long debate about how we should report growth rates for certain key quantities such as the amount of compute used for training runs. I have been an advocate of an unusual choice: orders-of-magnitude per year (abbreviated OOMs/year). Why is that? Let's look at other popular choices. Doubling times. This has become the standard in AI forecasting, and its a terrible metric. On the positive, it is an intuitive metric that both policy makers and researchers are familiar with. But it is absolutely horrid to make calculations. For example, if I know that the cost of AI training runs is doubling every 0.6 years, and the FLOP/$ is doubling every 2.5 years, then the FLOP per training run is doubling every (10.6+12.5)−1 years, which is very difficult to solve in your head!&nbsp;<span class="footnote-reference" role="doc-noteref" id="fnref695fkta3x2r">^{<a href="#fn695fkta3x2r">[1]</a>}</span>&nbsp;<span class="footnote-reference" role="doc-noteref" id="fnrefnhr7mmw210m">^{<a href="#fnnhr7mmw210m">[2]</a>}</span></p><p><strong>Percent growth</strong>. This is a choice often favoured in economics, where eg the growth rate of GDP is often reported as 3%, etc. Unlike doubling times, percent growth composes nicely - you just have to add them up!&nbsp;<span class="footnote-reference" role="doc-noteref" id="fnrefxqei73ax83">^{<a href="#fnxqei73ax83">[3]</a>}</span>&nbsp;However, I also find percent changes somewhat prone to confusion. For instance, when I tell people that model size has increased 200% since X years ago, I sometimes have had people misunderstand this as saying that it has increased by a factor of 2.&nbsp;</p><p>Ultimately, a very common operation that I find myself doing in my head is "if the effective FLOP used in AI training runs grows at a certain rate, how quickly will we traverse from the scale of current training runs (1e25 FLOP) to a certain threshold (eg 1e30)". OOMs/year makes this computation easy, even if I need to account for multiple factors such as hardware increases, investment and algorithmic improvements. Eg if these grow respectively as 0.4 OOM/year, 0.1 OOM/year and 0.5 OOM/year, then I know the total effective growth is 1.0 OOM/year, and it will take 5 years to cross that 5 OOM scale gap. And if investment suddenly stopped growing, then I would be able to quickly understand that the pace would be halved, and the gap would then take 10 years to cross.</p><p>Sadly OOMs/year is uncommon, and both researchers and policy makers struggle to understand it. I think this is a missed opportunity, and that AI forecasting would be easier to reason about if we moved to it, or at the very least abandoned the very badly behaved doubling time framing.</p><p>What do you think? Do you agree we should move past doubling times to a better choice? Which choice would you favour?</p><ol class="footnotes" role="doc-endnotes"><li class="footnote-item" role="doc-endnote" id="fn695fkta3x2r"><span class="footnote-back-link">^{<strong><a href="#fnref695fkta3x2r">^</a></strong>}</span><div class="footnote-content"><p>&nbsp;I won't enter into the technical details, but it also has some very unintuitive results when combining the results with uncertainty. Once we had a discussion because some doubling times looked like they had to be wrong. They spanned from days to years! But it turns out that doubling times are very sensitive to noise, which led to our intuitions being wrong.</p></div></li><li class="footnote-item" role="doc-endnote" id="fnnhr7mmw210m"><span class="footnote-back-link">^{<strong><a href="#fnrefnhr7mmw210m">^</a></strong>}</span><div class="footnote-content"><p>I'd also argue that <a href="https://sideways-view.com/2018/02/24/takeoff-speeds/">Christiano's operationalization of slow takeoff</a> is a terrible definition, and that a big part of that terribleness stems from doubling times being very unintuitive.</p></div></li><li class="footnote-item" role="doc-endnote" id="fnxqei73ax83"><span class="footnote-back-link">^{<strong><a href="#fnrefxqei73ax83">^</a></strong>}</span><div class="footnote-content"><p>This is because have <a href="https://fgeerolf.com/econ102/math/calculus-review2.pdf">the useful property</a> that&nbsp;<span><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="\ln(1+g) \approx g"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.372em; padding-bottom: 0.372em;">ln</span></span><span class="mjx-mo"><span class="mjx-char"></span></span><span class="mjx-mo"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.446em; padding-bottom: 0.593em;">(</span></span><span class="mjx-mn"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.372em; padding-bottom: 0.372em;">1</span></span><span class="mjx-mo MJXc-space2"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.298em; padding-bottom: 0.446em;">+</span></span><span class="mjx-mi MJXc-space2"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.519em; padding-right: 0.003em;">g</span></span><span class="mjx-mo"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.446em; padding-bottom: 0.593em;">)</span></span><span class="mjx-mo MJXc-space3"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.225em; padding-bottom: 0.298em;">≈</span></span><span class="mjx-mi MJXc-space3"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.519em; padding-right: 0.003em;">g</span></span></span></span></span></span></span>&nbsp;for percent changes&nbsp;<span><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="g"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.519em; padding-right: 0.003em;">g</span></span></span></span></span></span></span>&nbsp;close to zero, which links percent growth to a straightforward model of growth such as&nbsp;<span><span class="mjpage"><span class="mjx-chtml"><span class="mjx-math" aria-label="x_t = x_0 \exp\{gt\}"><span class="mjx-mrow" aria-hidden="true"><span class="mjx-msubsup"><span class="mjx-base"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.298em;">x</span></span></span><span class="mjx-sub" style="font-size: 70.7%; vertical-align: -0.212em; padding-right: 0.071em;"><span class="mjx-mi" style=""><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.372em; padding-bottom: 0.298em;">t</span></span></span></span><span class="mjx-mo MJXc-space3"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.077em; padding-bottom: 0.298em;">=</span></span><span class="mjx-msubsup MJXc-space3"><span class="mjx-base"><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.298em;">x</span></span></span><span class="mjx-sub" style="font-size: 70.7%; vertical-align: -0.212em; padding-right: 0.071em;"><span class="mjx-mn" style=""><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.372em; padding-bottom: 0.372em;">0</span></span></span></span><span class="mjx-mi MJXc-space1"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.151em; padding-bottom: 0.519em;">exp</span></span><span class="mjx-mo"><span class="mjx-char"></span></span><span class="mjx-mo"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.446em; padding-bottom: 0.593em;">{</span></span><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.225em; padding-bottom: 0.519em; padding-right: 0.003em;">g</span></span><span class="mjx-mi"><span class="mjx-char MJXc-TeX-math-I" style="padding-top: 0.372em; padding-bottom: 0.298em;">t</span></span><span class="mjx-mo"><span class="mjx-char MJXc-TeX-main-R" style="padding-top: 0.446em; padding-bottom: 0.593em;">}</span></span></span></span></span></span></span>. But this approximation breaks down for percent changes over 1 (which are often seen in AI forecasting).</p></div></li></ol>

EA NYC is soliciting applications for Board Members! We especially welcome applications submitted by Sunday, September 24, 2023, but rolling applications will also be considered. This is a volunteer position, but crucial in both shaping the strategy of EA NYC and ensuring our sustainability and compliance as an organization. If you have questions, Jacob Eliosoff is the primary point of contact. I think this is a great opportunity for deepened involvement and impact for a range of backgrounds!

People talk about AI resisting correction because successful goal-seekers "should" resist their goals being changed. I wonder if this also acts as an incentive for AI to attempt takeover as soon as it's powerful enough to have a chance of success, instead of (as many people fear) waiting until it's powerful enough to guarantee it. Hopefully the first AI powerful enough to potentially figure out that it wants to seize power and has a chance of succeeding is not powerful enough to passively resist value change, so acting immediately will be its only chance.

Although I'm enamoured as a mostly-neartermist that the front page (for the first time in my experience) is devoid of AI content, I really would like to hear the job experience and journey of a few AI safety/policy workers for this jobs week. The first 10ish wonderful people who shared are almost all neartermist focused, which probably doesn't represent the full experience of the community. I'm genuinely interested to understand how your AI safety job works and how you wonderful people motivate yourselves on a day to day basis, when seeing clear progress and wins must be hard a lot of the time. I find it hard enough some days working in Global health! Or maybe your work is so important, neglected and urgent that your can't spare a couple of hours to write a post ;).  

Thoughts on liability insurance for global catastrophic risks (either voluntary or mandatory) such as for biolabs or AGI companies? Do you find this to be a high-potential line of intervention?

What are examples of behaviors you engage in that you suspect are inconsistent with the values/behaviors most EAs would endorse, but that you endorse doing (i.e., because you disagree to some extent with standard EA values, or because you think that EAs draw the wrong behavioral conclusions on the basis of EA values)? Examples would (probably) not be: "I donate to political campaigns because I think this may actually be high EV" [not inconsistent with EA values] or "I eat meat but feel bad about it" [not endorsed]  Examples might be: "I donate to a local homeless shelter because it's especially important to me to support members of my community" [deviates from standard EA values] or "I eat chickens that were raised on a local farm because I think they have good lives" [different behavioral conclusions]