Hide table of contents

This is the current Metaculus forecast for AGI:

 

Do decisions by EAs working in other cause areas mostly ignore this?

 

For example, are timelines for transformative AGI taken into account in work such as:

  1. Deciding whether to donate money later vs now
  2. Estimating existential risk from climate change, nuclear war and engineered pandemics past 2043 / 2068 / 2199
  3. Deciding whether to pursue "move fast, break things" approaches vs lower downside risk, slower,  reform approaches

 

And additionally, should timelines for transformative AI be taken into account in this work?

20

0
0

Reactions

0
0
New Answer
New Comment


6 Answers sorted by

In my day-to-day work for AMF[1], AGI timelines don't matter. My work is about making bednet distributions more efficient and transparent, and AGIs simply don't help with that yet.

Of course, I'm open to suggestions regarding how my work should be influenced by AGI timelines (apart from the obvious "stop it and work on AI safety instead").

There are other technological developments that I think will affect AMF's work, for example vaccines, gene drives, and new types of insecticides. We follow these closely. AGI itself will certainly affect bednet distributions too, once it arrives. Until then, the right thing to do IMO is to continue working hard to help as many people as possible in their fight against malaria today.


    1. I'm speaking for myself here and not on behalf of AMF. ↩︎

I recently wrote about how  AGI timelines change the relative value of 'slow' acting neartermist interventions relative to 'fast' acting neartermist interventions.

It seems to me that EAs in other cause areas mostly ignore this, though I haven't looked into this too hard. 

My (very rough) understanding of Open Philanthropy's worldview diversification approach is that  the Global Health and Wellbeing focus area team operates on both (potentially) different values and  epistemic approaches to the Longtermism focus area team. The epistemic approach of the former seems more reliant on more "common sense" ways to do good.

Judging by the ratio, I'd like to clarify that I'm not disagreeing at all.

I'm saying that

  1. For the entire 1950-1990 period, if we follow present-day trends of AI risk gaining prominence within EA relative to global health, the nuclear risk faction could have made a credible case that it deserved the most priority by far. Humanity was just incredibly close to nuclear war at multiple points.
  2. However, crowding out other causes meant that things like global health would have been neglected for decades, even though the cause landscape remained roughly the same since no nuclear war occurred. There are projects like the Global Seed Vault that do combine ideas within nuclear risk and global health, but the actual neartermist global health faction would overall be drowned out.
  3. Even if EA research had concluded a lack of tractability, that doesn't mean they can steer discourse and resources back towards other causes that are tractable. If nuclear war was a cause area that received way more attention and funding in the movement, it will induce bias.

I'm not opposing investigations of inter-causal impacts, I'm cautioning that such a focus might overestimate the X-Risk impacts, because people catastrophise on highly uncertain scary risks and develop tunnel vision. You could make the opposite case, too. I think most EAs heavily underestimated COVID's impact on other cause areas, considering COVID was never an existential-level pandemic threat.

It's definitely a blindspot, since I generally don't see overlap between highly engaged EAs doing both AI risk work and global health work/animal welfare work. At this early stage of the movement, I'm seeing most of work done by early-mid career professionals who have prioritised building their expertise in applying EA to one field to yield exponential impact.

That said, some caveats:

  • "What AGI does" is not common consensus to begin with. Yes, there are compelling and influential theories, but an actionable, quantitative prediction of exactly how AGI will affect other causes is quite far off.
  • Existing AI risk organisations are mostly focusing on the technical details of AI, which reflects in their hiring. To actionably assess socioeconomic impacts would require hiring huge and well-resourced teams of economists, doctors, policy analysts that are specialised in both AI and policy topics. I'm not aware of a major hiring push in this direction, at least in EA. Some governments like Singapore and China are actively studying the societal impacts of AI.
  • I sometimes tell fellow EAs that cause tunnel vision can hinder progress. If the EA movement had been around in 1950, it would have spent 40 years dominated by the "nuclear war risk" faction, with dubious impact at the expense of research on other causes. After all, why would developing world poverty matter in a nuclear winter, which has such a high probability of occurring?

However, I get what you're saying. Overall, I do think cross-cause collaboration would yield some benefit, even if just in a speculative wargaming capacity. COVID-19 has showed that one existential risk coming to fruition, even on a limited scale, can greatly alter trajectories elsewhere.

The answer is implicitly yes, they do ignore AGI timelines because dealing with the Singularity is both model-wrecking, and crucially, may not happen.

And it's a good thing that they ignore AGI timelines, as they are hedge organizations in case the Singularity or X-risk scenario does not in fact occur, and they need to optimize for a world in which boring normal futures happen.

My impression is that AGI timelines are largely ignored in the substance of the work itself, in part because it is hard to know how exactly to incorporate these estimates. I work in EA community building and even in this role it is unclear how I should factor these estimates into my decision-making. I know what questions it raises and I know how the answers to those questions would inform my strategy, but the questions remain hard to answer based on the estimates alone. I would like to see much more discussion of the practical implications of AGI timelines.

Curated and popular this week
 ·  · 4m read
 · 
Forethought[1] is a new AI macrostrategy research group cofounded by Max Dalton, Will MacAskill, Tom Davidson, and Amrit Sidhu-Brar. We are trying to figure out how to navigate the (potentially rapid) transition to a world with superintelligent AI systems. We aim to tackle the most important questions we can find, unrestricted by the current Overton window. More details on our website. Why we exist We think that AGI might come soon (say, modal timelines to mostly-automated AI R&D in the next 2-8 years), and might significantly accelerate technological progress, leading to many different challenges. We don’t yet have a good understanding of what this change might look like or how to navigate it. Society is not prepared. Moreover, we want the world to not just avoid catastrophe: we want to reach a really great future. We think about what this might be like (incorporating moral uncertainty), and what we can do, now, to build towards a good future. Like all projects, this started out with a plethora of Google docs. We ran a series of seminars to explore the ideas further, and that cascaded into an organization. This area of work feels to us like the early days of EA: we’re exploring unusual, neglected ideas, and finding research progress surprisingly tractable. And while we start out with (literally) galaxy-brained schemes, they often ground out into fairly specific and concrete ideas about what should happen next. Of course, we’re bringing principles like scope sensitivity, impartiality, etc to our thinking, and we think that these issues urgently need more morally dedicated and thoughtful people working on them. Research Research agendas We are currently pursuing the following perspectives: * Preparing for the intelligence explosion: If AI drives explosive growth there will be an enormous number of challenges we have to face. In addition to misalignment risk and biorisk, this potentially includes: how to govern the development of new weapons of mass destr
jackva
 ·  · 3m read
 · 
 [Edits on March 10th for clarity, two sub-sections added] Watching what is happening in the world -- with lots of renegotiation of institutional norms within Western democracies and a parallel fracturing of the post-WW2 institutional order -- I do think we, as a community, should more seriously question our priors on the relative value of surgical/targeted and broad system-level interventions. Speaking somewhat roughly, with EA as a movement coming of age in an era where democratic institutions and the rule-based international order were not fundamentally questioned, it seems easy to underestimate how much the world is currently changing and how much riskier a world of stronger institutional and democratic backsliding and weakened international norms might be. Of course, working on these issues might be intractable and possibly there's nothing highly effective for EAs to do on the margin given much attention to these issues from society at large. So, I am not here to confidently state we should be working on these issues more. But I do think in a situation of more downside risk with regards to broad system-level changes and significantly more fluidity, it seems at least worth rigorously asking whether we should shift more attention to work that is less surgical (working on specific risks) and more systemic (working on institutional quality, indirect risk factors, etc.). While there have been many posts along those lines over the past months and there are of course some EA organizations working on these issues, it stil appears like a niche focus in the community and none of the major EA and EA-adjacent orgs (including the one I work for, though I am writing this in a personal capacity) seem to have taken it up as a serious focus and I worry it might be due to baked-in assumptions about the relative value of such work that are outdated in a time where the importance of systemic work has changed in the face of greater threat and fluidity. When the world seems to
Sam Anschell
 ·  · 6m read
 · 
*Disclaimer* I am writing this post in a personal capacity; the opinions I express are my own and do not represent my employer. I think that more people and orgs (especially nonprofits) should consider negotiating the cost of sizable expenses. In my experience, there is usually nothing to lose by respectfully asking to pay less, and doing so can sometimes save thousands or tens of thousands of dollars per hour. This is because negotiating doesn’t take very much time[1], savings can persist across multiple years, and counterparties can be surprisingly generous with discounts. Here are a few examples of expenses that may be negotiable: For organizations * Software or news subscriptions * Of 35 corporate software and news providers I’ve negotiated with, 30 have been willing to provide discounts. These discounts range from 10% to 80%, with an average of around 40%. * Leases * A friend was able to negotiate a 22% reduction in the price per square foot on a corporate lease and secured a couple months of free rent. This led to >$480,000 in savings for their nonprofit. Other negotiable parameters include: * Square footage counted towards rent costs * Lease length * A tenant improvement allowance * Certain physical goods (e.g., smart TVs) * Buying in bulk can be a great lever for negotiating smaller items like covid tests, and can reduce costs by 50% or more. * Event/retreat venues (both venue price and smaller items like food and AV) * Hotel blocks * A quick email with the rates of comparable but more affordable hotel blocks can often save ~10%. * Professional service contracts with large for-profit firms (e.g., IT contracts, office internet coverage) * Insurance premiums (though I am less confident that this is negotiable) For many products and services, a nonprofit can qualify for a discount simply by providing their IRS determination letter or getting verified on platforms like TechSoup. In my experience, most vendors and companies