Conversation on AI risk with Adam Gleave

by AI Impacts3 min read27th Dec 20193 comments


AI Alignment

You can see a full transcript of this conversation on our website.



We spoke with Adam Gleave on August 27, 2019. Here is a brief summary of that conversation:

  • Gleave gives a number of reasons why it’s worth working on AI safety:
    • It seems like the AI research community currently isn’t paying enough attention to building safe, reliable systems.
    • There are several unsolved technical problems that could plausibly occur in AI systems without much advance notice.
    • A few additional people working on safety may be extremely high leverage, especially if they can push the rest of the AI research community to pay more attention to important problems.
  • Gleave thinks there’s a ~10% chance that AI safety is very hard in the way that MIRI would argue, a ~20-30% chance that AI safety will almost certainly be solved by default, and a remaining ~60-70% chance that what we’re working on actually has some impact.
    • Here are the reasons for Gleave’s beliefs, weighted by how much they factor into his holistic viewpoint:
      • 40%: The traditional arguments for risks from AI are unconvincing:
        • Traditional arguments often make an unexplained leap from having superintelligent AIs to superintelligent AIs being catastrophically bad.
        • It’s unlikely that AI systems not designed from mathematical principles are going to inherently be unsafe.
        • They’re long chains of heuristic reasoning, with little empirical validation.
        • Outside view: most fears about technology have been misplaced.
      • 20%: The AI research community will solve the AI safety problem naturally.
      • 20%: AI researchers will be more interested in AI safety when the problems are nearer.
      • 10%: The hard, MIRI version of the AI safety problem is not very compelling.
      • 10%: AI safety problems that seem hard now will be easier to solve once we have more sophisticated ML.
  • Fast takeoff defined as “GDP will double in 6 months before it doubles in 24 months” is plausible, though Gleave still leans towards slow takeoff.
  • Gleave thinks discontinuous progress in AI is extremely unlikely:
    • There is unlikely to be a sudden important insight dropped into place, since AI has empirically progressed more by accumulation of lots of bags and tricks and compute.
    • There isn’t going to be a sudden influx of compute in the near future, since well-funded organizations are currently already spending billions of dollars to optimize it.
    • If we train impressive systems, we will likely train other systems beforehand that are almost as capable.
    • Given discontinuous progress, the most likely story is that we combine many narrow AI systems in a way where the integrated whole is much better than half of them.
  • Gleave guesses a ~10-20% chance that AGI technology will only be a small difference away from current techniques, and a ~50% chance that AGI technology will be easily comprehensible to current AI researchers:
    • There are fairly serious roadblocks in current techniques right now, e.g. memory, transfer learning, Sim2Real, sample inefficiency.
    • Deep learning is slowing down compared to 2012 – 2013:
      • Much of the new progress is going to different domains, e.g. deep RL instead of supervised deep learning.
      • Computationally expensive algorithms will likely hit limits without new insights.
        • Though it seems possible that in fact progress will come from more computationally efficient algorithms.
    • Outside view, we’ve had lots of different techniques for AI over time, so it would be surprising is the current one is the right one for AGI.
    • Pushing more towards current techniques getting to AGI, from an economic point of view, there is a lot of money going into companies whose current mission is to build AGI.
  • Conditional on advanced AI technology being created, Gleave gives a 60-70% chance that it will pose a significant risk of harm without additional safety efforts.
    • Gleave thinks that best case, we drive it down to 20 – 10%, median case, we drive it down to 40 – 30%. A lot of his uncertainty comes from how difficult the problem is.
  • Gleave thinks he could see evidence that could push him in either direction in terms of how likely AI is to be safe:
    • Evidence that would cause Gleave to think AI is less likely to be safe:
      • Evidence that thorny but speculative technical problems, like inner optimizers, exist.
      • Seeing more arms race dynamics, e.g. between U.S. and China.
      • Seeing major catastrophes involving AI, though they would also cause people to pay more attention to risks from AI.
      • Hearing more solid arguments for AI risk.
    • Evidence that would cause Gleave to think AI is more likely to be safe:
      • Seeing AI researchers spontaneously focus on relevant problems would make Gleave think that AI is less risky.
      • Getting evidence that AGI was going to take longer to develop.
  • Gleave is concerned that he doesn’t understand why members of the safety community come to widely different conclusions when it comes to AI safety.
  • Gleave thinks a potentially important question is the extent to which we can successfully influence field building within AI safety.
AI Alignment2