[[THIRD EDIT: Thanks so much for all of the questions and comments! There are still a few more I'd like to respond to, so I may circle back to them a bit later, but, due to time constraints, I'm otherwise finished up for now. Any further comments or replies to anything I've written are also still be appreciated!]]
Hi!
I'm Ben Garfinkel, a researcher at the Future of Humanity Institute. I've worked on a mixture of topics in AI governance and in the somewhat nebulous area FHI calls "macrostrategy", including: the long-termist case for prioritizing work on AI, plausible near-term security issues associated with AI, surveillance and privacy issues, the balance between offense and defense, and the obvious impossibility of building machines that are larger than humans.
80,000 Hours recently released a long interview I recorded with Howie Lempel, about a year ago, where we walked through various long-termist arguments for prioritizing work on AI safety and AI governance relative to other cause areas. The longest and probably most interesting stretch explains why I no longer find the central argument in Superintelligence, and in related writing, very compelling. At the same time, I do continue to regard AI safety and AI governance as high-priority research areas.
(These two slide decks, which were linked in the show notes, give more condensed versions of my views: "Potential Existential Risks from Artificial Intelligence" and "Unpacking Classic Arguments for AI Risk." This piece of draft writing instead gives a less condensed version of my views on classic "fast takeoff" arguments.)
Although I'm most interested in questions related to AI risk and cause prioritization, feel free to ask me anything. I'm likely to eventually answer most questions that people post this week, on an as-yet-unspecified schedule. You should also feel free just to use this post as a place to talk about the podcast episode: there was a thread a few days ago suggesting this might be useful.
FWIW, it mostly doesn't resonate with me. (Of course, my experience is no more representative than yours.) Just as you I'd be curious to hear from more people.
I think what matches my impression most is that:
On the other points, my impression is that if there were consistent and significant changes in views they must have happened mostly among people I rarely interact with personally, or more than 3 years ago.
Most people I can think of who in 2017 had any at least minimally considered view on questions such as probability of doom, takeoff speed, polarity, timelines, and which AI safety agendas are promising still hold roughly the same view as far as I can tell. E.g. I recall one influential AI safety researcher who in summer 2017 gave what I thought were extremely short timelines, and in 2018 they told me they had become even shorter. I also don't think I have changed my views significantly - they do feel more nuanced, but my bottom line on e.g. timelines or probability of different scenarios hasn't changed significantly as far as I can remember.
My impression is that there hasn't so much been a shift in views within individual people than the influx of a younger generation who tends to have an ML background and roughly speaking tends to agree more with Paul Christiano than MIRI. Some of them are now somewhat prominent themselves (e.g. Rohin Shah, Adam Gleave, you), and so the distribution of views among the set of perceived "AI risk thought leaders" has changed. But arguably this is a largely sociological phenomenon (e.g. due to prominent ML successes there are just way more people with ML background in general). [ETA: As Rohin notes, neither he nor Paul or Adam had an ML background when they decided which kind of AI safety research to focus on - instead, they switched to ML because they thought that was the more promising approach. So the suggested sociological explanation fails in at least their cases.]
More broadly, my impression is that for years there have been intractable disagreements on several fundamental questions regarding AI risk, that there hasn't been much progress on resolving them, that few people have changed their mind in major ways, and that sometimes people holding different views have mostly stopped talking to each other. E.g. I've for months shared an office with people who hold views which I think are really off but have never talked to them about it, and more broadly I think we both know that even within just FHI there is an arguably extreme spread of views on issues pertaining to AI risk and longtermism/macrostrategy more generally.
(NB I don't think this is necessarily bad. When disagreements prove intractable, it might be best if different groups make different bets and pursue their agendas separately. It might also not be that unusual for cases without decisive uncontroversial evidence, e.g. I'm sure there are protracted and intractable disagreements between, say, Keynesian and neoclassical economists or proponents of different quantum gravity theories.)
At the other extreme, I've seen dozens of collective person-hours being invested into experimenting with social technologies (e.g. certain ways of "facilitating" conversations) that were supposed to help people with different views understand each other, and to transmit some of that understanding to an audience of spectators. (I thought these were poorly executed and largely failures, but other thoughtful people seemed to disagree and expressed an eagerness to invest much more time into similar activities.)
I do recall instances of what I thought constituted exaggerated epistemic deference, especially in 2016 and to some extent 2017. Some of them were I think quite bizarre, with people essentially engaging in a long exegesis of brief, cryptic remarks that someone they know had relayed as something someone they know had heard as attributed to some presumed epistemic authority. Sometimes it wasn't even clear who the supposed source of some information was, e.g. I recall a period where people around me were fuzzed that "people at OpenAI had short timelines", with both the identities of these people and the question of just how short their timelines were being unclear. Usually I think it would have been more productive for the participants (myself included) to take an online course in ML, to google for some relevant factual information, or to try to make their thoughts more precise by writing them down.
(Again, some amount of epistemic deference is of course healthy. And more specifically it does seem correct to give more weight to people who have more relevant expertise or experience.)