Yeah, Matthews really should have replaced "CS" with "math" or "math and philosophy".
That would be more accurate, more consistent with AI safety researchers' self-conception, and less susceptible to some of these counterarguments (especially Julia's point that in CS, the money and other external validation is much more in for-profits than AI safety).
Perhaps a more charitable framing of Dylan's argument is not that computer scientists think AI is a problem but that they think the solution is to do more CS research -- rather than, e.g., to do political lobbying, changing society's general approach to AI, etc.
Of course, some replies include that (1) some people have done activities outside the realm of CS research, (2) it's often good to focus on the area you know, and (3) CS research may be particularly relevant to solving this problem, compared with other issues that are primarily political.
This is a DH1 argument on the part of Matthews:
...if a senator wrote an article saying senators' salaries should be increased, one could respond:
"Of course he would say that. He's a senator."
This wouldn't refute the author's argument, but it may at least be relevant to the case. It's still a very weak form of disagreement, though. If there's something wrong with the senator's argument, you should say what it is; and if there isn't, what difference does it make that he's a senator?
Also, it seems likely to me that publicizing concern with AI risk works against the interests of many in the tech community.
If Matthews wants to engage with this further, I think it's only fair to the people he's criticizing to take the time to read Superintelligence so he can understand the position he's arguing against.
http://www.vox.com/2014/8/19/6031367/oxford-nick-bostrom-artificial-intelligence-superintelligence
Dylan Matthews has read Superintelligence and wrote an article+interview with Bostrom about it in August 2014.
Thanks for the link, I wasn't aware of that. Still disappointed to see an ad hominem argument. It's true that Matthews only included it as a throwaway sentence, but this is the kind of characterization that readers can latch on to... feels like within-EA movement disagreements should be kept above the ad hominem level. (Also in favor of engaging opponents in a classy way, but let's walk before we run.)
When experts say the fate of the world lies in the hands of their field...it's not an indication they're wrong. It's just not really strong evidence they're right either. In any field, we can reasonably expect researchers to believe their field has great significance. From the outside, they sort of all sound the same. When I read this quip, that's what is think of.
Unfortunately, with nuclear weapons, it was virtually impossible to convince officials of their feasibility right up until the months leading to Hiroshima, and then, really only with demos, not expert opinions. It's a really tough sell. And if it's not being explained and argued extremely well, and even then, I'll be unlikely to be moved by just about anything, because it sounds very similar to all the times people have tried to convince me something they're really into has all the answers.
A few months ago effective altruism-sympathetic journalist Dylan Matthews wrote this about enthusiasm for AI safety research in Vox:
"At the risk of overgeneralizing, the computer science majors have convinced each other that the best way to save the world is to do computer science research."
I have seen this claim made in various forms several times. While I understand why Dylan Matthews could form this view at a distance from the people involved, I think it is an inaccurate description.
Reason one
The early adopters of the view were more mathematicians, philosophers or interdisciplinary researchers. I personally first became worried about artificial intelligence when the community was much smaller, and was studying genetics and economics. Many computer scientists have now come around, but were if anything relatively resistant compared to people in related fields.
Reason two
More importantly, belief that artificial intelligence presents a big risk is more likely to lead you to despair than a rosy outlook in which you can be a hero. In the early days, most people who were worried about AI felt completely disempowered because i) there was no clear path to a solution, and indeed one may not exist at all, ii) even if a solution exists, most people who were concerned thought they were not personally qualified to do any of the relevant work to solve the problem, including me.
The thought that superintelligent machines may destroy everything you care about, and there may well be nothing you can do about it, is hardly the most appealing belief. Rather than walking into a phone booth and putting on a superhero outfit, many people who read these arguments became anxious and despondent. That remains the case today.
But it is a belief that nevertheless spread, in my view because the underlying arguments, best laid out in the book Superintelligence, are remarkably hard to convincingly rebut.
Reason three
Almost everyone I know who is now especially worried about artificial intelligence initially thought they could use their skills well in another cause areas such as reducing poverty or animal suffering in factory farms. They didn't need to change the cause area they worked on to feel like they could greatly improve the world.
Even if the facts were true the underlying argument can't work
Finally, even if it the claim were true, I don't think it could be a coherent argument against worrying about AI risk.
The reason is this.
Imagine the reverse were the case and it wasn't the domain experts most qualified to work on the problem who thought it was a big deal. Instead it was chefs, musicians and magazine copy editors who were most concerned. Would that be an argument in favour of worrying more, because such non-field experts could clearly have no self-serving bias that would cause them to worry?
I don't think so. The concern of non-field experts is clearly less persuasive and I expect almost everyone would agree. Indeed, many people have claimed in the past that the fact that many computer scientists didn't seem so concerned was a good reason not to be worried.
But if you want to say it's both unconvincing when field experts like computer scientists are concerned, and also unconvincing when non-field experts like farmers are concerned, congratulations: you've just made your view unresponsive to anyone else's judgement. That's a very bad place to be.
In fact, computer scientists being concerned about artificial intelligence - and other relevant domain experts such as philosophers, mathematicians, brain scientists and machine learning experts - is, on balance, the strongest piece of evidence available, because they are most likely to know what they are talking about.
Finally, how could the situation ever be otherwise? It's natural that the first people to notice a new potential problem from a technology are domain experts, or at least people in adjacent domains. I bet the first people who sounded the alarm about potential threats from nuclear weapons were people who were close to relevant physics research. No one else could have been aware of the issue, or able to evaluate the merits of the arguments. I don't think the fact that physicists were initially the main group worried about the power of nuclear weapons would be a good reason to doubt them.
Jeff pointed out at the time that that the money in CS is in for-profits, not AI research. So CS majors do much better in terms of profit and glory to believe that earning to give, not research, is the best way to save the world. Which opens up a different line of possible criticism. :-)