In Twitter and elsewhere, I've seen a bunch of people argue that AI company
execs and academics are only talking about AI existential risk because they want
to manufacture concern to increase investments and/or as a distraction away from
near-term risks and/or regulatory capture. This is obviously false.
However, there is a nearby argument that is likely true: which is that
incentives drive how people talk about AI risk, as well as which specific
regulations or interventions they ask for. This is likely to happen both
explicitly and unconsciously. It's important (as always) to have extremely solid
epistemics, and understand that even apparent allies may have (large) degrees of
self-interest and motivated reasoning.
Safety-washing
[https://forum.effectivealtruism.org/posts/f2qojPr8NaMPo2KJC/beware-safety-washing]
is a significant concern; similar things have happened a bunch in other fields,
it likely has already happened a bunch in AI, and will likely happen again in
the months and years to come, especially if/as policymakers and/or the general
public become increasingly uneasy about AI.
Protesting at leading AI labs may be significantly more effective than most
protests, even ignoring the object-level arguments for the importance of AI
safety as a cause area. The impact per protester is likely unusually big, since
early protests involve only a handful of people and impact probably scales
sublinearly with size. And very early protests are unprecedented and hence more
likely (for their size) to attract attention, shape future protests, and have
other effects that boost their impact.
Quick updates:
* Our next critique (on Conjecture) will be published in 2 weeks.
* The critqiue after that will be on Anthropic. If you'd like to be a reviewer,
or have critiques you'd like to share, please message us or email
anonymouseaomega@gmail.com [anonymouseaomega@gmail.com].
TL;DR: Someone should probably write a grant to produce a spreadsheet/dataset of
past instances where people claimed a new technology would lead to societal
catastrophe, with variables such as “multiple people working on the tech
believed it was dangerous.”
Slightly longer TL;DR: Some AI risk skeptics are mocking people who believe AI
could threaten humanity’s existence, saying that many people in the past
predicted doom from some new tech. There is seemingly no dataset which lists and
evaluates such past instances of “tech doomers.” It seems somewhat ridiculous*
to me that nobody has grant-funded a researcher to put together a dataset with
variables such as “multiple people working on the technology thought it could be
very bad for society.”
*Low confidence: could totally change my mind
———
I have asked multiple people in the AI safety space if they were aware of any
kind of "dataset for past predictions of doom (from new technology)"? There have
been some articles and arguments floating around recently such as "Tech Panics,
Generative AI, and the Need for Regulatory Caution
[https://datainnovation.org/2023/05/tech-panics-generative-ai-and-regulatory-caution/]",
in which skeptics say we shouldn't worry about AI x-risk because there are many
past cases where people in society made overblown claims that some new
technology (e.g., bicycles, electricity) would be disastrous for society.
While I think it's right to consider the "outside view" on these kinds of
things, I think that most of these claims 1) ignore examples of where there were
legitimate reasons to fear the technology (e.g., nuclear weapons, maybe
synthetic biology?), and 2) imply the current worries about AI are about as
baseless as claims like "electricity will destroy society," whereas I would
argue that the claim "AI x-risk is >1%" stands up quite well against most
current scrutiny.
(These claims also ignore the anthropic argument/survivor bias—that if they ever
were right about doom we wouldn't
I thought the recent Hear This Idea podcast episode with Ben Garfinkel
[https://hearthisidea.com/episodes/garfinkel] was excellent. If you are at all
interested in AI governance (or AI safety generally), you probably want to check
it out.
Scattered and rambly note I jotted down in a slack in February 2023, and didn't
really follow up on
--------------------------------------------------------------------------------
thinking of jotting down some notes about "what AI pessimism funding ought to
be", that takes into account forecasting and values disagreements.The premises:
* threatmodels drive research. This is true on lesswrong when everyone knows it
and agonizes over "am I splitting my time between hard math/cs and
forecasting or thinking about theories of change correctly?" and it's true in
academia when people halfass a "practical applications" paragraph in their
paper.
* people who don't really buy into the threatmodel they're ostensibly working
on do research poorly
* social pressures like funding and status make it hard to be honest about what
threatmodels motivate you.
* I don't overrate democracy or fairness as terminal values, I'm bullish on a
lot of deference and technocracy (whatever that means), but I may be feeling
some virtue-ethicsy attraction toward "people feeling basically represented
by governance bodies that represent them", that I think is tactically useful
for researchers because the above point about research outputs being more
useful when the motivation is clearheaded and honest.
* fact-value orthogonality, additionally the binary is good and we don't need a
secret third thing if we confront uncertainty well enough
The problems I want to solve:
* thinking about inclusion and exclusion (into "colleagueness" or stuff that
funder's care about like "who do I fund") is fogged by tribal conflict where
people pathologize eachother (salient in "AI ethics vs. AI alignment".
twitter is the mindkiller but occasionally I'll visit, and I always feel like
it makes me think less clearly)
* no actual set of standards for disagreement to take place in, instead we have
wishy washy stuff like "the purple hats undervalue standpoint
I'm thinking about the matching problem of "people with AI safety questions" and
"people with AI safety answers". Snoop Dogg hears Geoff Hinton on CNN (or
wherever), asks "what the fuck?"
[https://twitter.com/pkedrosky/status/1653955254181068801], and then tries to
find someone who can tell him what the fuck.
I think normally people trust their local expertise landscape--if they think the
CDC is the authority on masks they adopt the CDC's position, if they think their
mom group on Facebook is the authority on masks they adopt the mom group's
position--but AI risk is weird because it's mostly unclaimed territory in their
local expertise landscape. (Snoop also asks "is we in a movie right now?"
because movies are basically the only part of the local expertise landscape that
has had any opinion on AI so far, for lots of people.) So maybe there's an
opportunity here to claim that territory (after all, we've thought about it a
lot!).
I think we have some 'top experts' who are available for, like, mass-media
things (podcasts, blog posts, etc.) and 1-1 conversations with people they're
excited to talk to, but are otherwise busy / not interested in fielding ten
thousand interview requests. Then I think we have tens (hundreds?) of people who
are expert enough to field ten thousand interview requests, given that the
standard is "better opinions than whoever they would talk to by default" instead
of "speaking to the whole world" or w/e. But just like connecting people who
want to pay to learn calculus and people who know calculus and will teach it for
money, there's significant gains from trade from having some sort of
clearinghouse / place where people can easily meet. Does this already exist? Is
anyone trying to make it? (Do you want to make it and need support of some
sort?)
Hey - I’d be really keen to hear peoples' thoughts on the following
career/education decision I'm considering (esp. people who think about AI a
lot):
* I’m about to start my undergrad studying PPE at Oxford.
* I’m wondering whether re-applying this year to study CS & philosophy at
Oxford (while doing my PPE degree) is a good idea.
* This doesn’t mean I have to quit PPE or anything.
* I’d also have to start CS & philosophy from scratch the following year.
* My current thinking is that I shouldn’t do this - I think it’s unlikely that
I’ll be sufficiently good to, say, get into a top 10 ML PhD or anything, so
the technical knowledge that I’d need for the AI-related paths I’m
considering (policy, research, journalism, maybe software engineering) is
either pretty limited (the first three options) or much easier to self-teach
and less reliant on credentials (software engineering).
* I should also add that I’m currently okay at programming anyway, and plan
to develop this alongside my degree regardless of what I do - it seems like
a broadly useful skill that’ll also give me more optionality.
* I do have a suspicion that I’m being self-limiting re the PhD thing - if
everyone else is starting from a (relatively) blank slate, maybe I’d be on
equal footing?
* That said, I also have my suspicions that the PhD route is actually my
highest-impact option: I’m stuck between 1) deferring to 80K here, and 2)
my other feeling that enacting policy/doing policy research might be
higher-impact/more tractable.
* They’re also obviously super competitive, and seem to only be getting
more so.
* One major uncertainty I have is whether, for things like policy, a PPE degree
(or anything politics-y/economics-y) really matters. I’m a UK citizen, and
given the record of UK politicians who did PPE at Oxford, it seems like it
might?
What mistakes am I making here/am I being too self-limiting? I s