I'm thinking about the matching problem of "people with AI safety questions" and
"people with AI safety answers". Snoop Dogg hears Geoff Hinton on CNN (or
wherever), asks "what the fuck?"
[https://twitter.com/pkedrosky/status/1653955254181068801], and then tries to
find someone who can tell him what the fuck.
I think normally people trust their local expertise landscape--if they think the
CDC is the authority on masks they adopt the CDC's position, if they think their
mom group on Facebook is the authority on masks they adopt the mom group's
position--but AI risk is weird because it's mostly unclaimed territory in their
local expertise landscape. (Snoop also asks "is we in a movie right now?"
because movies are basically the only part of the local expertise landscape that
has had any opinion on AI so far, for lots of people.) So maybe there's an
opportunity here to claim that territory (after all, we've thought about it a
lot!).
I think we have some 'top experts' who are available for, like, mass-media
things (podcasts, blog posts, etc.) and 1-1 conversations with people they're
excited to talk to, but are otherwise busy / not interested in fielding ten
thousand interview requests. Then I think we have tens (hundreds?) of people who
are expert enough to field ten thousand interview requests, given that the
standard is "better opinions than whoever they would talk to by default" instead
of "speaking to the whole world" or w/e. But just like connecting people who
want to pay to learn calculus and people who know calculus and will teach it for
money, there's significant gains from trade from having some sort of
clearinghouse / place where people can easily meet. Does this already exist? Is
anyone trying to make it? (Do you want to make it and need support of some
sort?)
Why aren't we engaging in direct action (including civil disobedience) to pause
AI development?
Here's the problem:
Yudkowksy
[https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/]: "Many
researchers steeped in these issues, including myself, expect that the most
likely result of building a superhumanly smart AI, under anything remotely like
the current circumstances, is that literally everyone on Earth will die."
Here's one solution:
FLI Open Letter
[https://futureoflife.org/open-letter/pause-giant-ai-experiments/]: "all AI
labs...immediately pause for at least 6 months the training of AI systems more
powerful than GPT-4. This pause should be public and verifiable, and include all
key actors. If such a pause cannot be enacted quickly, governments should step
in and institute a moratorium."
Here's what direct action in the pursuit of that solution could look like (most
examples are from the UK climate movement):
Picketing AI offices
[https://twitter.com/Radlib4/status/1653135998501662722?s=20] (this already
seems to be happening!)
Mass non-disruptive protest
[https://www.theguardian.com/world/2023/apr/21/big-one-extinction-rebellion-cliimate-protest-london-xr]
Strikes/walk-outs
[https://www.theguardian.com/science/2021/sep/24/people-in-99-countries-take-part-in-global-climate-strike]
(by AI developers/researchers/academics)
Slow marches
[https://www.itv.com/news/border/2023-04-29/just-stop-oil-protestors-stage-slow-march-through-town-centre]
Roadblocks [https://www.bbc.co.uk/news/uk-england-london-59061509]
Occupation
[https://www.theguardian.com/uk/2012/jan/16/belfast-occupy-bank-of-ireland] of
AI offices
Performative vandalism
[https://www.bbc.co.uk/news/uk-england-gloucestershire-64193016] of AI offices
Performative vandalism of art
[https://www.theguardian.com/environment/2022/oct/14/just-stop-oil-activists-throw-soup-at-van-goghs-sunflowers]
Sabotage of AI computing infrastructure (on the model of ecotage
[https://www.theguardian.
Together with a few volunteers, we prepared a policy document for the Campaign
for AI Safety to serve as a list of demands by the campaign.
It is called "Strong and appropriate regulation of advanced AI to protect
humanity [https://campaignforaisafety.org/policy-recommendations/]". It is
currently geared towards Australiand and US policy-makers, and I think it's not
its last version.
I would appreciate any comments!
This post is half object level, half experiment with “semicoherent audio
monologue ramble → prose” AI (presumably GPT-3.5/4 based) program audiopen.ai
[http://audiopen.ai].
In the interest of the latter objective, I’m including 3 mostly-redundant
subsections:
1. A ’final’ mostly-AI written text, edited and slightly expanded just enough
so that I endorse it in full (though recognize it’s not amazing or close to
optimal)
2. The raw AI output
3. The raw transcript
1) DUBIOUS ASYMMETRY ARGUMENT IN WWOTF
In Chapter 9 of his book, What We Are the Future, Will MacAskill argues that the
future holds positive moral value under a total utilitarian perspective. He
posits that people generally use resources to achieve what they want - either
for themselves or for others - and thus good outcomes are easily explained as
the natural consequence of agents deploying resources for their goals.
Conversely, bad outcomes tend to be side effects of pursuing other goals. While
malevolence and sociopathy do exist, they are empirically rare.
MacAskill argues that in a future with continued economic growth and no
existential risk, we will likely direct more resources towards doing good things
due to self-interest and increased impartial altruism. He contrasts this
eutopian scenario with an anti-eutopia: the worst possible world, which he
argues (compellingly, I think) less probable because it requires convoluted
explanations as opposed to simple desires like enjoying ice cream. He concludes
that the probability of achieving a eutopia outweighs the low likelihood but
extreme negative consequences of an anti-eutopia.
However, I believe McCaskill's analysis neglects an important aspect:
considering not only these two extremes but also the middle distribution where
neither significant amounts of resources nor agentic intervention occur.
When physics operates without agency-driven resource allocation, we have good
reason to expect evolution to create conscious beings whos
Reflection on a university discussion about the future and safety of AI:
Background: ~20 people, 2 assistant professors, other BSc and MSc level students
of AI/ML at Charles University
One of the professors sent us the FLI letter
[https://futureoflife.org/open-letter/pause-giant-ai-experiments/], Petition by
LAION
[https://www.openpetition.eu/petition/online/securing-our-digital-future-a-cern-for-open-source-large-scale-ai-research-and-its-safety]
(in my view accelerationist and insufficiently addressing issues raised by the
AI x-risk community), and On the Dangers of Stochastic Parrots by Bender et al
[https://dl.acm.org/doi/pdf/10.1145/3442188.3445922].. Supplemental materials
were criticisms of longtermism (by Torres and Thorn).
First we gave reasons for and against each petition. The professor in my view
framed it that FLI is a suspicious organization (because of longtermist ties),
x-risk issues are on the same level or less important than issues raised in
Stochastic Parrots (bias, environmental concerns), and that academics support
LAION more.
* open source development proposed in LAION resonated with the students as an
obviously good thing (at the start 16 said they would be in favor)
* only 2 people (including me) would be in favor of FLI letter
In spite of the framing, after a 1 hour discussion, more than a half of the
students were convinced that AI x-risk is a serious issue and rejected issues
from stochastic parrots being as important.
No one changed their mind about LAION.
One student (he was in a group discussion with me) changed his mind to be in
favor of the FLI letter.
I showed the professors the AI impacts survey
[https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2022_expert_survey_on_progress_in_ai]
and they found it interesting.
Some people seemed interested in the Lex Fridman with Eliezer Yudkowsky
[https://www.youtube.com/watch?v=AaTRHFaaPG8] podcast.
Who else thinks we should be aiming for a global moratorium on AGI research at
at
[https://forum.effectivealtruism.org/posts/3vDarp6adLPBTux5g/what-a-compute-centric-framework-says-about-ai-takeoff?commentId=ZPmPeZeMmTfEa8LEZ]
this [https://www.youtube.com/watch?v=gA1sNLL6yg4&ab_channel=BanklessShows]
point
[https://forum.effectivealtruism.org/posts/eAaeeuEd4j6oJ3Ep5/gpt-4-is-out-thread-and-links]?
I'm considering ending every comment I make with "AGI research cessandum est",
or "Furthermore, AGI research must be stopped"
[https://en.wikipedia.org/wiki/Carthago_delenda_est].
I often see people talking past each other when discussing x-risks because the
definition[1] covers outcomes that are distinct in some worldviews. For some,
humanity failing to reach its full potential and humanity going extinct are
joint concerns, but for others they are separate outcomes. Is there a good
solution to this?
1. ^
"An existential risk is one that threatens the premature extinction of
Earth-originating intelligent life or the permanent and drastic destruction
of its potential for desirable future development." (source
[https://existential-risk.org/])
Would an AI governance book that covered the present landscape of gov-related
topics (maybe like a book version of the FHI's AI Governance Research Agenda
[https://www.fhi.ox.ac.uk/wp-content/uploads/GovAI-Agenda.pdf]?) be useful?
We're currently at a weird point where there's a lot of interest in AI - news
coverage, investment, etc. It feels weird to not be trying to shape the
conversation on AI risk more than we are now. I'm well aware that this sort of
thing can backfire, and I'm aware that most people are highly sceptical of
trying not to "politicise" issues like these, but it might be a good idea.
If it was written by, say, Toby Ord - or anyone sufficiently detached from
American left/right politics, with enough prestige, background, and experience
with writing books like these - I feel like it might be really valuable.
It might also be more approachable than other books covering AI risk, like, say,
Superintelligence. It might also seem a little more concrete, because it might
cover scenarios that are easier for most people to imagine/scenarios that are
more near-term, and less "sci-fi".
Thoughts on this?