Shortforms

I'm thinking about the matching problem of "people with AI safety questions" and "people with AI safety answers". Snoop Dogg hears Geoff Hinton on CNN (or wherever), asks "what the fuck?" [https://twitter.com/pkedrosky/status/1653955254181068801], and then tries to find someone who can tell him what the fuck. I think normally people trust their local expertise landscape--if they think the CDC is the authority on masks they adopt the CDC's position, if they think their mom group on Facebook is the authority on masks they adopt the mom group's position--but AI risk is weird because it's mostly unclaimed territory in their local expertise landscape. (Snoop also asks "is we in a movie right now?" because movies are basically the only part of the local expertise landscape that has had any opinion on AI so far, for lots of people.) So maybe there's an opportunity here to claim that territory (after all, we've thought about it a lot!). I think we have some 'top experts' who are available for, like, mass-media things (podcasts, blog posts, etc.) and 1-1 conversations with people they're excited to talk to, but are otherwise busy / not interested in fielding ten thousand interview requests. Then I think we have tens (hundreds?) of people who are expert enough to field ten thousand interview requests, given that the standard is "better opinions than whoever they would talk to by default" instead of "speaking to the whole world" or w/e. But just like connecting people who want to pay to learn calculus and people who know calculus and will teach it for money, there's significant gains from trade from having some sort of clearinghouse / place where people can easily meet. Does this already exist? Is anyone trying to make it? (Do you want to make it and need support of some sort?)

Why aren't we engaging in direct action (including civil disobedience) to pause AI development? Here's the problem: Yudkowksy [https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/]: "Many researchers steeped in these issues, including myself, expect that the most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances, is that literally everyone on Earth will die." Here's one solution: FLI Open Letter [https://futureoflife.org/open-letter/pause-giant-ai-experiments/]: "all AI labs...immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. This pause should be public and verifiable, and include all key actors. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium." Here's what direct action in the pursuit of that solution could look like (most examples are from the UK climate movement): Picketing AI offices [https://twitter.com/Radlib4/status/1653135998501662722?s=20] (this already seems to be happening!) Mass non-disruptive protest [https://www.theguardian.com/world/2023/apr/21/big-one-extinction-rebellion-cliimate-protest-london-xr] Strikes/walk-outs [https://www.theguardian.com/science/2021/sep/24/people-in-99-countries-take-part-in-global-climate-strike] (by AI developers/researchers/academics) Slow marches [https://www.itv.com/news/border/2023-04-29/just-stop-oil-protestors-stage-slow-march-through-town-centre] Roadblocks [https://www.bbc.co.uk/news/uk-england-london-59061509] Occupation [https://www.theguardian.com/uk/2012/jan/16/belfast-occupy-bank-of-ireland] of AI offices Performative vandalism [https://www.bbc.co.uk/news/uk-england-gloucestershire-64193016] of AI offices Performative vandalism of art [https://www.theguardian.com/environment/2022/oct/14/just-stop-oil-activists-throw-soup-at-van-goghs-sunflowers] Sabotage of AI computing infrastructure (on the model of ecotage [https://www.theguardian.

Together with a few volunteers, we prepared a policy document for the Campaign for AI Safety to serve as a list of demands by the campaign. It is called "Strong and appropriate regulation of advanced AI to protect humanity [https://campaignforaisafety.org/policy-recommendations/]". It is currently geared towards Australiand and US policy-makers, and I think it's not its last version. I would appreciate any comments!

This post is half object level, half experiment with “semicoherent audio monologue ramble → prose” AI (presumably GPT-3.5/4 based) program audiopen.ai [http://audiopen.ai].  In the interest of the latter objective, I’m including 3 mostly-redundant subsections:  1. A ’final’ mostly-AI written text, edited and slightly expanded just enough so that I endorse it in full (though recognize it’s not amazing or close to optimal)  2. The raw AI output 3. The raw transcript 1) DUBIOUS ASYMMETRY ARGUMENT IN WWOTF In Chapter 9 of his book, What We Are the Future, Will MacAskill argues that the future holds positive moral value under a total utilitarian perspective. He posits that people generally use resources to achieve what they want - either for themselves or for others - and thus good outcomes are easily explained as the natural consequence of agents deploying resources for their goals. Conversely, bad outcomes tend to be side effects of pursuing other goals. While malevolence and sociopathy do exist, they are empirically rare. MacAskill argues that in a future with continued economic growth and no existential risk, we will likely direct more resources towards doing good things due to self-interest and increased impartial altruism. He contrasts this eutopian scenario with an anti-eutopia: the worst possible world, which he argues (compellingly, I think) less probable because it requires convoluted explanations as opposed to simple desires like enjoying ice cream. He concludes that the probability of achieving a eutopia outweighs the low likelihood but extreme negative consequences of an anti-eutopia.  However, I believe McCaskill's analysis neglects an important aspect: considering not only these two extremes but also the middle distribution where neither significant amounts of resources nor agentic intervention occur.  When physics operates without agency-driven resource allocation, we have good reason to expect evolution to create conscious beings whos

Reflection on a university discussion about the future and safety of AI: Background: ~20 people, 2 assistant professors, other BSc and MSc level students of AI/ML at Charles University One of the professors sent us the FLI letter [https://futureoflife.org/open-letter/pause-giant-ai-experiments/], Petition by LAION [https://www.openpetition.eu/petition/online/securing-our-digital-future-a-cern-for-open-source-large-scale-ai-research-and-its-safety] (in my view accelerationist and insufficiently addressing issues raised by the AI x-risk community), and On the Dangers of Stochastic Parrots by Bender et al [https://dl.acm.org/doi/pdf/10.1145/3442188.3445922].. Supplemental materials were criticisms of longtermism (by Torres and Thorn). First we gave reasons for and against each petition. The professor in my view framed it that FLI is a suspicious organization (because of longtermist ties), x-risk issues are on the same level or less important than issues raised in Stochastic Parrots (bias, environmental concerns), and that academics support LAION more. * open source development proposed in LAION resonated with the students as an obviously good thing (at the start 16 said they would be in favor) * only 2 people (including me) would be in favor of FLI letter In spite of the framing, after a 1 hour discussion, more than a half of the students were convinced that AI x-risk is a serious issue and rejected issues from stochastic parrots being as important. No one changed their mind about LAION. One student (he was in a group discussion with me) changed his mind to be in favor of the FLI letter. I showed the professors the AI impacts survey [https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines:ai_timeline_surveys:2022_expert_survey_on_progress_in_ai] and they found it interesting. Some people seemed interested in the Lex Fridman with Eliezer Yudkowsky [https://www.youtube.com/watch?v=AaTRHFaaPG8] podcast.

Who else thinks we should be aiming for a global moratorium on AGI research at at [https://forum.effectivealtruism.org/posts/3vDarp6adLPBTux5g/what-a-compute-centric-framework-says-about-ai-takeoff?commentId=ZPmPeZeMmTfEa8LEZ] this [https://www.youtube.com/watch?v=gA1sNLL6yg4&ab_channel=BanklessShows] point [https://forum.effectivealtruism.org/posts/eAaeeuEd4j6oJ3Ep5/gpt-4-is-out-thread-and-links]? I'm considering ending every comment I make with "AGI research cessandum est", or "Furthermore, AGI research must be stopped" [https://en.wikipedia.org/wiki/Carthago_delenda_est].

I often see people talking past each other when discussing x-risks because the definition[1] covers outcomes that are distinct in some worldviews. For some, humanity failing to reach its full potential and humanity going extinct are joint concerns, but for others they are separate outcomes. Is there a good solution to this? 1. ^ "An existential risk is one that threatens the premature extinction of Earth-originating intelligent life or the permanent and drastic destruction of its potential for desirable future development." (source [https://existential-risk.org/])

Would an AI governance book that covered the present landscape of gov-related topics (maybe like a book version of the FHI's AI Governance Research Agenda [https://www.fhi.ox.ac.uk/wp-content/uploads/GovAI-Agenda.pdf]?) be useful? We're currently at a weird point where there's a lot of interest in AI - news coverage, investment, etc. It feels weird to not be trying to shape the conversation on AI risk more than we are now. I'm well aware that this sort of thing can backfire, and I'm aware that most people are highly sceptical of trying not to "politicise" issues like these, but it might be a good idea. If it was written by, say, Toby Ord - or anyone sufficiently detached from American left/right politics, with enough prestige, background, and experience with writing books like these - I feel like it might be really valuable. It might also be more approachable than other books covering AI risk, like, say, Superintelligence. It might also seem a little more concrete, because it might cover scenarios that are easier for most people to imagine/scenarios that are more near-term, and less "sci-fi". Thoughts on this?