Warning shot

A related notion is that of a fire alarm, a warning sign that creates common knowledge that some technology—typically dvancedavanced artificial intelligence—actually poses an existential risk.[2]

A warning shot is a global catastrophe that indirectly reduces existential risk by increasing concern about risks of future catastrophes. 

A warning shot is a global catastrophe that increases awareness andindirectly reduces existential risk by increasing concern about risks of similar future catastrophes and thus reduces the probability of an existential catastrophe.catastrophes. 

The expression warning sign is sometimes used to describe any event that increases awareness and concern about a particular category of existential risk, regardless of whether the event itself constitutes a global catastrophe. For example, plausible candidates for an AI warning sign include not only a catastrophic failure by an AI system but also public outreach campaigns or the publication of an exceptionally persuasive book on AI safety.[1] 

A warning shot is a global catastrophe that increases awareness and concern about risks of similar future riskscatastrophes and thus reduces the probability of an existential catastrophe.

A related notion is that of a fire alarm, a warning sign that creates common knowledge that some technology—typically dvanced artificial general intelligence—actually poses an existential risk.[2]

artificial intelligence | existential risk | global catastrophic risk

Carlsmith, Joseph (2021) Is power-seeking AI an existential risk?, Open Philanthropy, April, section 6.2.

Note, however, that both "warning shot" and "fire alarm" are sometimes used as synonyms for "warning sign".[3][4]

The expression warning sign is sometimes used to describe any event that increases awareness and concern about a particular category of existential risk, regardless of whether the event itself constitutes a global catastrophe. For example, plausible candidates for an AIAI safetywarning sign include not only a catastrophic failure by an AI system but also public outreach campaigns or the publication of an exceptionally persuasive book on AI safety.[1] However, sometimes the expression "warning shot"

A related notion is itself used in this expansive sense.that of a fire alarm, a warning sign that creates common knowledge that some technology—typically artificial general intelligence—actually poses an existential risk.[2] 

Note, however, that both "warning shot" and "fire alarm" are sometimes used synonyms for "warning sign".[3][4]

  1. ^
  2. ^

    Grace, Katja (2021) Beyond fire alarms: freeing the groupstruck, AI Impacts, September 26.

  3. ^

    Kokotajlo, Daniel (2020) What are the most plausible ‘AI Safety warning shot’ scenarios?, AI Alignment Forum, March 26.

  4. ^

    McCluskey, Peter (2021) AI fire alarm scenarios, Bayesian Investor, December 23.

The expression warning sign is sometimes used to describe any event that increases awareness and concern about a certainparticular category of existential risk, regardless of whether the event itself constitutes a global catastrophe. For example, plausible candidates for an AI safety warning sign include not only a catastrophic failure by an AI system but also public outreach campaigns or the publication of an exceptionally persuasive book on AI safety.[1] However, sometimes the expression "warning shot" is itself used in this expansive sense.[2] 

The expression warning sign is sometimes used to describe any event that increases awareness and concern about a certain category of existential risk, regardless of whether the event itself constitutes a global catastrophe. For example, plausible candidates for an AI safety warning sign include not only a catastrophic failure by an AI system but also public outreach campaigns or the publication of an exceptionally persuasive book on AI safety.safety.[1] However, sometimes the expression "warning shot" is itself used in this expansive sense.[2] 

Sometimes theThe expression "warning shot"warning sign is sometimes used more generally to coverdescribe any event that increases awareness and concern about a certain typecategory of existential risk, regardless of whether the event itself constitutes a global catastrophe.[1] A For example, plausible candidates for an AI safety warning sign include not only a catastrophic failure by an AI system is a plausible candidate for an AI Safety warning shot, but candidates may also include a public outreach campaigncampaigns or the publication of an exceptionally persuasive book on AI safety.[1] However, sometimes the expression "warning shot" is itself used in this expansive sense.[2] 

  1. ^

    Kokotajlo, Daniel (2020) What are the most plausible ‘AI Safety warning shot’ scenarios?, AI Alignment Forum, March 26.

  2. ^
  3. ^

    Kokotajlo, Daniel (2020) What are the most plausible ‘AI Safety warning shot’ scenarios?, AI Alignment Forum, March 26.

Sometimes the expression "warning shot" is used more generally to cover any event that increases awareness and concern about a certain type of existential risk, regardless of whether the event constitutes a global catastrophe.

I agree with that characterisation.

Fwiw I've also heard the term "warning sign" being used for non-catastrophes that increase such awareness and concern. In those contexts, "warning shot" has been defined more narrowly as catastrophes that increase such awareness and concern.

4Pablo3mo
Thanks, I reworded the paragraph to reflect this. If you have any references discussing "warning signs", please let me know. (I recall ASB had written some stuff on the topic but it doesn't seem to have been published.)
4Stefan_Schubert3mo
Sorry, no. Yes, I also recall that. I also had some conversations with Eric Drexler where I think this terminology may have been used.

Sometimes the expression "warning shot" is used more generally to cover any event that increases awareness and concern about a certain type of existential risk, regardless of whether the event constitutes a global catastrophe.[1] For example, aA catastrophic failure by an AI system is a plausible candidate for an AI Safety warning shot, but candidates may also include a public outreach campaign or the publication of an exceptionally persuasive book on AI safety.[2]

Terminology

Sometimes the expression "warning shot" is used more generally to cover any event that increases awareness and concern about a certain type of existential risk, regardless of whether the event constitutes a global catastrophe.[1] For example, a catastrophic failure by an AI system is a plausible candidate for an AI Safety warning shot, but candidates may also include a public outreach campaign or the publication of an exceptionally persuasive book on AI safety.[2]

  1. ^

    Kokotajlo, Daniel (2020) What are the most plausible ‘AI Safety warning shot’ scenarios?, AI Alignment Forum, March 26.

  2. ^

BibliographyFurther reading