Biblical advice for people with short AI timelines

The Christians in this story who lived relatively normal lives ended up looking wiser than the ones who went all-in on the imminent-return-of-Christ idea. But of course, if christianity had been true and Christ had in fact returned, maybe the crazy-seeming, all-in Christians would have had huge amounts of impact.

Here is my attempt at thinking up other historical examples of transformative change that went the other way:

• Muhammad's early followers must have been a bit uncertain whether this guy was really the Final Prophet. Do you quit your day job in Mecca so that you can flee to Medina with a bunch of your fellow cultists? In this case, it probably would've been a good idea: seven years later you'd be helping lead an army of 100,000 holy warriors to capture the city of Mecca. And over the next thirty years, you'll help convert/conquer all the civilizations of the middle east and North Africa.

• Less dramatic versions of the above story could probably be told about joining many fast-growing charismatic social movements (like joining a political movement or revolution). Or, more relevantly to AI, about joining a fast-growing bay-area startup whose technology might change the world (like early Microsoft, Google, Facebook, etc).

• You're a physics professor in 1940s America. One day, a team of G-men knock on your door and ask you to join a top-secret project to design an impossible superweapon capable of ending the Nazi regime and stopping the war. Do you quit your day job and move to New Mexico?...

• You're a "cypherpunk" hanging out on online forums in the mid-2000s. Despite the demoralizing collapse of the dot-com boom and the failure of many of the most promising projects, some of your forum buddies are still excited about the possibilities of creating an "anonymous, distributed electronic cash system", such as the proposal called B-money. Do you quit your day job to work on weird libertarian math problems?...

People who bet everything on transformative change will always look silly in retrospect if the change never comes. But the thing about transformative change is that it does sometimes occur.

(Also, fortunately our world today is quite wealthy -- AI safety researchers are pretty smart folks and will probably be able to earn a living for themselves to pay for retirement, even if all their predictions come up empty.)

Relatedly, a behaviour I dislike is  being repeatedly publicly wrong without changing and acknowledging fault. Mainstream Christianity is guilty of this, though so are many other social movements.

I think if  it turns out that short AI timelines are wrong, those with short timelines should acknowledge it and the EA as a whole should seek to understand why we got it so wrong. I will think it odd  if those who make repeatedly wrong predictions continue to be taken seriously.

Also, I'd like to see more concrete testable short term predictions from those we trust with AI predictions. Are they good forecasters in general? Are they well calibrated or insightful in ways we can test?

or b) kill/enslave everyone

Tangent: did you mean this literally? I know some folks who are worried about people being killed, but I haven't heard of anyone worrying about human enslavement (and that distinction seems like a point in favor of "people worried about this stuff aren't picking scary-sounding scenarios at random," since automated labor would presumably be way more efficient than human labor in these scenarios).

A cynical atheist would say that early Christians on some level were not certain of their beliefs, which was an important factor in the recommendations. 

People who believe in transformative AI can openly acknowledge that there is uncertainty about the future but maybe that will amount to the same thing. 

It seems like these observations could be equally explained by Paul correctly having high credence in long timelines, and giving advice that is appropriate in worlds where long timelines are true, without explicitly trying to persuade people of his views on timelines. Given that, I'm not sure there's any strong evidence that this is good advice to keep in mind when you actually do have short timelines, regardless of your views on the Bible.

I think an important difference is the explicitness of credences. I expect most of the short-timeline AI people to have explicit probability distributions and I expect them to behave accordingly. This would then definitely entail retirement savings etc. as (from my personal encounters) many have non-negligible probability mass on AGI after their lifetime.

I'm sure there are also the "99.9% within the next 20 years"-people but I'm sure they're doing better within their subculture than early Christians did and usually don't risk unemployment, starvation, or ostracism.

Summary: 

1. While some narratives about AI alignment bear a conspicuous resemblance to the apocalyptic thinking and eschatology of some Christians in history, there isn't much that fundamentally distinguishes that mindset towards AI alignment from similar mindsets towards other ostensible existential risks.
2. This has been true at times during the last century and remains true today. It was at times crucial, if not necessary, for some of those involved in other communities similar to long-termist effective altruism to make decisions and take actions that contradicted much of this advice.

                                                                                                                                                                                                                                                            ---

This advice is in one way also applicable to other potential global catastrophic or existential risks as well but in another way may not be applicable to any of them. Even before the advent of nuclear weapons, World War II (WWII) was feared to potentially destroy civilization. Between the Cold War that began a few years later and different kinds of global ecological catastrophe, there are hundreds of millions of people across several generations who have experienced for more than half a century a life in way that had them convinced they were living at the hinge of history. While such concerns may have been alleged to be fears too similar to religious eschatology, almost all of them were rooted in secular phenomena examined from a naturalistic and materialist perspective.

This isn't limited to generic populations and includes communities that are so similar to the existential risk (x-risk) reduction community of today that they serve as a direct inspiration for our present efforts. After the Manhattan Project, Albert Einstein and other scientists who contributed to the effort but weren't aware of the full intentions of the government of the United States for nuclear weapons both wanted to do something about their complicity in such destruction. For the record, while they weren't certain either way, at the time many of those scientists feared a sufficiently large-scale nuclear war could indeed cause human extinction. Among others, those scientists founded the Bulletin of Atomic Scientists, likely the first ever 'x-risk reduction' organization in history.

In both the United States and the Soviet Union, scientists and others well-placed to warn the public about the cataclysmic threat posed by the struggle for more power by both superpowers took personal and professional risks. Some of those who did so were censured, fired and/or permanently lost their careers. Some were even criminally convicted or jailed. Had they not, perhaps none of us would have ever been born to try reducing x-risks or talk about how to think about that today.

To some extent, the same likely remains true in multiple countries today. The same is also true for the climate crisis. Employees of Amazon who have made tweets advocating for greater efforts to combat the climate crisis have been fired because their affiliation with Amazon in that way risks bringing too much attention to how Amazon itself contributes to the crisis. There also more and more people who through civil disobedience have gotten arrested for their participation in civil disobedience to combat the climate crisis or other global catastrophic risks. 

I've known many in effective altruism who've changed their careers so to focus on x-risk reduction not limited to AI alignment. There are millions of young people around the world who are pursuing careers intended to do the same because they both believe it's more important than anything else they could do and it's futile to pursue anything else in the face of looming catastrophe. All of this is anticipated to be critical in their lifetimes, often in the next 20-30 years. All of those people have also been presumed to be delusional in a way akin to the apocalyptic delusions of religious fanatics in history.

While for the other risks there isn't the same expected potential for transhumanism, indefinite life extension and utopian conditions, the future of humankind and perhaps all life is considered to be under threat. Beyond effective altruism, I've got more and more friends, and friends of friends, who are embracing a mindset entailing much of the above. Perhaps what should surprise us is that more people we don't know from in and around effective altruism aren't doing the same.