This is the question. I agree with finm that we should stay alive since: 1) we just might figure out a way to stop the mass suffering, and 2) we just might develop the intention to do something about it. 

To add on a third, point, I would say: 3) if humanity goes extinct, then there is a possibility that either: 

• a) no other species capable of humanity's intelligence and empathy ever comes into being, whereas nature stays on, thus guaranteeing mass suffering until the end of the universe; or 
• b) even if another another species like humanity (or humanity itself) emerges, that would require hundreds of millions of years, during which sentient beings would suffer.

So I'm of the belief that humanity should be kept alive, because it is the only—albeit small—specter of hope for sentient beings. Now, I am a bit more hopeful than you, simply because within the span of a mere 4000 years of civilization (which is a blink of an eye in the grand scheme of things), humanity has, in many places: 

• recognized the evil of slavery, caste system, etc.; 
• outlawed discrimination on the basis of race, ethnicity, sex; 
• done away with the belief that war is "glorious"; 
• even passed laws outlawing certain practices against animals (California's Proposition 12); 
• actually tried to realize utopia (ex. French and Russian Revolutions, etc.) (even though they failed spectacularly)

Vive humanity! Well, of course we have done as much—if not much more—horrible things to each other and to animals, but ultimately...  upon whom else can we rest our hopes, my friend?

I agree that, right now, we're partly in the dark about whether the future will be good if humanity survives. But if humanity survives, and continues to commit moral crimes, then there will still be humans around to notice that problem. And I expect that those humans will be better informed about (i) ways to end those moral crimes, and (ii) the chance those efforts will eventually succeed.

If future efforts to end moral crimes succeed, then of course it would be a great mistake to go extinct before that point. But even for the information value of knowing more about the prospects for humans and animals (and everything else that matters), it seems well worth staying alive.

Some people make the argument that the difference in suffering between a worst-case scenario (s-risk) and a business-as-usual scenario, is likely much larger than the difference in suffering between a business-as-usual scenario and a future without humans. This suggests focusing on ways to reduce s-risks rather than increasing extinction risk.

A helpful comment from a while back: https://forum.effectivealtruism.org/posts/rRpDeniy9FBmAwMqr/arguments-for-why-preventing-human-extinction-is-wrong?commentId=fPcdCpAgsmTobjJRB

Personally, I suspect there's a lot of overlap between risk factors for extinction risk and risk factors for s-risks. In a world where extinction is a serious possibility, it's likely that there would be a lot of things that are very wrong, and these things could lead to even worse outcomes like s-risks or hyperexistential risks.

No, there is no way to be confident. 

I think humanity is intellectually on a trajectory towards greater concern for non-human animals. But this is not a reliable argument. Trajectories can reverse or stall, and most of the world is likely to remain, at best, indifferent to and complicit in the increasing suffering of farmed animals for decades to come. We could easily "lock in" our (fairly horrific) modern norms.

But I think we should probably still lean towards preventing human extinction. 

The main reason for this is the pursuit of convergent goals. 

It's just way harder to integrate pro-extinction actions into the other things that we care about and are trying to do as a movement. 

We care about making people and animals healthier and happier, avoiding mass suffering events / pandemics / global conflict, improving global institutions, and pursuing moral progress. There are many actions that can improve these metrics - reducing pandemic risk, making AI safer, supporting global development, preventing great power conflict - which also tend to reduce extinction risk. But there are very few things we can do that improve these metrics while increasing x-risk. 

Even if extinction itself would be positive expected value, trying to make humans go extinct is a bit all-or-nothing, and you probably won't ever be presented with a choice where x-risk is the only variable at play. Most of the things you can do that increase human x-risk at the margins also probably increase the chance of other bad things happening. This means that there are very few actions that you could take with a view towards increasing x-risk that are positive expected value.

I know this is hardly a rousing argument to inspire you in your career in biorisk, but I think it should at least help you guard against taking a stronger pro-extinction view. 

If humans go extinct, surely wild animal suffering will continue for another billion years on Earth - and there are a lot more wild animals than farmed animals. If we survive, we can continue the work of improving the lives of both farmed and wild animals.

Utilitarians aware of the cosmic endowment, at least, can take comfort in the fact that the prospect of quadrillions of animals suffering isn't even a feather in the scales. They shut up and multiply.

(Many others should also hope humanity doesn't go extinct soon, for various moral and empirical reasons. But the above point is often missed among people I know.)

To answer your question very directly on the confidence of millions of years in the future, the answer I think is "no", because I don't think we can be reasonably confident and precise about any significant belief about the state of the universe millions of years into the future.^[1] I'd note that the article you link isn't very convincing for someone who doesn't share the same premesis, though I can see it leading to 'nagging thoughts' as you put it.

Other ways to answer the latter question about human extinction could be:

• That humanity is positive (if human moral value is taken be larger than the effect on animals)
• That humanity is net-positive (if the total effect of humanity is positive, most likely because of belief that wild-animal suffering is even worse)
• Option value, or the belief that humanity has the capacity to change (as others have stated)

In practice though, I think if you reach a point where you might consider it to be a moral course of action to make all of humanity extinct, perhaps consider this a modus tonens of the principles that brought you to that conclusion rather than as a logical consequence that you ought to believe and act on. (I see David made a similar comment basically at the same time)

1. ^{^}

   Some exceptions for phyisics especially outside of our lightcone yada yada, but I think for the class of beliefs (I used significant beliefs) that are similar to this question this holds