Co-founder of and researcher at Convergence. Convergence does foundational existential risk strategy research. See here for our growing list of publications.

Past: R&D Project Manager, Software Engineer.


Assessing the impact of quantum cryptanalysis

Thanks Linch; I actually missed that the prediction had closed!

Assessing the impact of quantum cryptanalysis

Metaculus: Will quantum computing "supremacy” be achieved by 2025? [prediction closed on Jun 1, 2018.]

While I find it plausible that it will happen, I'm not personally convinced that quantum computers will be practically very useful due the difficulties in scaling them up.

Improving the future by influencing actors' benevolence, intelligence, and power

Excellent points, Carl. (And Stefan's as well.) We would love to see follow-up posts exploring nuances like these, and I put them into the Convergence list of topics worth elaborating.

Effective Altruism Stipend: A Short Experiment by EA Estonia

Sounds like you got some pretty great engagement out of this experiment! Great work! This exact kind of project, and the space of related ideas seems well worth exploring further.

The five people that we decided to reject were given feedback about their translations as well as their motivation letters. We also provided two simple call-to-actions to them: (1) read our blog and join our newsletter, and (2) follow our FB page and attend our public events. None of these five people have so far done these actions to our awareness.

Semi-general comment regarding rejections: I think, overall, rejection is a sensitive matter. And if we do want rejected applicants (to stipends, jobs, projects, ...) to try more or to maintain their interest in the specific project and in EA overall, we need to take a lot of care. I'm, for example, concerned that the difficulty of getting jobs at EA orgs and the situation of being rejected from them discourages many people from engaging closer with EA. Perhaps just being sympathetic and encouraging enough will do a lot of good. Perhaps there's more we could do.

EA Forum feature suggestion thread

Variant of Korthon's comment:

I never look at the "forum favorites" section. It seems like it's looked the same forever and it takes up a lot of screen real estate without any use for me!

Why making asteroid deflection tech might be bad

Vision of Earth fellows Kyle Laskowski and Ben Harack had a poster session on this topic at EA Global San Francisco 2019: https://www.visionofearth.org/wp-content/uploads/2019/07/Vision-of-Earth-Asteroid-Manipulation-Poster.pdf

They were also working on a paper on the topic.

Clarifying existential risks and existential catastrophes

Thank you for this article, Michael! I like seeing the different mainline definitions of existential risk and catastrophe alongside each other, and having some common misunderstandings clarified.

Just a minor comment:

That said, at least to me, it seems that “destruction of humanity’s longterm potential” could be read as meaning the complete destruction. So I’d personally be inclined to tweak Ord’s definitions to:

  • An existential catastrophe is the destruction of the vast majority of humanity’s long-term potential.
  • An existential risk is a risk that threatens the destruction of the vast majority of humanity’s long-term potential.[4]

Ord was presumably going for brevity in his book, and I think his definition succeeds quite well! I don't think generally adding 4 words to Ord's short nice definition would be worth it. There's other details that could be expanded on as well (like how we can mostly consider the definition in Bostrom 2012 to be a more expanded one). Expanding helps with discussing a particular point, though.

Database of existential risk estimates

I think this is an excellent initiative, thank you, Michael! (Disclaimer: Michael and I work together on Convergence.)

An assortment of thoughts:

  • More and more studious estimates of x-risks seem clearly very high value to me due to how much the likelihood of risks and events affect priorities and how the quality of the estimates affect our communication about these matters.
  • More estimates should generally should increase our common knowledge of the risks, and individually, if people think about how to make these estimates, they will reach a deeper understanding of the questions.
  • Breaking down the causes of one's estimates is generally valuable. It allows one to improve one's estimates, understanding of causation, and to discuss them in more detail.
  • More estimates can be bad if low quality estimates swamp out better quality ones somehow.
  • Estimates building on new (compared to earlier estimates) sources of information are especially interesting. Independent data sources increase our overall knowledge.
  • I see space for someone writing an intro post on how to do estimates of this type better. (Scott Alexander's old posts here might be interesting.)
Some thoughts on Toby Ord’s existential risk estimates

This kind of complexity tells me that we should talk more often of risk %'s in terms of the different scenarios they are associated with. E.g., the form of current trajectory Ord is using, and also possibly better (if society would act further more wisely) and possible worse trajectories (society makes major mistakes), and what the probabilities are under these.

We can't disentangle talking about future risks and possibilities entirely from the different possible choices of society since these choices are what shapes the future. What we do affect these choices.

(Also, maybe you should edit the original post to include the quote you included here or parts of it.)

State Space of X-Risk Trajectories

Happy to see you found it useful, Adam! Yes, general technological development corresponding to scaling of the vector is exactly the kind of intuition it's meant to carry.

Load More