Uncertainty, Structure, and Optimal Integration

The EA community and Future Fund are running a risk in wasting good analysis of propositions when considering long-term predictions that entail complex, compounding uncertainties. Specifically, by ignoring or over-simplifying the complex ways our propositions interrelate (i.e., how they are “structured”), we stand to make substantial errors in our predictions. Moreover, the failure to appropriately “unpack” the structure behind our propositions risks substantial inefficiency in our focus of which uncertainties (whether they be based on a lack of information, or substantive disagreement) are more critical to resolve.

In Sections 1 and 2 I provide a primer to the uninitiated on the nature and importance of unpacking and disentangling the uncertain propositions that comprise our prediction problems.

In Sections 3 and 4 I lay out my analytical approach that seeks to capture and resolve these issues. I provide a fully worked through, illustrative decomposition of the misaligned AI X-Risk prediction problem, using a combination of Bayesian Network modelling and simulation techniques. My intent is to illustrate this analytical approach as a viable framework for a) making arguments and assumptions explicit and quantifiable in a unified manner, b) drawing mathematically optimal inferences (i.e., predictions) even in the face of multi-layered uncertainties, and c) laying the foundations for accessible computational methods that will assist further research. My assumptions and propositions within the model are predominantly grounded in extant estimates and analyses drawn from AI experts and fellow contributors. However, I highlight a number of previously overlooked or underspecified propositions and conditional probabilities that affect our predictions, but have previously remained hidden.

In Section 5 I step through the results of my analysis. Among them, I show that once structural disentanglement is taken into account, under reasonable, informed assumptions, the P(Misaligned AGI X-Risk) by 2070 is 1.68% (+/- 1.25%), rising to 6.53% (+/- 4.32%) by 2100. Though if we wish to assume an AGI has been built by 2070 (i.e., P(Misaligned AGI X-Risk|AGI)), then this rises further to 5.32% (+/- 3.42%) by 2070, and 11.22% (+/- 6.86%) by 2100. I additionally provide predictions for AGI build likelihoods across the ...

TD_Pilditch

Posts 1

Posts
1