The rationality community has a tradition of checking for biases, particularly when it comes to evaluating the non-intuitive risks of general AI.
We thought you might like this list, adapted from a 2015 essay by Forrest Landry. Many names of biases listed may already be familiar for you. If you "boggle" more at the text, you will find curious new connections to evaluating upcoming risks of AI developments.
About Forrest Landry
Forrest is a polymath working on civilisation design and mitigating risks of auto-scaling/catalysing technology (eg. Dark Fire scenario). About 15 years ago, he started researching how to build in deep existential alignment into the internals of AGI, applying his deep understanding of programming, embodied ethics, and meta-physics. Then, Forrest discovered the substrate-needs convergence argument (as distinct from yet much enabled by instrumental convergence). Unfortunately, because of substrate-needs convergence, any approach to aligning any AI at the embedded level turned out unsound in practice (and moreover, inconsistent with physical theory). To inquire further, see this project.
Introduction
Note on unusual formatting: Sentences are split into lines so you can parse parts precisely.
Ideally,
in any individual or group decision making,
there would be some means, processes,
and procedures in place to ensure that
the kinds of distortions and inaccuracies
introduced by individual and collective
psychological and social bias
do not lead to incorrect results,
and thus poor (risk-prone) choices,
with potentially catastrophic outcomes.
While many types of bias
are known to science
and have been observed
to be common to all people
and all social groups, the world over
in all working contexts, regardless
of their background, training, etc,
they are also largely unconscious,
being 'built-in' by long-term
evolutionary processes.
These unconscious cognitive biases,
while they are adaptive for the purposes
of our being able to survive in
non-technological environments,
are also not able to serve us equally well
when attempting to survive our
current technological contexts.
The changes in our
commonly experienced world
continue to occur far too fast
for our existing evolutionary
and cognitive adaptations
to adjust naturally.
We will therefore need to
add the necessary corrections to
our thinking and choice-making process,
our own evolution, 'manually'.
The hope is that
these 'adjustments' might
make it possible to mitigate
the distortions and inaccuracies
introduced by the human condition
to the maximum extent possible.
Bear in mind
that each type of bias
does not just affect individuals –
they also arise due to specific
interpersonal and trans-personal effects
seen only in larger groups.
These bias aspects affect
all of us, and in all sorts of ways,
many of which are complex.
It is important for everyone involved
in critical decisions and projects
to be aware of these general
and mutual concerns.
We all run on corrupted hardware.
Our minds are composed of many modules,
and the modules that evolved to make
us seem impressive and gather allies
are also evolved to subvert the ones
holding our conscious beliefs.
Even when we believe that
we are working on something
that may ultimately determine
the fate of humanity, our signaling
modules may hijack our goals so as
to optimize for persuading outsiders
that we are working on the goal,
instead of optimizing for
achieving the goal.
What is intended herein
is to make some of these
unconscious processes conscious,
to provide a basis, and
to identify the need,
for clear conversation
about these topics.
Hopefully, as a result
of these conversations,
and with the possibility
of a reasonable consensus reached,
we will be able to identify (or create)
a good general practice of decision making,
which when implemented both
individually and collectively
(though perhaps not easily),
can materially improve
our mutual situation.
The need for these practices
of accuracy, precision, and correctness
in decision making are especially
acute in proportion to the degree
that we all find ourselves faced
with a seemingly ever increasing
number of situations for
which our evolution has
not yet prepared us.
Where the true goal
is making rational, realistic,
and reasonably good choices
about matters that may
potentially involve many people,
larger groups and tribes, etc,
many specific and strong
cognitive and social biases
will need to be compensated for.
Particularly in regards
to category 1 and 2 extinction risks
nothing less than complete and full
compensation for all bias,
and the complete application
of correct reason
can be allowed for.
This sequence will not attempt
to outline or validate any of the
specific risk possibilities and outcomes
for which there is significant concern
(this is done elsewhere).
Nor can it attempt to outline or define
which or what means, processes, or procedures
should be used for effective individual
or group decision making.
As the 'general problem of governance',
the main issue remains one of the identification,
development, and testing/refining of
such means and methods by which
all bias can be compensated for,
and a basis for clear reason
thereby created.
Hopefully this will
lead to real techniques of
group decision making –
and high quality decisions –
that can be realistically defined,
outlined, and implemented.
A Partial List Of Affecting Bias...
The next posts cover a list
of some of the known types of bias
that have a significant and real potential to
harmfully affect the accuracy and correctness
of extinction risk assessments.
Each bias will be given its
common/accepted consensus name,
along with relevant links to Wikipedia
articles with more details.
Each bias will be briefly described
with particular regard to its potential impact
on risk assessment in an existential context.
