Update October 2022: Please see an updated and expanded version of this post linked here on arXiv (co-authored with Anders Sandberg). The original post is below. 

Summary

The leading discussion in existential risk and effective altruism centers on investigating each risk separately, such as misaligned AI and nuclear war. While the outcomes of many risk situations hinge on the performance of human groups---whether a collective (examples include democratic governments and scientific communities) can work together and generate good outcomes. This issue is often referred to as coordination problems in the existential risk and effective altruism communities. Here, I first argue that it is too narrow to think about these rich and encompassing phenomena as one of coordination. I then propose to think about them as Collective Intelligence problems. Collective Intelligence is a transdisciplinary field that studies how a collective of individuals performs together to achieve goals in a broad sense. The applications cross-cut humans (tribes, corporations, cities, etc.), animal groups, robotic swarms, and collections of neurons. I give an overview of this field and highlight findings relevant to improving the collective intelligence of human groups. Finally, existential risk and effective altruism communities can benefit from learning from other transdisciplinary disciplines and put a greater effort into identifying commonalities across various risk domains to help improve the robustness of our society in the face of many potential risks.

About the author

Vicky Chuqiao Yang is a fellow at the Santa Fe Institute in the United States. Her scientific research uses quantitative methods to understand and predict collective human behavior. More about her work on her website, Google Scholar, or Twitter. 

 ______________________________________________

We’ve got to be as clear-headed about human beings as possible, because we are still each other’s only hope.    ---James Baldwin
 

Many existential risk efforts, such as improving AI safety, preventing large-scale nuclear war, and preventing climate catastrophe, hinge on better collective performance of human groups. For example, mitigating global pandemic risks consists of coordinating individual lifestyle changes, the research efforts of the scientific community, and policies of nation-states. Our ability for effective collective decision-making and collective action are infrastructures for responding to a wide variety of, if not all, catastrophic risks. The vulnerability of having a weak infrastructure, such as in collective decision-making and collective action, can be compared to a pyramid sitting on its tip. Many forces can lead the pyramid to fall, be it a gust of wind, a shake of the ground. Compared to focusing on which forces will tip the pyramid over, and preventing these forces from happening, it is more useful to find ways to shift the pyramid out of the vulnerable position. Similarly, discussion in existential risks has been centered around which risks are likely to occur and how to circumvent each one. While this line of work is necessary and valuable, the complementary perspective of improving the system's infrastructure to be more stable under all risks can be fruitful but remains less explored. 

In effective altruism and existential risk communities, work concerned with human collective decision and action is often referred to as “coordination problems.” However, I find it narrow and restricting to think about these complex phenomena of collective decisions and action as problems of coordination. Firstly, the term “coordination problem” is strongly related to coordination games from game theory. While a helpful framework in many applications, it typically considers a small number of agents with a fixed set of explicit rules. While many hard problems we face are among a large number of agents, and the mechanisms of interaction may be implicit or change over time. This distinction is at the core of some most important human endeavors. According to Harari (2015), Homo Sapiens have accomplished astonishing achievements and dominated the earth because humans can cooperate in large numbers and in flexible ways. “In large numbers” is in contrast to chimpanzees and other mammals, which can only cooperate in small groups where one develops a personal relationship with their counterparts. “In flexible ways” is as opposed to bees and ants, which can work in large numbers but with more rigid roles. This idea is echoed in the social brain hypothesis (Dunbar 1998), which posits that human intelligence developed because of the need to handle the increased complexity of living in large social groups, suggesting human intelligence and group behavior are inextricably linked. Second, besides the consideration of group size and flexible ways of interaction, another issue with viewing the human collective through the lens of coordination is that it tends to dwell on the negative outcomes of bad coordination like tragedy of the commons. These are important to avoid, however making it the sole focus risks neglecting the upsides from the human collective, where the group is more capable than the sum of the individuals within it. The focus on avoiding bad coordination may lead to neglecting how to achieve that. Instead of thinking about these human-collective issues as an issue of coordination, I would advocate for thinking about them more broadly as an issue of Collective Intelligence. 

A brief introduction to the transdisciplinary field of Collective Intelligence 

The issue of how a collective work together is a central question in many applications in human society and beyond. Examples include human groups, such as in mitigating existential risks, elections, prediction markets, and juries; animal groups, such as in deciding the direction for migration; robotic swarms, such as designing rules for individual robots such that the collective can perform certain tasks; and neurons, such as the brain making coherent sense of the world while each neuron responds to different, and sometimes conflicting stimuli. In all these diverse applications, a shared problem is how to process distributed information effectively. Researchers have come together to study the phenomena cross-cutting these application domains, and the transdisciplinary field is referred to as Collective Intelligence. I refer to this field as transdisciplinary, as opposed to interdisciplinary, because it studies a shared phenomenon that manifests in many disciplines, spanning computer science, neuroscience, robotics, animal behavior, and many social sciences. It transcends the disciplinary boundaries, while interdisciplinary often refers to combining two fields to discover new things in either one. Collective intelligence is still a young field. It has many commonalities with the community engaging in effective altruism and existential risk---not only because its subject of study is closely related to mitigating existential risk (see Bak-Coleman et al. 2021, for a good argument for collective behavior to be considered a “crisis discipline”), but also because they share similar challenges in organizing a transdisciplinary effort. 

With many rigorous research efforts in the past two decades in Collective Intelligence, researchers have found many factors that help or hurt the performance of human groups in solving problems and making collective decisions, predictions, or estimations. While research does not yet offer recipes for constructing a good human collective, as collective performance is the outcome of complex interactions of many variables. These findings point to a few general directions for improving the collective performance of human groups and I offer a brief (and non-exhaustive) summary of the prominent findings below. 

Summary of Collective Intelligence findings for human collective performance

Collective intelligence. Similar to individual general intelligence (commonly known as IQ), that one individual can excel at a wide range of tasks, such as math and music, Woolley and colleagues (2010) find a similar property for human groups, named collective intelligence. When asking small groups to perform a wide range of tasks, including brainstorming, sudoku, and unscrambling words, the performance on a subset of the tasks gives a good out-of-sample prediction. In other words, some groups outperform others on a wide variety of tasks, suggesting the equivalent of general intelligence for groups. This finding is replicated by later studies (see Riedl et al. 2021 for a review). The studies look into what makes some groups more effective than others. Among the peer-reviewed studies (see Riedl et al. 2021 for a summary), the consensus is that the social process is more important than the skill of individual members. Social perceptiveness, the ability of individuals to identify social cues, is a key factor in improving group performance. This is manifested in group behaviors such as even conversational turn-taking---each group member speaks roughly equal amount. Consequently, groups with a higher proportion of women tend to have higher collective intelligence, as being female is correlated with greater social perceptiveness. The IQ of individual members plays a much less, and some argue negligible role.  

Diversity. Besides the social perceptiveness of group members, research finds benefit in having a diverse group of individuals. The diversity referred to here is diversity in knowledge and cognitive models. Abundant research has found in lab experiments, theoretical and computational models, and in real-world problem-solving scenarios, the phenomenon of a “diversity bonus”--- that a diverse group performs better than a homogenous group (Page 2019, Aminpour et al. 2021). Some even find a diverse group of non-experts can outperform a homogeneous group of experts (Hong & Page 2004). For a group of diverse agents to work together, an important aspect is cognitive alignment---such as commitment to group goals and shared beliefs (Krafft 2019). Another line of work finds that maintaining diversity in the group is hard---conformity and traditional market forces are against it. Mann & Helbing (2016) proposes alternative incentives to reward accurate minority predictions for maintaining diversity in the group---especially reward the minority who is right when the majority is wrong. 

Committed minorities. There has been much evidence that the presence of committed minorities, those who stubbornly hold on to their opinion, can lead to substantial changes in the collective behavior of a group. Most notably, Centola et al. (2018) find through human subject experiments that a critical mass of committed individuals (around 25% in their experiments) can tip social conventions towards the direction of these committed individuals. This critical transition is also predicted by an abundance of theoretical studies (such as Xie et al. 2011). It would require further investigation to understand the precise critical mass needed in different scenarios, however, the powerful effects of committed minorities on groups can be a fruitful direction in thinking about how to elicit (or prevent) social change.

Social influence. An area under heated debate is whether and how individuals should communicate with each other in a collective. There are quality experimental studies that have found social influence can have both positive and negative effects on collective performance (see Jayles et al., 2017 for an example of positive effect, and Lorenz et al., 2011 for an example of negative effect). On the one hand, letting individuals exchange information can lead to the loss of independent information, and worse collective performance (also referred to as groupthink). On the other hand, communication may help individuals deliberate and discover better answers. I think the story is complicated because the effect of social influence is likely to depend on several other variables. A paper led by the author (Yang et al. 2021) predicts it depends on the proportion of individuals using social information for their decisions, and whether committed minorities are present. Others find they also depend on the social network structure and adaptability (Almaatouq et al. 2020; Becker, Brackbill & Centola, 2017). The bottom line is that most people blindly following others does not lead to good outcomes in most scenarios.

Better sensors. One important component for improving group performance is to let individuals gather better information. This is especially important in forecasting and prediction tasks, such as election forecasts. This can be achieved through asking better survey questions. For example, in a method called “surprisingly popular” (Prelec, Seung & McCoy 2017), instead of asking people what they think, researchers also ask people what they think other people think. This helps to discover surprising phenomena that are hidden by social norms, such as Trump winning the 2016 US election. Another example is to use individual’s social circles as better sensors (Galesic et al., 2021)--- instead of asking individuals who they will vote for, ask who their friends will vote for. 

Overall. The literature in this field is vast, and here I highlight a small subset of findings relevant to the performance of human groups. I provide a list at the end of the post as resources for learning more. One theme that appears in these scientific studies is that individuals’ ability to perform the task matters less than how the individuals interact. 

Closing thoughts

In the coordination perspective, a central issue is how to resolve different people wanting different things. Collective intelligence offers a broad and, I think, hopeful, perspective on this issue. This difference may first appear as an obstacle to coordination, while in the collective intelligence perspective, this difference, if harnessed with the right aggregation mechanism, is a source of strength. Take the example of a flock of fish. Each fish senses their local environment---food, temperature, potential predator, etc., and has a different preferred direction of where to go next. Nevertheless, the collective effectively aggregates information from all fish through local interactions in movement. The collective’s movement responds to a much wider environment, more than the range of any individual fish. Thus collective intelligence can be generated from different individuals acting on different information and wanting different outcomes. The differences should be a source of hope, and the key question should be how to harness and aggregate the individuals’ differences in productive ways. 

I do not have the answers to the big collective intelligence problems facing human collectives. But I am hopeful because nature has solved many similarly hard problems. Besides the example of fish, each neuron in our brain receives different, and sometimes conflicting information, but the brain is able to make coherent sense of the world. Hopefully, we will learn from nature's approaches to help human groups through the transdisciplinary research effort of the many scientists working in this area (including myself). 

The existential risk and effective altruism communities are also collectives, whose performance are influenced by the overarching principles discussed above. The summary mentions that both the presence of women and diversity help improve the performance of a collective. At the time of writing, the academic community in effective altruism and existential risk seem to be male-dominated, especially among those who are senior. Thinking about how to involve a more diverse community can help improve the collective performance of these fields. This also applies to attracting people from other domains with different experiences and backgrounds to effective altruism and existential risk reduction.

There is much that the existential risk and effective altruism communities can learn from other transdisciplinary efforts like collective intelligence (see also blog posts What complexity science and simulation have to offer effective altruism on their relationship with Complexity Science, another long-standing transdisciplinary community). These transdisciplinary communities look across various application domains (such as neurons, humans, and robots) and identify common central processes. They study these cross-cutting phenomena so that robotic scientists learn from animal scientists, social scientists learn from neuroscientists, etc. I think a similar spirit can also help in the existential risk and effective altruism communities. What fundamental processes make AI, nuclear war, and pandemics dangerous? Some research has suggested candidates----it may have to do with the pace of regulatory innovation being far slower than that of technological innovation; or they all have to do with dangerous technology being accessible to a small number of bad actors. These exercises of identifying commonality across various risk domains help recognize key characteristics underlying humanity's greatest dangers. With the understanding of commonalities across risks, there is hope for finding cross-cutting solutions to many problems at once. 

Acknowledgments

This post benefited greatly from the input of Rory Greig (@rory_greig), who inspired me to write this post and provided valuable feedback and additions that help improve this post. Also, thank Jenna Marshall for feedback.

Suggested resources for learning more about collective intelligence

Besides the articles and books cited in this post, below we list some resources for learning more about collective intelligence

• The wikipedia page of collective intelligence. Contains a detailed summary of research findings in various application domains.
• Bak-Coleman, JB. et. al. (2021) Stewardship of global collective behavior, Proceedings of the National Academy of Sciences, 118(27) (Link). This recent article lays out a summary of recent findings in the field, and discusses connections of collective behavior with challenges of the long-term future.
• J. Surowiecki (2004) The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations. A great introductory book for many phenomena around collective intelligence. It is a bit dated, so does not contain the most recent research results.

References

• Almaatouq, A., Noriega-Campero, A., Alotaibi, A., Krafft, P. M., Moussaid, M., & Pentland, A. (2020). Adaptive social networks promote the wisdom of crowds. Proceedings of the National Academy of Sciences, 117(21), 11379-11386. link
• Aminpour, P., Gray, S. A., Singer, A., Scyphers, S. B., Jetter, A. J., Jordan, R., & Grabowski, J. H. (2021). The diversity bonus in pooling local knowledge about complex problems. Proceedings of the National Academy of Sciences, 118(5). link
• Bak-Coleman, JB. et. al. (2021) Stewardship of global collective behavior, Proceedings of the National Academy of Sciences, 118(27).  link
• Becker, J., Brackbill, D., & Centola, D. (2017). Network dynamics of social influence in the wisdom of crowds. Proceedings of the national academy of sciences, 114(26), E5070-E5076. link
• Centola, D., Becker, J., Brackbill, D., & Baronchelli, A. (2018). Experimental evidence for tipping points in social convention. Science, 360(6393), 1116-1119. link
• Dunbar, R.I.M. (1998), The social brain hypothesis. Evol. Anthropol., 6: 178-190. link
• Galesic, M., de Bruin, W. B., Dalege, J., Feld, S. L., Kreuter, F., Olsson, H., & van der Does, T. (2021). Human social sensing is an untapped resource for computational social science. Nature, 1-9. link
• Gilbert, D. (2013). The surprising science of happiness, TED Talk. Link
• Harari, YN (2015). Why humans run the world, TED Ideas. link
• Hong, L., & Page, S. E. (2004). Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proceedings of the National Academy of Sciences, 101(46), 16385-16389. link
• Jayles, B., Kim, H. R., Escobedo, R., Cezera, S., Blanchet, A., Kameda, T., ... & Theraulaz, G. (2017). How social information can improve estimation accuracy in human groups. Proceedings of the National Academy of Sciences, 114(47), 12620-12625. link
• Lorenz, J., Rauhut, H., Schweitzer, F., & Helbing, D. (2011). How social influence can undermine the wisdom of crowd effect. Proceedings of the National Academy of Sciences, 108(22), 9020-9025. link
• Mann, R. P., & Helbing, D. (2017). Optimal incentives for collective intelligence. Proceedings of the National Academy of Sciences, 114(20), 5077-5082. link
• Navajas, J., Niella, T., Garbulsky, G., Bahrami, B., & Sigman, M. (2018). Aggregated knowledge from a small number of debates outperforms the wisdom of large crowds. Nature Human Behaviour, 2(2), 126-132. link
• Krafft, P. M. (2019). A simple computational theory of general collective intelligence. Topics in Cognitive Science, 11(2), 374-392. link
• Page, S. (2019). The Diversity Bonus. Princeton University Press.
• Prelec, D., Seung, H. S., & McCoy, J. (2017). A solution to the single-question crowd wisdom problem. Nature, 541(7638), 532-535. link
• Riedl, C., Kim, Y. J., Gupta, P., Malone, T. W., & Woolley, A. W. (2021). Quantifying collective intelligence in human groups. Proceedings of the National Academy of Sciences, 118(21). link
• Surowiecki J. (2004) The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations.
• Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N., & Malone, T. W. (2010). Evidence for a collective intelligence factor in the performance of human groups. Science, 330(6004), 686-688. link
• Xie, J., Sreenivasan, S., Korniss, G., Zhang, W., Lim, C., & Szymanski, B. K. (2011). Social consensus through the influence of committed minorities. Physical Review E, 84(1), 011130. link
• Yang, V. C., Galesic, M., McGuinness, H., & Harutyunyan, A. (2021). When do Social Learners Affect Collective Performance Negatively? The Predictions of a Dynamical-System Model. arXiv preprint arXiv:2104.00770. link