TL;DR: I think embedding new technologies in our lives too fast, and without analogue alternatives, could lead us towards an unrecoverable dystopia. I share my observations and concerns in this post and look forward to reading your views on AI’s impact on your community or surroundings. I use “AI” as an all-encompassing term for chatbots and generative models.
[Transition risks] arise during a transition to a new technological or social regime. [...] Rushing the transition may do nothing to lower these risks—indeed it could easily heighten them. But if the transition is necessary or highly desirable, we may have to go through it at some point, so mere delay is not a solution, and may also make things worse. The general prescription for these risks is neither haste nor slowness, but care and foresight. (Ord, 2020: 209)
I am in The Precipice reading group, and it gave me a new lens through which to look at the increasing entrenchment of tech in our lives. To me, the rapid adoption of AI is a transition risk. Some use cases and early studies show risks to cognitive capacities and communities. Meanwhile opting out from AI use is already penalised in some fields, and we are running towards making it quasi-impossible.
Ord’s quote, from the book, reflects my worry. This transition is happening too fast, without knowledge about its future social consequences, that go well beyond job displacement. I am afraid we are making ourselves dependent on technology, as a systemic practical obligation.
In an AI-first future, cognitive decay may be widespread because of AI dependency. First because of the deployment of applications in all areas of life, and second because AI natives will grow up without alternatives. To me, it fits Ord’s unrecoverable dystopia and the second category of Bostrom’s flawed realisation: the ephemeral realisation, where mature technological development starts by providing value, but degenerates because it is unsustainable.
In terms of scale, I think it could turn current societal problems into an existential-risk. I would say I am fairly confident about the visible trend, but I would allocate a low risk of irreversibility as of now.
In the 2000s, any institutional or administrative procedure required a phone number and email address. Then the smartphones, in the 2010s, incrementally became prerequisites to getting by in urban areas. Staying offline was extremely impractical.
We are now in the 2020s, AI is following the same pattern, but faster. The perception of enhanced productivity contributes to AI being pushed in many areas of life, such as the workplace and in education. The automation of redundant tasks and the tailoring of teaching to students’ needs seem brilliant applications; if one disregards some observed negative impacts (mental and practical), such as psychosis cases, burnout, but also the time-consuming task of verifying outputs’ inaccuracies, for instance in meeting reports. Interacting with AI is increasingly forced onto people as workers and students, but also as consumers, as citizens, etc.
A downward spiral started. Technological tools are now used to accelerate flows of information, as well as to craft them. First offloading, then substituting thinking.
When individual performance metrics are compared to peers’, AI becomes a perceived ally to boost chances and opportunities. A pay raise goes to whoever “ships” fastest and someone’s access to a scholarship strict numerus clausus may depend on it as well.
Similarly, I like using my voice when writing (as you have heard throughout this post), but my unsuccessful job search has forced me to automate emails and parts of the process so I could send out more applications. Open vacancies receive hundreds of expressions of interest, so recruitment teams use AI tools to fast-forward through them.
It means we may be individually averse, or using it as reasonably as we can, but the collective effect intensifies the noise.
In life, growth comes from recognising our weaknesses and improving by addressing them over time. When access to AI is given before the end of this formative period or when tasks seem too inconvenient, people will opt for an AI-enabled cognitive offload. While a recent study showed a correlation with reduction in critical thinking skills, it is just one of the many symptoms.
I reject the parallel made between offloading thinking to AI and calculating with machines or researching with engines, because the former threatens to take over in all sections of our life rather than a limited amount of it:
A mathematician, or economist, will still know or create the formulas, and run demonstrations off the calculator. A researcher will still gather theories, compare existing data, write interpretations and conclusions. But if thinking is offloaded for planning a holiday, cooking a meal, drafting an essay, deciding who to vote for, it means it touches all parts of life and limits the development of independent mindsets.
A related phenomenon I have observed among my peers (and, to some extent, myself) is how difficult it is to start a project from scratch and carry it through independently without external support. This is, in my opinion, the biggest challenge of young people's AI dependency. A few months after my initial observations, I have the bitter-sweet feeling of discovering a study covering how LLM use reduces persistence.
Studies show that poor mental health is a strong predictor of AI dependency. Young people often overlook LLMs mistakes. Youth, and in particular vulnerable groups, lack the resilience to spend emotional or cognitive effort addressing an issue or answering a question.
In AI safety, interventions touch on existential and catastrophic risks, such as misalignment and extreme misuse. In society, the concern is critical thinking. What I fear is a fast entrenchment that will prevent the disentangling from technological crutches in the long-term, in turn decreasing cognitive functions and fragmenting communities. It would be another existential risk to add to the list, for which humanity would not go extinct, but stagnate.
Sweden, an early adopter of digital learning, is reversing to handwriting and books because of the negative outcomes of trials in the past few years. This change was possible because teachers still had the skills and materials to train them. The Swedish education system is recoverable because it is taking action early and will proceed with enhanced foresight to ensure technology is incorporated in schools appropriately to develop youth’s skills.
If the widespread AI applications are overflowing society, persistence will reduce and AI natives might grow without sharp problem-solving skills. That would create a system where turning back on trials is no longer possible (because AI adoption is still in its trial period), we might reach Bostrom’s “flawed realisation” (2013: 21).
Literacy rates are decreasing, and I have heard colleagues from different countries in the world struggling to make their students read and think for themselves. Once entire generations have succumbed to offload their thinking, there could be no undoing. There could be no reversing to a kind of thinking that is seemingly slower and harder.
Concerns about critical thinking are missing problems that have to be addressed preliminarily. Initiatives for youth emphasise teaching how to prompt better, optimise interactions or be critical about outputs but do not provide platforms to raise other concerns about youth’s experiences with AI.
I believe people should have a choice instead of being isolated or reducing their chances in life. Administrative processes in cities, governments and corporations should maintain the right to contest certain decisions, with a human pathway that could overrule decisions.
Similarly, tackling new tasks or problems independently is a skill that needs to be taught before accessing LLMs. Parents and educational institutions could receive guidance on how to facilitate this development. Following the example of a Dutch study on how offline play improves digital literacy, we could introduce non-digital learning to build transferable skills. Schools could require students to undertake personal projects each term.
This could encourage hobbies and build emotional resilience. Initiating offline activities can support the development of autonomy in a tangible setting where errors can be seen easier and physically corrected. The fear of making mistakes would be reduced through iterative improvement of experiences in a real-life scenario.
What I mean is: implementing solutions that seem revolutionary without adequate testing, nor preparation, may trigger a lock-in, not of values, but of a system in which self-extraction is impossible. Thus, preventing humanity from changing its course in the future.
Making technology a facilitator (if you accept the conditions) or a gatekeeper (if you opt out), could push humanity towards a mistaken path. Maybe not down a precipice, but down a cenote (or an aven, depending on where you are in the world) without climbing rope to come back up.
I am open to changing my mind on the issue of forced dependence if the benefits for communication and mutual understanding were to surpass previous technologies, while people’s cognitive functions would not be affected.
Maintaining critical thinking and epistemic plurality in a population means, to me, the safeguarding of humanity’s potential while we seek existential security and await the Long Reflection.
Just a quick comment: When you throw stone in water it unsettles for a while, but with AI it is difficult to track the number of stones and there seems to be no settling down. Considering the examples though in sandboxed environment such as AI pretending to be blind and hiring someone to to fill catcha to enter a website, or hallucinations to make up unexisting facts, shows the unpredictable nature of AI. Even AI companies have not been able to predict the profit they may gain from deploying AI. Sustainable AI is a dream today without concrete solutions on how to achieve it!
Exactly, my point here is that there are no solutions, just quick fixes, to the problems arise because of AI.
To use your metaphor, I am advocating to start putting the rocks in the water in a more orderly manner, being mindful of the fish and not getting anyone hurt, and towards a goal, maybe making a walkable path, rather than just trying to deviate the river.