When we talk about AI safety and what’s at stake, the discussion usually centers around three categories: misuse, misalignment, and systemic risk. These are valid concerns about likely irreversible harms. However, while we still speculate on the arrival of AGI and ASI, there is something that’s softly and steadily pulling the carpet from under our feet, so to speak: epistemic erosion.
Understanding the world/one’s existence is an individual responsibility. It is a heavy responsibility, but a necessary burden. History has shown us time and again that when a group is denied or relieved of this responsibility, disempowerment follows and persists. In India, Dalits were never allowed to learn to read and write. It was a privilege, and it was considered the responsibility of the Brahmins/priestly class to learn and understand. The impact this had on both these classes of people is centuries old and still very evident in India. The Brahmin caste became the "uppered caste" by the very act of monopolizing knowledge, and the Dalits ended up being "the lowered class" because they were denied it. That's why every strong democracy and every struggle for freedom has prioritized the education of its citizens.
As AI becomes smarter than humans at economically valuable activities, the systems in which we inhabit are less and less being run on individuals’ thinking energy. The systems are being automated. From organizations and businesses to education and entertainment. This isn't misuse: no bad actor is required. It isn't misalignment: the systems may be doing exactly what we asked. It's closer to what Kulveit et al. (2025) call gradual disempowerment. However, my focus is the individual mind: a channel underlying the economic, cultural, and state channels they describe.
But, what does it do to the individual? As AI becomes increasingly capable of generating, interpreting and acting on information, there is a risk that humans will increasingly delegate not just the execution of their goals, but the cognitive work of deciding what is worth knowing, pursuing and imagining. Social media algorithms already do some of this by shaping what we attend to and, consequently, what we desire. Instead of AI being used simply to achieve human visions, humans may increasingly become local executors of visions shaped by systems optimizing for engagement.
Information being readily available everywhere does not necessarily mean that individuals are more capable of understanding the world. The question is therefore not simply whether humans will have access to knowledge, but whether they will continue to take individual responsibility for understanding. Let's look at our education system - institutions which I believe are responsible for developing epistemic agency in young individuals. We are still wiring students for clerical tasks and automatable roles, or at best for creating economic value and influence. In the last few centuries, one of the major factors shaping education has been the economic value that an educated individual provides to society. Yet, it is still quite a lot aligned to human values and needs. But AI tutors are becoming increasingly integrated to leading education systems, albeit for human curated curricula. But the curricula itself is being reconsidered as the larger systems are undergoing rapid changes. Whether education system can maintain their role as the primary institution through which societies develop human capability is under question as AI assisted learning is economically viable and easily accessible. (Shaping the Future of Learning - WEF)
Basically, when humans are no longer needed for economically valuable cognitive tasks, then what is education for? And who determines it? This is a pressing problem and I would like to initiate the conversation on the need for capabilities, institutions and forms of human formation that are necessary to preserve human epistemic agency in a world where machine intelligence is becoming substantially superior to individual human intelligence. The risk is not only that humans lose economically valuable work. It is that humans may gradually lose epistemic agency: the capacity and habit of independently constructing models of reality through questioning, critical thinking, and meaning-making grounded in lived experiences. AI may gradually become the authorized producer and interpreter of knowledge, while we become dependent upon it for access to knowledge.
Photo by Isaac Davis on Unsplash
