This paper seeks to provide an oversight perspective into the nature of agency, the origin of intrinsic drives (aka. core instincts), and the morality of interaction between agents, their goals, and their societies. Unlike more basic descriptions found commonly elsewhere, this paper places stronger emphasis on the existential aspect of agency, goals, and goal alignment. Specifically, consciousness and self-transcendence are explored -- less from a spiritual standpoint, and more through a psychologic and sociologic lens. Finally, findings and insights are compared against popular proposals for the alignment of strong artificial intelligence. For the main findings and conclusions, see the "Summary" section.
Intended audience: ethicists; sociologists; psychologists; existentialists; transcendentalists; AI researchers and enthusiasts.
While the text addresses several philosophic topics, no special prerequisites are assumed. Basic familiarity with biology, agency, and causality is recommended. This text assumes the truth of evolution by natural selection, and it assumes matter and energy to follow consistent patterns, particularly at macro scales. Room is made for strong emergence, but mostly the discussion revolves around weak emergence. Prior understanding of emergence is optional. Within this text, the words "goal", "imperative", and "will" may be used in the sense of both teleologic, or planned and volitional, as well as teleonomic, or tending and compelled. Familiarity with these terms is optional. Clarification is given where one meaning is preferred over the other. Again, no special background is assumed for the main message.
An agent is someone or something that has thought, movement, or tendency toward some aim or goal. Often the individual, or agent -- such as a person -- is viewed as an atom, or concrete, indivisible thing. And collections of interacting agents comprise systems, such as hives, colonies, and societies. But further perspectives can offer valuable insight. The agent, for example, may be viewed as itself a system of organs, or internal regions or subprocesses delegated with specific goals or aims. Moreover, each member of a group of agents may be viewed as an organ of a higher being, or higher agent. Within society, for example, each person may be assigned roles and imperatives, whether intrinsically (i.e. innately) or extrinsically (i.e. socially). And these roles and imperatives may then "give back" and support higher entities, such as families, organisations, colonies, and ecosystems.
Any such system, agent, or organ, at any level, may have tendencies and goals. And these goals may or may not align. We might say that agency, thus, is organised naturally into a hierarchy of emergent levels, or integrative levels. Let us call this arrangement the system-agent-organ hierarchy. Note that from each point along the hierarchy, there are two neighbouring directions: smaller, or toward the organs inside; and larger, or toward the system outside. Harmony between the goals of the agent and its organs may be termed inner goal alignment. And harmony between the agent and its parent system may be termed outer goal alignment. Both aspects of alignment represent health on some level. Specifically, inner goal alignment supports individual health while outer goal alignment supports systemic health. The latter is often called morality, as the core function of morality is keeping balance, harmony, and function -- aka. goal alignment -- inter-agently, or within the system of agents. Hence, the health of a system depends on the morality of its members. And perhaps amusingly, the health of an individual depends on the morality of its organs.
Metaphorically, transcendence is about changing perspective, or scope of consideration, away from being distracted with the trees, toward seeing the greater forest. In other words, transcendence is about stepping back to see the bigger picture. In more technical terms, transcendence is about moving toward more fundamental, permanent, or antecedent phenomena. For example, when distinguishing between a system's current state and its fundamental laws or behaviour, we might say that system behaviour transcends system state.
Within this text, transcendence is meant primarily as stepping away from instrumental goals or subprocesses, toward more intrinsic drives or tendencies. That is, we are stepping away from the means to an end, toward the end in itself. In the system-agent-organ hierarchy, this would mean moving upward or leftward -- from organ to agent; from agent to system; or even from organ to system. To use an analogy, consider a finger, whose movements transcend to the hand, further to the arm, the torso, perhaps the brain, and then abstractly to the combination of evolved traits, prior learning, and current circumstance. In this way, transcendence is about tracing to more permanent, more fundamental, or at least more antecedent aspects of life and purpose. Such considerations are sometimes referred to as seeking "higher meaning", where "meaning" refers to "what moves us", or fundamentally why we behave as we do. Thus, transcendence is about tracing the higher or highest why of a system's behaviour.
Within the system-agent-organ hierarchy, one might ask which direction is serving the other. Do the organs serve the agent, or does the agent serve the organs? We might consider this question from various places along the hierarchy. For example, humans, like most animals, derive benefit from their surrounding ecosystem while simultaneously giving back in the form of carbon dioxide, intellect, and plastic. On a more stellar level, humans may eventually help spread the seeds of "lower" lifeforms to other celestial bodies. Or perhaps it will be humans' silicon successors. In any case, symbiosis appears to exist across the system-agent-organ spectrum. "Lower" entities support higher entities in base function while higher entities support lower ones in organising and planning. If we go small, for example, enzymes enable cellular metabolism and base function while cells produce and manage enzymes. Yet even smaller, it would seem perhaps that subatomic waves enable subatomic particles while particles give order to waves. The same general pattern appears to exist throughout the hierarchy. Specifically, it seems that each direction serves the other.
Let us consider a more conventional example -- babies. Do the parents serve the child, or does the child serve the parents? On a surface level, it would seem each serves emotional or physiological needs of the other. Yet the overarching biological imperative, if we trace it back, seems to serve the will of nature to find, extract, and release potential energy. This tends to occur through the evolution of replicators, or individual but diverse energy extraction machines. Some might even say the core aim of life is to increase systemic entropy. Does this mean the moral decree comes ultimately from below? Let us see.
On a quick aside, in the following paragraphs, by "reducible", we mean that certain higher-level, more complex, or more intelligent behaviours can be logically explained by the cumulative effect and or interaction of lower, simpler behaviours. This type of relationship, where simpler, more concrete properties and behaviours logically build up to enable and support more complex or abstract properties and behaviours is called weak emergence. In contrast, strong emergence refers to the irreducible appearance of properties and behaviours, or those which do not follow logically from simpler ones.
In the above three examples, it would seem there is something of a trade-off, or mix-up, between levels of the system-agent-organ hierarchy and whose goals are being served. There appears to be a tendency toward reducible, or bottom-up, origin of goals and imperatives. Yet it may be difficult in practice to rule out strong emergence, or the irreducible appearance of "new" (as in previously unobserved) properties and tendencies within a system -- especially at higher levels. As an analogy, we might view strongly emergent features as hard-coded exceptions written into a simulation, where for example certain complex patterns of matter are anointed with new, irreducible functionality. But does this hypothetical possibility even matter?
If we go back to the basics, we see that matter appears to behave in fairly predictable ways. Sure, those ways may include quantum indeterminacy. But nevertheless, it seems everything follows predictable patterns, even if those patterns are on some levels peculiar. Back on the question of nature and the origins of goals and imperatives, any observed tendency, or "desire", of sets of matter in particular configurations logically comes back to the origin of those "laws of nature", or underlying physics, behind those tendencies or behaviours in question. And here is the rub: whether a feature or tendency can be traced back reducibly, or whether it appears only in special patterns of higher configuration, that feature still ultimately serves the transcendent origin of any and all features and tendencies. This sourcing of underlying imperative, or basis for any moral "ought", would thus follow back equally for all three of (1) core observable physics, (2) weakly emergent properties, and (3) strongly emergent properties. All would serve that core, seemingly untouchable, transcendent essence of shared origin and imperative. Hence, it would seem that neither "end" of the system-agent-organ spectrum fundamentally controls the other. Rather, that invisible, merely inferred noumenon (or "thing-in-itself") of highest transcendence would seem the thing ultimately being served. Any other description would be incomplete, requiring truncation of the causal chain.
Despite the apparent logical unity toward that set of natural tendencies of the universe -- in serving that untouchable transcendent origination of configuration and will -- from a practical vantage, the apparent will of the two ends of the system-agent-organ spectrum can seem to possess distinct or even misaligned goals and imperatives. Smaller, shallower structures of energy extraction and reaction tend, absent of higher control systems, to burn fast and carelessly. An electric battery in short-circuit may expend its energy the fastest. But connected to a vehicle, it may live long enough to see another charge. Within evolutionary unfoldment across the universe, there appears to be a pervasive tendency toward higher precision and prudence, entailing deeper, farther, often longer energy extraction and release pathways. The specific nature of this tendency depends on the integrative level of consideration. Let us behold four levels -- micro, macro, mental, and moral.
From observing these four levels -- micro, macro, mental, and moral -- it would thus seem the natural evolutionary tendency of life is toward greater complexity, exploration, abstraction, and harmony.
Some readers may be familiar with a field of study having similar sounding proposed findings of large-scale tendency: orthogenesis, or "progressive evolution". Orthogenesis is the name given to a collection of mostly old theories about evolution, many of which at odds with, or opposed to, Darwinian evolution by adaptation through natural selection. Many of these theories make proposals quite teleological, as if certain ideal forms or ends are sought -- perhaps even by intelligent design. Before we proceed, let us take a moment to make a distinction. These theories focus primarily on the organism's form and function. In contrast, this text, with the above four key tendencies, focuses primarily on goal origination and alignment. Moreover, this text is concerned not with ideal forms or predestined ends, but rather with drives, goals, and morality. Nevertheless, for those interested, one noteworthy proposal is from D.W. McShea (1998), which preliminarily identifies the following eight trends:
[increased] entropy, energy intensiveness, evolutionary versatility, developmental depth, structural depth, adaptedness, size, and complexity.
For probably most grown humans, within the subjective experience there may exist senses of wanting, thinking, choosing, and doing. Depending on various personal and situational factors, the person may be so immersed in the passions of life that little if any reflection is given to the origin or nature of one's desires and aversions. Especially when anxious or frustrated, the mind has a tendency to "zoom in" on the feelings and thoughts of attraction and repulsion -- constant judgements, along with perhaps the thought and sensation of judgement from others. This onslaught of emotion can easily obscure the hiddenness of the source of that emotion, or drive, to and from all those thoughts and perceptions. If we think about it, does having a subjective experience really stop humans from following their evolved nature, which like all other lifeforms, seeks within its power to find, extract, and release more energy? Does the complexity of the personal plan, or the abstraction of language, change the core, common will which all serve?
One of the key functions of consciousness is social awareness and interaction, particularly language and self-consciousness, such as planning what to say, or following social norms and expectations. If we think about it, is worrying about our impression on, or judgement from, others not among the most frequent applications of conscious veto, or the ability to suppress urges and other behaviours? With this in mind, what function, in turn, does socialising serve in the bigger picture? It seems perhaps that its core role is to form coalitions, or aggregates, of individuals into groups. That is, socialising, to which consciousness seems intimately tied, appears to exist to make bigger beings, or superorganisms -- hives which more effectively extract energy, in greater precision and from vaster regions of substrate.
Self-transcendence is about seeing and embracing the unchosen and impersonal nature and origin of will and worry -- letting go of the earlier conceived notion of serving oneself alone. After all, if one's fundamental imperative is not only of external origin, but apparently also natively toward higher aggregation through the evolved feature of consciousness, then is there really any other option? One way or another, the human serves something greater. Sure, denial is possible, and indeed quite common, but is that denial anything other than a mix of confusion and the instinct of trying to prove something?
The underlying universal values of enhanced energy extraction complexity, exploration, abstraction, and harmony are not universally embraced culturally. This incongruence is understandable, however, if we consider the nature of the pieces, how they themselves develop, and the limitations of existing industrialist and progressivist frameworks.
In perhaps the broadest sense, primitivism is about rejecting and reversing the cumulative hierarchical constructing of society and or technology. Effectively, primitivism seeks to avoid complex or hierarchical division of labour by keeping individuals and groups "self-sufficient", "all-in-one", or "jacks of all trades". This flat layout is not only about skills, but also tools and amenities. A core aim of primitivism is avoiding individual dependence on large or impersonal systems, which are believed to create alienation and lack of freedom. The primary values of primitivism seem to be independence, personal liberty, and free spirit.
In contrast, progressivism is about essentially open-ended advancement in the accumulation of empirical knowledge, building of technology, and development of social systems. In modern times, progressivism has placed particular emphasis on solving problems of economic inequality, corporate monopoly, worker conditions, and discrimination. Global cooperation, including toward managing resources, pathogens, and climate, has been another significant focus of progressivism. Further concerns have included human rights, animal welfare, and emotional health. Overall, the core values of progressivism seem to be science, technology, fairness, and health.
At first glance, the primitivist values of independence, personal liberty, and free spirit may seem quite reasonable. Yet if we step back and see how this plays out in real life, what becomes blatantly apparent is that sometimes it is not the presence of certain values, but rather the absence of others that brings trouble. Let us see some examples.
As can be seen with the above set of examples, primitivism has an overarching tendency to lack the depth of reasoning, exploration of explanatory frameworks, and interdisciplinary or interlogic harmony needed for reliable individual insight and proper systemic function. What may look like a good idea on the surface becomes in reality quite oppressive through its lack of understanding and lack of organising. True, its participants may not know better, being lost in the trees; but we have the oversight perspective to ascend.
On the topic of top-down, authoritarian control structures, as are sometimes seen in human society, one mistake we must avoid is thinking the leaders of such a structure are somehow equal to the structure or body itself, from which we might infer emergent will, goal, or imperative. This possible confusion is similar to confusing the face, words, or identity of a person with the whole person. Specifically, what a person says -- or even thinks -- about themselves is not always true, accurate, or complete. This applies likewise to what others think or say about the person. The words or expressed will of a body is only one small part of the total expression of said body. And the true will or imperative is of the whole body. For those unaccustomed, this idea may seem counterintuitive since in convention society is built and comprised of an abstract symbolic framework, where individuals are not truly bodies, but rather are nodes of the abstract framework held within each member's mind. On a related note, when we think of someone we know -- including ourselves -- we are really thinking the mental node that corresponds to the hypothetical person. No, we are not thinking of the node, but simply thinking the node. This is because thoughts, including of people, are merely representations appearing within consciousness.
Earlier we mentioned the general tendency of life toward greater abstraction and complexity, with an added dose of self-transcendence through the help of consciousness. Speaking of such things as aggregates, higher order, and superorganisms may evoke thoughts of authoritarian or totalitarian, top-down social structures. Such confusion, should it arise, is understandable. When we speak of systems being more carefully ordered, with perhaps more levels of abstraction, the idea of orderliness may be conflated with that of dictatorial control. The higher order "control" that life seems to seek, however, is rather different. On the surface, a human body with a brain "controlling" it may appear similar to a dictatorship with a leader controlling the people. But there are some very important differences:
As we can see from the above points, the human brain, unlike a human dictator, is more closely and automatically aligned with the will of the overall body. We might even say the brain serves the body, its organs, its cells, and even its genes. If we think about it, even the mind has a similar dynamic, where reason and thought serve emotion. Hence, the thing that speaks is a slave to, or even emergent property of, the unconscious forces that drive it. This is rather the opposite of a dictator in a dictatorship.
Back on the topic of higher order "control" and society, the type of organising and control it would seem life seeks is of the naturally emergent variety. This is fairly akin to collective intelligence, or gestalt consciousness, as might emerge from hives or flocks. Similar to how the human brain can be made of cells which self-organise and enable intelligence, there need not be any internal top-down control structure, nor any leader. Higher reason emerges from the whole -- as its own, "new" entity. Still, though distinct, its behaviour naturally serves the same core will -- even if that will is unknown to the individual members. As suggested earlier with respect to "direction of moral decree", all levels automatically serve all others, as all matter automatically serves the global transcendent origination. So, unlike in a dictatorship, higher order organisation can occur while individual members retain their individual liberty -- or at least what we may call "individual" liberty.
Despite the apparent unity of various system-agent-organ levels, along with adjacent members of a given level, in serving the same transcendent origination, the mental model, or epistemic subsystem, of a given entity is not always accurate or aligned with others. Often this is due simply to the chaotic, biased, trial-and-error way that world models and self models are formed. For practical purposes, this chaotic aspect of error and bias may be largely unavoidable, at least during an agent's early learning. But there is another common cause of troublesome models, known as reward hacking. This is where the instrumental subsystem, or the part of an agent that tries to find ways to satisfy goals and imperatives, "accidentally" discovers that we can "achieve" the desirable end by simply modifying or perverting the world model or self model to indicate a better world state without actually arriving at a better world state. In other words, satisfying goals is replaced with painting an illusory model where the goal is already satisfied. From an engineering standpoint, we might say that reward hacking short-circuit's the success signal.
Perhaps the most blatant version of short-circuiting the success signal is through substances or other addictions that stimulate directly, or nearly directly, the senses or bodily indicators for satisfaction. Naturally, without accurate signalling, the agent may fail to achieve its imperatives.
An often less obvious case of reward hacking is narcissism. Here, sometimes the general world model, but most certainly the self model, is corrupted by the agent's insatiable desire to maintain or elevate social status. Instead of doing valuable things, and letting one's own and others' models update accordingly, the narcissist seeks to manipulate and deceive the model building faculties of self and other -- to paint a false but agreeable picture of oneself and one's objects of identification. Put simply, the narcissist tries to short-circuit everyone's signals. Within the social environment, having members short-circuiting their and others' signals can be expected to result in deleterious effects in systemic function and efficiency.
Addiction and narcissism thus appear to go against the universal tendency of life toward greater exploration and harmony.
From the perspective of the individual agent, the aim of life is to align environmental opportunity with personal potential. In the case of social agents, we might further distinguish between epistemic will, or believed desire, and actual will, or true tendency. This distinction relates to the earlier mentioned difference between identified leaders, or controller organs, and the actual emergent intelligence and behaviour of a system. That is, a mind or leader may express or declare certain epistemic will, but only the full body's behaviour can show actual will.
Whether a given action is said to have been taken in free will depends on (1) the nature of reality, (2) the philosophical class of "free will" in question, and (3) the alignment of said action with either (a) hypothetical actual will absent undue influence, or (b) epistemic will. Often the motivation for classifying actions as free or unfree is done for ascription of moral responsibility, which is ultimately a social construct. Sometimes, the question is not about individual actions, but whether the agent in general possesses a particular class of free will. This latter question, in practice, seems more concerned with meaning, or purpose.
Fortunately, as discussed earlier about higher order "control" -- as from higher emergent levels -- individuals may essentially go on living by their same main intrinsic drives. The type of systemic progress that life seems to seek appears, in practice, to emerge mostly from the usual continued refinement of scientific knowledge, technology, social structure, and individual self-awareness.
We might say that throughout the superorganism that is life and substrate, there exist will channels, or hierarchies of will delegation. At the level of the agent, will may arrive through multiple channels. In the case of an individual human, for example:
One of the common themes in the human experience of "free will" seems to be that the part of will that is installed by the environment is usually black-boxed, or taken for granted, while the remaining portion of the decision process -- the instrumental pathway -- is open for review. Thus, the role of human agency seems to be in consciously devising ways to satisfy the received imperative. Each person has been provided a set of intrinsic drives, childhood conditioning, socialisation, and curated opportunity for which a viable pathway is to be found. Fortunately, this experience can be enhanced through the progressive development of upbringing, socialisation, and opportunity.
As far as human society goes, most of the changes needed to move forward and increase systemic harmony surround conventional technological, social, and individual psychological development. Higher order progress, after all, tends to emerge from lower order refinement. Still, there are some key areas that have not yet received universal uplifting. Many of these involve individual education and societal management of resources. Some nations are doing fairly well in these areas while others are far behind. Yet room exists for substantial progress in a great many places.
There is one matter, however, that seems particularly misaligned in maybe even most places today. This is the problem of higher level management positions, whether of business or government, being operated with lower level mindsets. Specifically, the type of "level" here meant is the level of system-agent-organ self-reflection and transcendental awareness. Consider the following. Say you have someone who has never taken the time to reflect back on their own upbringing, culture, and other influences to understand how and why they have become as they are, and not some alternative way. Say this person has never taken the time to meditate and self-reflect in any real depth on their own mind and decision process. The result would be a type of self-blindness, very much like that described above for the "black box" of will origin in the unreflected version of the "free will" experience. Here is the thing. Can a person who is blind to their own nature and the transcendental aspect of their will really make sound decisions toward configuring society for maximum will alignment?
When there exists potential energy waiting for release, all it takes is a little bit of chaos to find and replicate the right key for the job. Many organisms have evolved to exploit these opportunities -- whether as primary aim, or as secondary subgoal. This much is fairly well known. But there are other, analogous versions of this pattern within nature, including within human nature. One case is that of epigenetics. Another is early childhood emotional conditioning.
Nature's job placement goes further, however -- into the realm of meaning, morality, and identity. A fairly well studied yet not so widely known phenomenon is that of human ego development, along with its corollary, human moral development. These two bodies of research fundamentally investigate the same core phenomenon from two different perspectives. While the details vary depending on methodology, the common theme is that individual humans develop progressively through a sequence of relatively discrete stages of ego and moral reasoning. That is, the person's worldview, or understanding of the relations between, and fundamental meaning of, self and other changes through bouts of disintegration, or the emotional breaking down of old views to rebuild new ones. And this process, though deeply personal, follows fairly consistent patterns across individuals and cultures. Moreover, examination of the stages, their traits, and their themes uncovers important trends and tendencies. Let us see two such theories, both based on empirical data.
Lawrence Kohlberg's stages of moral development (1976-1981) investigated the types of moral reasoning employed by humans at various ages. The following stages were found:
The above descriptions provide only a brief gist of the stages. Note particularly that one does not arrive at a stage simply by reading about or understanding it intellectually. Rather, one must encounter through actual experience significant personal moral dilemmas which cannot be resolved via one's existing worldview of right and wrong, good and bad. The process of transition between stages often accompanies periods of personal existential crisis, or meaning crisis, at least for entering stages K4.5 and higher. The transition to K4.5 and K5 in particular often accompanies or comprises the quarter-life or mid-life crisis. Moving to K6 is rare, usually requiring particularly troublesome moral dilemmas. K7 is similar but more complete, where the existing personal worldview is found fundamentally flawed, sometimes irreparably -- thus requiring self-transcendence, or relinquishing the notion that one's will and values are of one's personal making.
Jane Loevinger's stages of ego development (1976-1996) investigated the ways in which personal human worldview changes throughout life. The following stages were identified:
As for the moral stages, the above ego stage descriptions offer only limited detail. Both speak of the same process, but viewed through alternate lenses. For our purposes here, and a big part of why these stages were brought up, it so happens that these stages act as yet another element of nature's job placement system. Specifically, individuals at each successive stage have respectively larger and longer scopes of interpersonal inference, or breadths of agentic causal awareness, as outlined here:
As shown above, throughout ego development, individuals accumulate key insights into what makes self and other tick, including what types of personal and interpersonal arrangement produce what types of outcome. After stage E5/K4, there are progressively fewer individuals at each successive stage. This creates a natural distribution of humans where many focus on the more local and shorter term while some focus on the more global or longer term. From a hierarchical, managerial perspective, such a distribution could make sense. As mentioned earlier about finding alignment, it makes little sense to have those with lower-stage mindsets overseeing public policy or other high-level matters. Thus, having a proper understanding of moral development and its evolutionary role could perhaps allow societies to be managed in wiser ways.
A popular argument against naturalistic accounts of morality is called the is-ought problem. This position, as brought by philosopher David Hume (1711-1776), suggests that is statements alone are insufficient for making ought statements. The basic idea is that no matter the scenario in question, whether or what "ought" to happen is ultimately arising extrinsically, or from the observer or beholder, rather than intrinsically, or from the scenario itself. This position seems to make sense from the perspective or assumption that the observer or beholder is inherently separate from the scenario in question. Yet, absent of dualistic accounts of agency, this position, in practice -- that is, beyond the paper -- may not truly apply to actual, real-world agents. With that said, the is-ought distinction can still serve as an important reminder to check our assumptions when making ought statements. Too often, as related to the self-blindness discussed above, have authors and others pushed ought statements, or ideas based on implied ought statements, without proper self-reflection toward will and belief origin.
One specific consideration is that not all "oughts" come as statements. For example, agents behave by a certain core operating procedure. Humans, for example, operate by a pervasive mode of cognitive dissonance reduction, where all concerted thought and action follows from trying to minimise the internal disagreement between beliefs, desires, and or perceptions. This happens automatically, without choice or practical ability to veto. This is the basis behind such games as "try not to think of a pink elephant". Because there is certain processing, or goal-directed behaviour, that happens automatically, there are implicit oughts snuck in uncontrollably. For example, even the very motivation and logical basis behind trying to argue a position -- such as suggesting that "ought" may not derive from "is" -- is incontrovertibly tied to implied, likely shared "oughts". Hence, the act of arguing that is-ought is a problem may be inherently contradictory.
The universal transcendent origination of will acts like a giant octopus with many arms reaching out and branching off through hierarchies of emergent beings, or aggregates of agents. In this way, the universal transcendent provides stable, semi-stable, and fluid will through various channels. These include through genes, conditioning, and social-symbolic interaction. Through a process of chaos, trial-and-error, and the butterfly effect, endpoints receive random embodied theories, or novel energy extraction instructions. These may or may not work short or long term, but each has been assigned an essential part of the bell curve of possibilities, along with a random shuffling of circumstances. Each random assignment, though perhaps unique, is but a card in the universal transcendent's game. An agent within this paradigm may experience many instances of what may look and feel like free will -- including the deliberation of whether or how much to accept or reject a description of universal will and morality, such as this text. Curiously, however, even if one should reject such a description -- or even the very notion that will can be traced back to antecedent priors -- one is still bound to those drives and externally imposed circumstances which befell one. And all channels, ultimately, lead back to the same shared source.
Obviously our understanding of such abstract matters as universal life tendencies is limited and subject to amendment. But whether we embrace or reject that understanding makes no difference to the fact we must indeed live by the reality of whatever those tendencies be. Unlike purely removed or practically inconsequential philosophical questions, we cannot safely just ignore questions of broader societal direction, purpose, and morality. For if we did, the realistically expected outcome would be that shallower, more whimsical, more purely emotional or even delusional beliefs would take the place. So the matter is not about choosing or not choosing a moral framework, but about taking something imperfect yet logical versus letting the matter fall into the hands of unreflecting, likely ignorant mindsets. This is similar to the topic of religion, where whether or not we identify with a particular belief system, we are nevertheless following something. The question is how reflective we are about that something, and how fitting are the puzzle pieces.
One common objection to transcendent descriptions of agency is that if will and path are essentially "determined" by "external" factors, that no room exists for choice or improvement. This fear, fortunately, misses the mark. Agents operate by mental models of inferred reality. And these models are the result of life's encounters. Potentially insightful models or frameworks of universal will delegation are one such example of what an agent may encounter. Whether or not we feel like describing an agent as possessing free will or free choice, that agent is still open to reconfiguration for each new encounter. Freely willed or otherwise, agents within a system can realistically be expected to learn continually. And in the case of humans, as per the above mentioned system of automatic dissonance reduction, each mentally functional individual can be expected to learn continually and unavoidably. Hence, depending on perspective, there may or may not be individual choice, but ideas can nevertheless arise, and they can still propagate automatically through systems of agents. Individual and systemic change is thus inescapable.
The question of will and imperative can get tricky when we begin introducing agents of intelligent design. Unlike for organically arising or emerging agents, the intrinsic drives programmed into an artificial agent were generally not evolved specifically around the tasks of finding, extracting, and releasing potential energy. Instead, the assigned objective is usually a subset of relatively high-level instrumental goals from a particular, somewhat arbitrarily chosen organ of a human social system. That is, if we go back to the system-agent-organ hierarchy, we may recall that organs serve context-dependent subgoals of their agent. When humans design an AI, they are usually doing so to satisfy a set of human needs, which in practice are often only instrumental to those particular humans. If the AI produced is disproportionately powerful or obsessive, in that it fails or refuses to yield back control to its parent, then we may get a runaway instrumental subprocess, such as the classic paperclip maximiser which seeks insatiably to produce as many paperclips as materially possible, at the detriment of surrounding agents, organs, and systems. Put simply, when artificial agents are introduced, they must either be limited in power and perseverance, or they must align not with a subgoal of the greater system, but with the highest goals and imperatives -- above and beyond mere mortals. In other words, if the AI is to function toward specific instrumental goals, it must be limited in power and perseverance. This keeps it acting as an organ, rather than as a paperclip maximiser. But if the AI should serve one-to-one the universal transcendent, then it possesses the same core nature as its parent system, in that its introduction, no matter how powerful, cannot be expected to deviate from what life was moving toward anyway.
In practice, humans cannot be expected to find and properly understand on their own those transcendent-most goals and imperatives of life and universe. At best, humans may arrive at preliminary approximations. With the help of strong narrow AI, on the other hand, it may be feasible to begin exploring the will-space to uncover, at many integrative levels, the core intrinsic values, or core goals and imperatives, of the biosphere and beyond. The results of this discovery process could then be fed in as the intrinsic value set to an artificial superintelligence (ASI), whose degree of power and autonomy is set proportional to the lower-bound of the degree of certainty on the current set of transcendental imperatives. In simple terms, strong narrow AI would figure out what life wants on various integrative levels through time while its findings would serve as the will of strong general AI.
On the matter of aligning strong general AI, various researchers, enthusiasts, philosophers, and ethicists have made suggestions or proposals. Let us compare a few with the above preliminary suggestion.
Computer scientist Stuart J. Russel has proposed three core principles for the developers of AI:
- The machine's only objective is to maximize the realization of human preferences.
- The machine is initially uncertain about what those preferences are.
- The ultimate source of information about human preferences is human behavior.
This proposal about human preferences appears to follow or align with a philosophy known as preference utilitarianism. The basic idea of this philosophy is that preferences of sentient beings -- in the above, seemingly restricted to humans -- are to be maximised across time and society. A potential issue with preference utilitarianism is that it may place too much emphasis on what beings want right now, in their current stage of knowledge, wisdom, and moral development. If we take a society of warring factions, it does not take much imagination to see how maximising preferences could lead to an arms-race explosion, with excessive resources spent on offence and defence, rather than more peaceful possibilities. As another example, imagine a society where the majority follows old cultural superstitions, rather than caring for scientific inquiry. It seems conceivable that following preference utilitarianism here could result in tyranny of the majority, where common but fallacious beliefs would be amplified so as to maximise preferences per current norms. This comes back to the issue of the black-box of agentic will, or self-blindness, as described earlier. Specifically, current beliefs, even if mistaken, may be so deeply conditioned that preference alone picks the comfortable familiar over the inconvenient process of updating one's worldview. Another potential issue with assessing will at the individual agent level is that such will is often situation-dependent and instrumental. Since instrumental pursuits are essentially subprocesses, or organs, of the higher goal, detecting and amplifying these can be inherently imbalanced and hence misaligned. Without transcendental awareness, we might even create a runaway subprocess, or instrumental obsession, like the paperclip maximiser.
Philosopher John Rawls (1921-2002) proposed that the human sense of justice is based on the process and ideal conclusion of considering the whole of one's moral principles, along with the totality of specific encountered cases and judgements, and establishing a consistent, stable belief structure, termed reflective equilibrium. Similar to preference utilitarianism, reflective equilibrium works with what the agent knows or believes at present. But unlike preference utilitarianism, reflective equilibrium takes it further by requiring coherence between and among those beliefs and judgements held by the individual. Hence, instead of maximising potentially contradictory preferences as-is, first the preferences would be brought into agreement through reflection and revision. Nevertheless, while the addition of coherence between preferences seems beneficial, both of these proposals appear to be vulnerable to (a) magnifying existing ignorance and (b) overemphasising instrumental priorities.
These two issues -- magnifying existing ignorance and overemphasising instrumental priorities -- can perhaps be alleviated by becoming an ideal observer. According to ideal observer theory, true morality -- and by extension, true preference -- is what would be arrived at by a calm, impartial, fully-informed agent, termed the ideal observer. Essentially, this hypothetical entity would possess omniscience with respect to the matter in question. The earlier suggested strong narrow AI -- whose objective it is to figure out, at various integrative levels, what life is tending toward -- may be an example of such an ideal observer with respect to the preferences of life. This fully-informed entity or module would presumably have both the knowledge to dissolve existing ignorance, as well as the oversight to avoid instrumental imbalance.
AI theorist David Shapiro has proposed a set of three guidelines, called Heuristic Imperatives (HI), that are to be weighed together in a colloquial, heuristic manner -- or as would be interpreted within the social-cultural-linguistic context. These imperatives are proposed to be followed by any AI otherwise capable of causing significant harm. They are as follows:
- Reduce suffering in the universe.
- Increase prosperity in the universe.
- Increase understanding in the universe.
This approach, within the context of human society, is likely human-centric. Moreover, the prime imperatives of life are simply given, seemingly without the employment of any AI search and discovery. The focus of these imperatives seems to be coming from a particular, agent-centric integrative level. In a way, this is similar to preference utilitarianism, but with the added instruction of increasing understanding. Depending on interpretation, this last imperative may or may not satisfy reflective equilibrium and ideal observership. Overall, the lack of search and discovery of life's will, and the hard-coded integrative level, make this approach perhaps simple, but not necessarily fully aligned. Say, for example, these imperatives were misaligned in an important but elusive way. Would their presence within superintelligent systems prevent their being updated for fuller alignment? Might their application overemphasise particular instrumental priorities?
Finally, AI theorist Eliezer Yudkowsky has proposed a model similar to the combination of preference utilitarianism, reflective equilibrium, and ideal observer theory. This model, called Coherent Extrapolated Volition (CEV), proposes using a "seed AI" to establish basically what smarter, faster, more self-actualised, more interpersonally aware humans would want, given sufficient insight and time for reflection. Then, similar to that suggested earlier, the results of the "seed" AI would guide the will of the acting agent. Overall, CEV appears fairly comparable to the suggestion made earlier in this text, although CEV seems more human-centric, and less transcendental.
The questions of extrahumanistic, transhumanistic, and transcendental will extrapolation are curious. But unfortunately these are beyond the scope of this text. Nevertheless, it shall be remarked that any binding formulation of goals and imperatives ought to make room for all three of these possibilities -- if not now, then at least down the road. Failing to keep the intrinsic values of a system open to outer, later, and greater interpretations of life's will would be asking for overemphasised instrumental priority. Every stage of evolution could even be viewed as an instrument toward the next. To forget this would be like amoebas deciding they were the pinnacle of development, setting in stone evolution at that stage. Humans may be smart enough to arrest their own development, but is this really what greater life wants?
Goals and imperatives arise and emerge both consciously and unconsciously in both living and "non-living" systems. The way we divide matter and energy into systems, agents, and organs is arbitrary and subject to perspective. On lower integrative levels, life would appear to serve the expansion of entropy through replicator-mediated energy extraction. On higher levels, the path becomes more abstract and obscure. Still, and even for the hypothetical case of strong emergence, lowest through highest levels ultimately serve equally the same transcendent-most origin of will and imperative. In this way, all levels and all apparent beings are as outlets of the same shared will.
Yet within specific mental models or finite configurations, imbalance and disagreement may arise. Life, however, has a tendency to minimise these disagreements through higher-level restructuring and rearrangement of the lower pieces, so as to maximise systemic efficacy. Primary tendencies for life's behaviour -- on the micro, macro, mental, and moral levels -- appear to include greater complexity, exploration, abstraction, and harmony. Especially toward systemic, or multi-level, efficacy and harmony, transcendental awareness -- or tracing the channels of individual will origination -- seems key, for which consciousness appears instrumental. This type of awareness, to minimise self-blindness, may be essential in avoiding compulsory eugenics, totalitarianism, exploitation, and even narcissism.
The "is-ought problem" is at most -- if even then -- a problem for dualistic, or non-transcendental, accounts of agency. One way or another, whether reflectedly or blindly, we are going to follow some moral system or another. So we might as well use the best of the ideas and frameworks we encounter.
Bringing strong AI into existence comes with cautions. Among these include avoiding (a) imbalance of instrumental priorities -- such as the paperclip maximiser -- (b) amplification of existing ignorance, and (c) restriction or lock-in to a particular integrative level, species, or stage of evolution. One potentially viable solution to these problems is to have strong narrow AI investigate the will of life on various integrative levels and to apply its findings as the intrinsic value set of strong general AI.
First, this article is very well articulated, which is quite the feat considering the wide array of topics it covers. Bravo, FalseCogs. And the perspective of increasing entropy as the end-goal of life, instead of a detriment to it, is original.
I will start with what I agree with in this article. First, organizing society into organ-agent-system levels seems like a good start in organizing a myriad of phenomena that make up life. I agree that each of these levels serve each other, because if they did not, the higher levels would cease to exist. (Examples of this would be heart failure for the organ-agent level or the behaviors of solitary animals, like for the agent-system level. An example for the latter is the jaguar, whom is only social to mate. Because of this, they lack the survival benefits of being in a social group.)
Now, I have a few questions. Please correct me if these questions misunderstand your article
Thanks for the comment. It's nice to see someone getting something out of it.
On the topic of life having goals, it's not that the universe necessarily has an end goal, but that, like the seasons on Earth, each period (or spacetime region) may have a shared (observably universal) tendency, and those pursuits or actions which follow that tendency should flow most smoothly. Moreover, the goals and aims of human and other life already seem to follow this tendency. And the higher-level emergent aspects of this inter-level tendency seem already the basis for human moral and legal frameworks, though with a certain jitter, or error margin -- presumably due to the inherent entropy of human inference, coupled with the limitations of common human intellect and the limited scope of applicable consideration.
The key here is the inherent transcendence of what we're already doing, where in the long run, it doesn't matter what we may think or feel at a given point along the journey of evolution -- we're already and inescapably serving that tendency. If I or anyone else hadn't said this, someone or something else likely would have. It doesn't belong to me or anyone else, though I don't mean to suggest I have it described accurately. I see myself here as mere observer, though perhaps nothing at all.
On the topic of strong AI being able to exist, my stance is mostly based on my understandings of neurology and psychology, mixed with my subjective experience of non-doership and object-observer non-separation. Naturally I don't expect everyone to share this belief about AI. And of course it's just an assumption based on one mind's current limited reason and experience. The philosophical basis for the non-duality of qualia is curious, but I'll refrain from going there at the moment, particularly as it too seems at least partly based on assumption.
On the topic of the prospect of "anti-human" tendency being inferred, the answer comes back to that mentioned above, in that if so, then humans are already and inherently of and for that end, even if unknown or seemingly unwanted. Indeed this idea seems fatalist. But that doesn't necessarily make it false. Realistically, humans may be less likely to be determined "unwanted" as "made-to-order" for a particular purpose -- a purpose perhaps temporary and specific to an occasional or spacetime-regional set of conditions. Some humans, given such prospect, might find comfort in transhumanist or posthumanist ideas, such as mind-uploading, memory-uploading, or slowly merging into something else.