Contration: The next threat from AI may not be like the risks we’ve feared

Concerns about the risks from artificial intelligence tend to focus either on current issues or long-term existential ones. But there is a threat that falls in-between. “Contration” refers to competition between autonomous programs or devices. It’s already seen in embryonic forms like the targeted ads and social media bots that compete for our attention, but it could escalate into more unwieldly varieties. Paradoxically, this cyber-vying is not only triggered by AI’s scope and competition, but by our defenses. 

A tiny part of this risk is glimpsed in the havoc following a CrowdStrike cybersecurity update. But there’s a bigger concern that needs to be recognized to avert the threat of safeguards and governmental regulations becoming rampantly counterproductive.

That danger is compounded by the hidden aspects of AI’s black box processing and our conflicting needs. Our need to know underlying details conflicts with our need to offload increasingly complicated processing to autonomous agents. That clash makes the alignment between AI and human values even more tenuous. 

But another concern is less obvious. It’s the relationship the intelligent agents have with one another that’s epitomized by the arrival of generative AI worms. Dare we forget, GenAI was built on adversarial dynamics. Competition is endemic to it.

*****

Futuristic storytelling typically frames the most fearsome risk as AI galvanizing its myriad entities against us. But what if the AI battles start among the programs and devices in competition with one another?

For network administrators, that might sound repugnant. But is it too far-fetched to consider that prospect when competing groups program their agents for AI-generated optimization? Given the many ways to manipulate generative AI, it’s no wonder hackers are already automating those techniques to use against other large language models (LLMs) in automated adversarial attacks. Could AI find ways to sidestep standard restrictions and usurp administrator access? Could it make privileged statuses more dynamic and escalate them into a war of default settings?

Competition is of course a hallmark of the free market, but weights and balances matter. When we program AI to be aligned with us, but we are not aligned with one another, what does it learn? We don’t want AI to mimic, let alone magnify, our rivalries and polarization, but it already does. Fortunately, we can all agree on our contempt for hypocrisy. Unfortunately, AI might also learn from our inclination to be hypocritical. If so, it might not be enough to emphasize our benevolence. Even hypocrites do as much.

At the very least, deep learning needs to go deeper than virtue signaling and superficial expressions of prosocial behavior. We cannot afford to be satisfied by glib expressions of AI merely imitating emotions and human virtues. Without integral formation, AI could become similar to those humans that persuasively mimic empathy but don’t have it. They’re called psychopaths.

*****

We’re defensive-minded, so consider what self-learning AI can pick up from our tactics and strategies. Military and intelligence tactics range from decoys, spoofing and misdirection to flank attacks. There’s misinformation and the soft power of propaganda. Even if the public is paying more attention to surveillance and privacy issues, program leaders have increasingly considered offensive approaches in cybersecurity. There are also the subtleties of social engineering, synthetic media and sophisticated hacking techniques like video-based cryptanalysis. What happens when these are automated? 

Like autonomous weapons, we can’t ignore the threats from aggressive autonomous operations by simply refraining from using them. But, if the agency we give to AI goes sideways, could it evolve into autonomous agent-to-agent cyber-warfare?

*****

Cybersecurity emphasis has understandably given weight to vulnerabilities that lead to direct human harm. So, could intelligent autonomous agents emphasize targets that have the best odds for keeping their attacks under human radar? 

The “incubation period” would increase their virality. It would be genetically selective.

The stealth would also be apropos for AI-to-AI black box battles. Even so, the obscurity may not last when GenAI programs compete with one another. If one autonomous agent’s intrusions could raise another LLM’s temperature (a setting for creativity versus accuracy), could it then induce hallucinations? If so, add another term to your cybersecurity lexicon: Hallucination-injection attacks. 

There is already evidence that hallucinations are inevitable with LLMs and popular techniques like Retrieval-Augmented Generation (RAG) are helpful but insufficient. Some tech leaders are dismissive about the fabrications. But, when misinformation proliferates, shouldn’t we be more – not less – stringent about limiting hallucinations? Most tech leaders love to tout exponential change and inflection points – except when they’re detrimental. 

Data poisoning programs like Nightshade are also expanding in their capability to skirt defenses, leverage anchor images and generalize their impact. Could the next generation of these programs have unintended consequences on a global scale? Could they reach a tipping point where they metastasize and unleash a pervasive infection?

When generative and degenerative dynamics can be inverse operations of one another, similar questions could be asked about generating everything from sensor malfunctions to multidimensional Distributed Denial-of-Service (DDoS) attacks. Denoising could take on new meaning amidst the computational cacophony. 

Some of this scenario is based on how condensed complexity increases volumetric pressure and the rate of collisions. But some critics may object, saying there’s an open sky for GenAI. However, near the operational surface the constraints can force the density to reach a critical mass. 

Others may object, and say there are natural correctives. Fire that’s contained, smothers. Over-crowded biological organisms, self-cull. Yes, that’s true – unless the pressure is released by a leak or an explosion. Then you’ll have proliferation and havoc.

Consider how cybersecurity defenses have heightened not only because of the volume of attacks on any given platform or node of vulnerability but through a proliferation of those systems and data entry points. Lots more low-hanging fruit for vectors to attack. Just as CS (computer science) has permeated nearly all other disciplines to become CS+, so will cybersecurity. But, as a parallel to the human immune system, consider the prospect of cyber-defense agents becoming hyper-sensitive and going on the offensive. 

AI will then have an autoimmune disease. 

*****

Our own biological systems are dynamically balanced among an array of subsystems, symbiotic relationships like those with our microbiome, and interactions that can alter the expression of our coding. One element might be mediated by multiple mechanisms before its impact is felt. In like fashion, multimodal machine learning will combine with the expanding elements of the Internet of Things (IoT) and the decentralized Web 3.0 to offer more dimensions for the bots to bait, mate and obliterate one another. 

The AI can also be analogous to a genetically based medication that alters the body’s biochemical reactions. But what if that medication progressively altered itself while altering the body? That will be the case when AI eats itself and overindulges in the synthetic data it generates. 

Research has shown how that process can collapse the model, magnify the polluted data and make it go “MAD.” But given the proliferation of GenAI, the effects will not remain static. In a dynamic context, the amplified artifacts and biases will clash.

Seen within a genetics framework of an autoimmune disorder, the scenario will be akin to inbreeding a cyber-recessive disorder. The ensuing vulnerabilities, dysregulation and dysfunction will be bad enough. But if low genetic diversity can reduce hardiness and increase the risk of cancer in the human body, imagine what a similar condition will do for our world’s body of data.

Once in that state, will AI be able to cure the problem it created? Not likely. We’ll need to take the initiative with well-executed tools like orthogonal neural networks and LLMs used as optimizers. Character training can also make a difference. 

But if we wait too long, the real fun will start when Automated Reinforcement Learning (AutoRL) and quantum computers get in the game. No longer limited to covert AI clashes, the attacks will be bewildering and insufferably hard to contain. To the general populace, they may even seem magical if not diabolic.

*****

Even if some of these prospects are statistically unlikely, it doesn’t relegate them to being insignificant. As we’ve seen with events ranging from a pandemic to the generative AI revolution, something that seems improbable can suddenly hit an inflection point and have explosive impact. 

But emergent behavior is not always so mysterious – if we are attentive.