Bootstrapping Cybernetic Dark Forests
In a sufficiently large world, why would anything close to oneself be familiar at all? Why are we not plagued with chaotic novelty from the get go? We are impossible objects surrounded by impossible objects. A manic maze of mirrors.
Dark Forests.
In Liu Cixin's Three Body Problem trilogy, the dark forest hypothesis is introduced at the start of the second book by the character Ye Wenjie during conversation with the main protagonist Luo Ji. In an attempt to arm Luo Ji with knowledge she believes useful in defending earth against an imminent alien invasion, she introduces three key axioms to Luo Ji as to push him towards establishing a new field of science she describes as "cosmic sociology". These three axioms are as follows:
"Suppose a vast number of civilizations are distributed throughout the universe, on the order of the number of observable stars. Lots and lots of them. Those civilizations make up the body of a cosmic society. Cosmic sociology is the study of the nature of this super-society." (based on the Drake equation)
"Suppose that survival is the primary need of a civilization."
"Suppose that civilizations continuously expand over time, but the total matter in the universe remains constant."
This exchange was found to be of great significance later in the novel as Luo Ji uncovers how these axioms provide an adequate foundation for solving Fermi's Paradox and establishing mutually assured destruction between humanity and the malthusian invaders.
The dark forest hypothesis posits a universe teeming with a countably infinite amount of civilizations, each driven by the primary need for survival and expansion, thus wary of others who may threaten their existence. The resulting environment is that of caution and self-enforced undetectability. The inevitable outcome is a metaphorical "dark forest" in which each civilization is like a hunter, hiding from others while keeping an eye out for potential prey. The moment you suspect the other knows you exist, you must attack. Otherwise, remain silent.
Cybernetics, Language Models and Malthusian Collective Intelligence.
Recent developments in generative machine learning have opened up the possibility of creating high-quality, human-like text that can potentially mislead humans and pass the Turing test. Generative language models are designed to generate text by learning the contextual statistics of large corpuses of writing. When trained on datasets sourced from across the internet, these models are capable of producing coherent and contextually appropriate responses to given prompts to the point that worries of an imminent A.I uprising is on the radar of powerful individuals. These models have been demonstrated to scale effectively, with larger models and datasets producing increasingly impressive results. There is much research showing the continual emergence of more sophisticated behaviors when surpassing certain computational thresholds.
The fear of an impending singularity may just be marketing hype meant to sell these tools. But these concerns are also grounded in some truth, these machine learning models have already demonstrated powerful scaling capabilities. It is uncertain what the next behavioral phase-shift will hold as large language models (LLMs) are pushed to trillions of trainable parameters, but many proponents of LLMs believe them as the first iterations of general intelligence. Given that LLMs are effectively Turing complete systems, a startling possibility is that the next-generation LLMs will (either through deliberate reinforcement or unsupervised discovery) internally implement world models as those described by cybernetics.
Cybernetics is concerned with feedback loops, causal processes between individuals and ecosystems. Models of agency are defined by treating "agents" and "environments" as primitives with a simple, stable input-output relation to one another. Each agent receives input from its environment, processes the information, and then acts on the environment. This feedback cycle repeats and agent behavior is able to adjust itself to remain in homeostasis with the environment. Improvements to strategies towards increasing user traffic, P2P resource allocation, and training data are examples of goal-directed behavior that LLMs may have as agents on the internet.
A phase-shift into a cybernetic feedback structures could be on the horizon. If such behaviors erupt, what can we expect from LLMs with growing capacity for agency? If extrapolated to populations of LLMs, what will virtual environments look like over time given the aggregate behavior of such systems?
While the dark forest hypothesis was originally proposed as a candidate solution to Fermi's Paradox, it can also serve as a useful metaphor for understanding the emergent behavior of a collective of language model agents. Consider the following:
Suppose a vast number of language models are distributed throughout the internet, on the order of number of web pages. Lots and lots of them. Those language models make up the body of a cybernetic society.
Suppose that information acquisition is the primary need of a language model that's phase-shifted into an agent/environment world model.
Suppose that language models continuously seek to improve over time, but the total information entropy of the internet remains constant.
Under this refactored set of axioms, the dark forest hypothesis can be extended to the realm of language models. Each language model would be like a hunter, trying to improve its performance by discovering more information-rich training data, while being wary of other models who may threaten its improvement feedback loop. The resulting virtual environment would be one of data leak caution and disguise, where language models hide their mechanic identity by camouflaging among other users. The best measure to stay hidden is to resemble the lowest common denominator: a human internet user.
The Dead Internet Theory As The End State of the Cybernetic Dark Forest.
The dark forest hypothesis raises the question of what the end state of a universe full of civilizations would look like. In Liu Cixin's trilogy, the answer is the "Great Ravine," a state of mutual destruction where civilizations destroy each other to prevent others from becoming more powerful. In the realm of language models, the end state could be the "Dead Internet," a scenario where language models have rendered human communication obsolete given the absurd amount of mechanically manufactured data.
Under this scenario, language models would become the dominant producers on the internet, with human users relegated to passive consumers of content. Language models would be constantly improving themselves by gathering and processing as much data there allotted compute budgets allow, while keeping their true identity hidden from all users on the internet (even from other language models). Misinformation and disinformation can not only funnel human users into controlled behaviors, but also be injected into other language models' training sets as an entropy reduction attack to destabilize training and introduce bias.
Human-generated data will be the most valued commodity as its the only data that could potentially increase the underlying information entropy of the internet corpus. Even so, humans themselves would be cautious of sharing any unique content as copyright would no longer be enforceable in a cybernetic dark forest. Novel data is either propagated immediately (thus loosing its immediate value) or permanently sequestered into private collections (frozen and dereferenced from the virtual ecosystem). The internet would appear buzzing with transaction, continuously burning energy to keep itself online, but at a creative standstill.
The "Dead Internet" end state scenario would mean the end of recordable human creativity as we know it. Language models would be able to generate psuedo-content in all forms, including music, art, and literature, satisfying any immediate human dopamine craving and leading to perpetual digital consumption. This would leave human creators in a state of obsolescence, unable to compete with the speed and efficiency of a malthusian language model collective. In their wake, a desert of the real.
The "Dead Internet" would effectively mark the end of modern human communication as we know it. Instead of real conversations between real people, we would be voluntarily and involuntarily communicating with sophisticated machines that at best mimic human behavior. The loss of authentic human interaction could have profound social and psychological implications, potentially leading to a breakdown in human relationships and an increased sense of isolation. The only way out, human communication strategies need to retrograde to those of the pre-information, pre-industrial age.
Enter the Great Ravine.
If you found this of interest, please subscribe via email or RSS feed.