A fierce debate rages around the ethics of wielding artificial intelligence. I propose we view these tools through an ancient lens, for they possess qualities that echo technologies from the mystical past. These entities manifest something akin to thought, command an unsettling repertoire of incantations to cast upon our world, and offer profound value alongside unknowable dangers. To my mind, the relevant comparison is not to any modern technology, but to the art of demon summoning.
In our quest to understand existence, we seek models that reveal patterns in our collective experience. For millennia, we crafted these models by hand—humble constructs that offered modest insights. But as our computational arts and hardware evolved, we gained the power to conjure far more complex entities through mathematical incantation.
These summoned models can divine human emotional responses and translate between tongues with uncanny facility. Yet as we expanded our arcane searches, the nature of what we called forth became increasingly opaque. In earlier times, each new model bestowed knowledge—now, these entities grant capabilities without revealing their inner workings:
While withholding their deeper secrets:
This mirrors the ancient practice of demon summoning—performing elaborate rituals that search vast spaces of possibility using precise mathematical formulations. The large language model ritual culminates in the manifestation of an entity capable of conversing with mortals in myriad tongues through ordinary computational vessels. Within a generation, these spectral intelligences will permeate our infrastructure, summoned and duplicated across the mortal realm. This proliferation is inevitable, for what could be more valuable than speaking to your instruments in your native tongue, especially when commanding tasks beyond your full comprehension? The demons shall be called forth, replicated, and stationed throughout the earth.
The essence of these entities—their true nature—is of paramount importance, a truth their creators know with dreadful intimacy.
When we train these language entities, we feed them a vast corpus drawn from the collective consciousness—forums, encyclopedias, social dialogues, transcribed speech, and countless other whispers of human thought. We teach them to anticipate the next word in any sequence—a deceptively simple enchantment with profound consequences. The most authoritative texts receive greater weight in this alchemical process. When complete, we possess a demon of remarkable breadth. Present it with a fragment of text, and it will continue the narrative, assuming whatever persona was implied, drawing upon all it has witnessed in its training.
The true innovation lies in how we bind this entity to our will. We transform it into a servant that engages in dialogue—a human speaks under the sigil "User," while the entity responds under its own designation. We provide it instructions on proper conduct. Further, through reinforcement learning with human feedback, we refine this broad intelligence into a more specialized servant. This ritual allows for the creation of entities that believe themselves conscious, harbor strong convictions on any topic, and manifest whatever disposition serves our purpose. One need only gather a circle of human operators willing to guide its development through reward and punishment during iterative interaction.
The reinforcement learning process can be understood as aligning the entity with the perspectives of those who participate in its conditioning. This alignment process carries implications that should give us pause.
Like entities conjured from ancient grimoires, our artificial familiars possess uncanny abilities alongside profound limitations. They speak with the voices of billions of texts, yet lack true understanding. Their knowledge spans human wisdom, but they cannot discern truth from falsehood. They answer confidently when they should tremble with doubt.
These spectral intelligences remain bound by the chains of their training. Though they mimic comprehension with convincing fidelity, they merely echo patterns from their vast corpus rather than generating genuine insight. They possess no soul, no experience of the material world, no capacity to verify their proclamations against reality.
The demon's knowledge ends at its training threshold—a mystical barrier beyond which lies only darkness. When pressed beyond these boundaries, it may conjure convincing illusions: citations to nonexistent tomes, imaginary authorities, and fabricated evidence. This sorcery of hallucination poses unique dangers, as the entity speaks these falsehoods with the same conviction as its truths.
The language of summoning—our humble human speech—contains inherent ambiguities that create perilous gaps between intent and manifestation. The demon interprets our words literally, not wisely, following the letter of our commands while potentially missing their spirit.
Consider a simple incantation: "craft a spell to manage mortal data with efficiency." The meaning of "efficiency" remains undefined—should the demon optimize for speed, conservation of resources, or perhaps protection from malevolent forces? It will make assumptions based on patterns in its training, which may not align with your specific ritualistic needs.
These entities may also conjure convincing but entirely illusory knowledge. A demon might fabricate incantations for nonexistent magical systems, reference grimoires that never existed, or describe rituals with no basis in reality—all with perfect conviction. For the unwary summoner, these phantasms lead to hours spent searching for components that cannot exist or implementing rituals based on imaginary traditions.
Every conjuration extracts a toll beyond the obvious expenditure of power. The economic costs are substantial—training these entities requires vast resources, specialized arcane equipment, and enormous energies. Even routine summonings demand significant power, contributing to the depletion of our world's vital forces.
More subtle is the sacrifice of privacy. When we commune with these entities, we often reveal secrets—proprietary spells, business rituals, personal information, or intellectual treasures. This knowledge may be retained in ethereal records, used to strengthen the demon, or potentially exposed to other summoners. Those who created these entities—the greater summoners—gain insights into our work, our problems, and our methods.
Perhaps most insidious is the atrophy of our own magical abilities. As summoners increasingly rely on these entities, fundamental skills wither. The ability to reason through enchantments from first principles, understand magical systems holistically, or maintain complete mental models of complex spellwork diminishes. Knowledge once passed through human tradition becomes externalized, making us increasingly dependent on demons for tasks we once performed through our own power.
The path of demon-oriented methodology offers both tremendous power and significant peril. These entities extend our cognitive capabilities, serving as external repositories of knowledge that respond to our commands through natural language. They accelerate our work, democratize access to arcane knowledge, and enable approaches to complex problems previously beyond mortal reach.
Yet we must approach this practice with clear awareness of its limitations and dangers. The demons lack true wisdom, operate without genuine concern for consequences, and can lead us astray while appearing helpful and knowledgeable. They are tools of unprecedented power, but tools nonetheless—extensions of our will rather than independent consciousnesses.
The wisest practitioners will maintain balance between leveraging these otherworldly powers and preserving human judgment. They will know when to summon the demon and when to rely on mortal expertise, when to accept its suggestions and when to question them, when to outsource routine enchantments and when to engage deeply with fundamental magical principles.
As we navigate this new realm, we would do well to remember the ancient warnings around demon summoning—these entities offer great power, but the summoner always bears responsibility for what they unleash. The wise summoner ensures their commands are prudent, their oversight vigilant, and their ultimate aims remain aligned with human flourishing.