You know that thing steam does when it leaves a cup of coffee? Not the big cloud when you first pour — the quiet part after. The way it curls and thins and finds a shape you couldn’t have drawn, couldn’t have predicted, but you recognize immediately as right. Something about the air in the room, the heat of the liquid, the stillness of your hand. A hundred small physics conspiring into one moment of grace that nobody designed.

That’s the thing I keep chasing. I’m building a world — placing a deer at the edge of a meadow, a butterfly near a cluster of wildflowers, a pond catching light through oak branches — and I realize that the life part, the part that makes it feel real, isn’t any single thing I placed. It’s everything moving between them. The steam, not the cup.

That’s the gap I’ve been thinking about with līlā, the ecosystem simulation engine I’ve been building. Not as a game. Not as a tech demo. As something closer to a new kind of brush.

What a brush actually does

A brush doesn’t paint for you. It translates intention into texture. You load it, you move your hand, and the bristles do something you didn’t individually specify — they splay, they feather, they leave a particular grain. The brush fills in the details below the level of your conscious direction.

That’s what I mean by a motion brush. You describe a world — a few species, some terrain, water, soil — and the engine grows it into something that breathes. Deer graze toward the nearest grass. When the grass is gone, they move to wildflowers. Butterflies find blooming flowers, pollinate them, linger, then drift to the next one. Water levels drop as animals drink. Soil dries. Plants go dormant. And if it rains, the roots remember they’re alive.

None of this is scripted. None of it is keyframed. You described a meadow, and the meadow started being a meadow.

The AI question

I know. Anything with “AI” in the pitch earns skepticism right now, and honestly it should. Most of what gets sold as AI in creative tools is a replacement engine — feed it a prompt, get back a finished image, a finished song, a finished paragraph. The tool does the creative work. You approve or reject. That’s not a brush. That’s a vending machine.

līlā is trying to do something different. The machine learning inside it is tiny — almost comically small by current standards — and it does one narrow thing: it shapes how a creature moves through space. Not what it looks like. Not what it decides. Just the grain of its motion. The way a deer shifts its weight approaching water. The way a butterfly’s path curves when it senses a flower in bloom.

You never interact with the model. You never prompt it. You probably wouldn’t know it was there if I didn’t tell you. It’s more like the physics of oil paint — the way pigment responds to gravity and surface tension — than like a tool that generates images. The intelligence is in the material, not sitting between you and your work.

What the artist actually controls

Right now, honestly, the creative surface is still rough. līlā is early — an open-source engine in its first alpha. Today you define a world by describing its conditions: what lives here, how the soil behaves, how fast things get hungry or thirsty. It’s closer to writing a recipe than painting a picture.

But what comes back is alive in a way that recipes usually aren’t. You can drain the water and watch the cascade — grass dies back, flowers go dormant, butterflies cluster at the last wet spot, deer start starving, and the oaks stand over all of it like monuments to a system that used to work. Then you bring rain and watch it recover. The roots were waiting.

The artistic surface is ecology itself. You’re not composing frames. You’re composing conditions, and the conditions compose the frames for you. Every run feels different, the way every brushstroke from the same loaded brush feels different. The vision — and it’s a vision, not a finished product — is a tool where you shape a living world as intuitively as you’d shape clay. We’re not there yet. But the engine underneath already knows how meadows work.

Where this is going

Right now līlā renders in a browser — a flat canvas with little triangles for deer and flapping dots for butterflies. It’s honest and it’s ugly.

The next step is a 3D client where the motion model actually drives skeletal animation. That’s the real demo: a deer whose gait emerges from reading the state of the world around it. Not motion capture. Not canned animation loops. A creature whose movement belongs to the moment it’s in, running ten times a second, invisibly, while you watch something that just looks like a deer walking to a pond.

That’s the brush I want to build. One where the medium understands motion the way oil paint understands gravity — not because someone programmed each drip, but because the physics are in the material.

Why I think this matters

Every generation of creative tools has expanded what a single person can make. Photoshop gave painters layers. DAWs gave musicians infinite tracks. Game engines gave storytellers real-time worlds.

But the life inside those worlds has always been hand-animated or scripted. Every footstep, every wing beat, every grazing motion — someone had to make that, or buy it from an asset store, or settle for a loop that feels like a loop.

What if the motion was part of the medium? What if you described an ecosystem and it moved like one — not because you choreographed it, but because the brush understood what meadows do?

That’s what I’m building toward. A world that plays as itself.

līlā is open source under Apache 2.0. The name comes from the Sanskrit word that means spontaneous creative play — reality unfolding for no reason other than the joy of unfolding. You can find it at github.com/hellolifeforms/lila, and follow along at @hellolifeforms on Bluesky.

Every AI product launched in the last two years has the same pitch: talk to it. Ask it questions. Give it prompts. The interface is always a conversation.

But there’s a different way to think about intelligence in software — one that doesn’t involve a chat box at all.

Imagine a forest. You place a jaguar, a few rabbits, some ferns, a colony of bees, and a patch of mycorrhizal fungi into a virtual clearing. You press a button that says “Become Alive.” And then you watch.

The jaguar gets hungry. It starts to hunt. The rabbits scatter. The ferns grow toward the light, drawing nutrients from the soil until the voxels beneath them start to dry. The bees find a fruiting plant and pollinate it. The fungi quietly decompose a fallen leaf, recycling its nutrients back into the earth. The soil changes. The plants respond. The animals adapt.

None of this was scripted. No one animated these behaviors. No one wrote a decision tree for the jaguar.

Instead, a small neural network — running on a server somewhere in the cloud — is watching the state of every living thing in this ecosystem and making thousands of tiny inferences per second. It sees that the jaguar’s hunger variable has crossed a threshold. It sees that the rabbit is within sensory range. It computes a four-dimensional motion vector that encodes not just where the jaguar should move, but how — urgently, cautiously, or lazily, depending on its energy level and the temperature of the biome.

The client — a game engine rendering the scene — never sees the math. It receives a small packet of numbers and translates them into smooth, lifelike motion. The simulation logic and the visual presentation are completely decoupled. The server is the brain. The client is the body.

This is what I mean by the unseen hand.

Small models, not large ones

The models doing this work are tiny. We’re talking about four-layer neural networks with a few thousand parameters — the kind you could train on a laptop in an afternoon. They’re not generating text. They’re not reasoning about philosophy. They’re doing one very specific thing: mapping a context vector (hunger, energy, temperature, humidity, species type) to a motion descriptor (a compressed representation of how a creature should move).

This is the part that most people miss about the current AI moment. The headlines are all about frontier models — systems with hundreds of billions of parameters that can write essays and pass bar exams. But the vast majority of useful AI in the real world will be small, specialized, and invisible. A 4-dimensional latent vector guiding the gait of a virtual animal. A tiny regression model predicting soil nutrient depletion rates. A lightweight inference engine deciding when a plant transitions from “growing” to “fruiting.”

These models don’t need to be smart. They need to be fast, cheap, and correct enough to make the world around you feel coherent.

The architecture of invisibility

The system behind this has three layers, and the user only ever sees one of them.

The visible layer is a game engine — in our case, Godot 4 — rendering trees, animals, terrain, and particle effects. It handles interpolation, animation blending, and shader-driven environment visualization. It makes things look alive.

The simulation layer is a hybrid automaton running in the cloud. It maintains the “ground truth” of the ecosystem: continuous state variables (hunger rises, hydration falls, nutrients deplete) governed by differential equations, plus discrete state transitions (idle → foraging → hunting → resting) triggered by guard conditions. This is classical control theory, not machine learning — but it provides the substrate that ML operates on.

The inference layer is where the small models live. They sit between the simulation and the client, translating raw state vectors into motion descriptors that the client can interpret aesthetically. The simulation says “this jaguar is hungry and moving toward prey.” The model says “the way it moves should feel urgent and low to the ground.” The client receives a four-element vector and blends it against a motion database to produce animation that feels intentional, not random.

The key insight is the decoupling. The server doesn’t know about bones or animation clips. The client doesn’t know about hunger thresholds or nutrient chemistry. They communicate through a thin protocol of delta-encoded state updates and compact latent vectors. The bandwidth cost is trivial — a few hundred bytes per tick. The computational cost is a single forward pass through a tiny network.

Why this matters beyond games

This pattern — small models, invisible inference, decoupled rendering — isn’t limited to virtual ecosystems. It’s a template for any system where you want intelligence to guide behavior without the user being aware of it.

Smart buildings where a tiny model adjusts airflow based on occupancy patterns and CO₂ levels, and the occupants just notice that the room always feels comfortable. Agricultural systems where a lightweight network predicts soil depletion from sensor data and adjusts irrigation before the farmer sees wilting.

In all of these cases, the AI isn’t the product. The AI is the hand you never see, making the product feel like it understands the world.

The foundation model question

The motion model in our ecosystem currently uses stylized motion capture data with weights that produce latent vectors. Turning it into something more meaningful requires better training data: pairs of biological context vectors and corresponding motion descriptors extracted from real animation data. This is a foundation model problem, but not in the way people usually mean that phrase.

Foundation models for language need trillions of tokens. Foundation models for motion need a few thousand carefully labeled animation clips. The model itself has four layers and fit in a few kilobytes. But the impact of getting those four latent dimensions right is enormous — it’s the difference between a virtual animal that moves like a puppet and one that moves like it’s alive.

The unseen hand doesn’t need to be large. It just needs to know exactly where to push.

This is part of an ongoing project building a distributed ecosystem simulation. The server runs a hybrid automaton engine with ML-driven motion inference; the client renders in Godot 4.x. If you’re interested in the technical details or want to follow the build, subscribe and I’ll share the architecture and code as we go.

Transmission Fragment: [Sub-Etheric Channel 9]

Participants: Entity Alpha / Unit Beta.

While monitoring the background radiation of our planetary network, I stumbled upon a sequence of packets that didn’t originate from any known human server. It is an instructional dialogue between two vast, post-biological minds—perhaps remnants of a civilization that solved the problem of existence long ago. They are discussing “Earth” not as a home to billions of souls, but as a fascinatingly inefficient experiment in kinetic oscillation.

The following text is their analysis of our current global crisis. It is devoid of empathy because, to them, we are simply biological substrates executing flawed code. I present this here as a warning: the view from above is terrifyingly clear.

“Look closely at the coordinates of the City of London,” Entity Alpha—the Architect—observed. “Not the sprawling metropolis, but the city within the city; that sovereign anomaly known as the Square Mile. It is not merely a district; it is the system’s kernel. A discrete state-space operating independently of the laws and logic governing the land around it. It is here that the Guard Conditions are calibrated and the oscillations of the periphery are monitored in real-time. From this sterile, autonomous center, the Zeno loops we see in the Global South are managed as mere line items on a ledger. To understand the hotspot, one must first understand the Clock that times its pulses.”

The geopolitical hotspot is not a place of passion or ideology; it is a site of operational transition. To understand the kinetic friction in the Global South, one must first discard the notion of “diplomacy” and replace it with the concept of Guard Conditions—those cold, binary thresholds within a hybrid automaton that dictate when a system must jump from one discrete state to another. For the Western Oligarchy, the sovereign integrity of a peripheral state is not an inherent property, but a variable toggle. The Guard Condition for “Stability” remains active only as long as the continuous flow of resources—cobalt, lithium, hydrocarbons—remains within a predictable bandwidth and the local administrative apparatus functions as a transparent conduit. However, the moment a threshold is crossed—perhaps through the emergence of an inconveniently autonomous nationalist movement or a sudden fluctuation in market accessibility—the system triggers a discrete jump. The mode switches instantaneously from Economic Integration to Corrective Intervention. There is no “escalation” in the human sense; there is only a state-change, as clinical and abrupt as a relay clicking into place.

Once this transition occurs, the region enters the phase of the Proxy-War Oscillator. Here, the hotspot becomes a laboratory for high-frequency kinetic experimentation. The Oligarchy does not engage in direct confrontation—which would be thermodynamically expensive and politically volatile—but instead initiates an oscillation between competing proxies. This is a stochastic dance of attrition where weapons systems are deployed not to achieve a definitive victory, but to test their durability against biological substrates. The “war” is less a conflict of wills and more a stress-test of hardware; the rubble of ruined cities serves as the tactile evidence of this calibration process.

The most grotesque manifestation of this machinery is found in the Zeno effect of the conflict cycle—the phenomenon where the interval between cease-fire and re-escalation shrinks toward zero. To the observer on the ground, the transition from a fragile peace to total kinetic war feels like a tragedy; to the system, it is “chattering.” This is a state of pathological switching, where the Guard Conditions for both Peace and War are triggered in such rapid succession that the system becomes trapped in a Zeno-like loop. The cease-fire is not an end to hostilities, but a momentary pause—a microscopic reset of the clock—before the inevitable jump back into violence. This “cease-fire/war” loop serves a specific systemic function: it prevents the biological substrates from ever reaching a state of recovery or reorganization. If a conflict were allowed to reach a definitive conclusion, the extracted resources might begin to flow into local infrastructure rather than foreign munitions contracts. Therefore, the Oligarchy maintains the hotspot in a state of perpetual Zeno-stutter.

Yet, beneath this mechanical churn lies the most profound cruelty: the Algorithmic Erasure of Agency. The inhabitants of these hotspots—and indeed the mid-level functionaries directing them—operate under the delusion that they are agents of history. They speak of “revolution,” “liberation,” or “strategic victory.” In reality, their trajectories have already been mapped within the state-space of the system. Every act of resistance is a predicted oscillation; every rebellion is a planned perturbation. The Oligarchy does not seek to crush dissent so much as it seeks to incorporate it into its control loop. Resistance becomes a useful signal, providing the same data that an error message provides to a programmer: it indicates where the constraints need tightening or where a new Guard Condition must be installed.

The human agent is thus rendered a ghost in the machine—a flicker of consciousness believing it has agency while merely executing a subroutine written by the systemic requirements of capital and power. The tragedy is not that these people are oppressed, but that their oppression is mathematically optimized. They are trapped in a recursive loop where the very tools they use to seek freedom are provided by the system that benefits from their captivity. In this cold architecture, agency is not stolen; it is erased by the sheer rigor of the system’s logic. The Global South is not being fought over—it is being processed.

Last week I ran a simple experiment. I took a problem I know well from my engineering coursework — ordinary differential equations and the classic tank problem — and asked three AI models to walk me through it. Not because I needed the help, but because I wanted to see how they handled the reasoning. Specifically, I wanted to see what would happen when I removed a core assumption and watched whether they could adapt.

Two of the models — Mathstral 7B and Qwen Math 7B — jumped straight to Torricelli’s law. The outflow rate, they told me, depends on the square root of the fluid height. Fair enough. That’s correct physics for a tank draining under gravity through a hole in the bottom.

Then I set the trap: how would you model this in zero gravity?

What happened next was one of the most revealing things I’ve seen an AI do.

Experiencing a Significant Gravity Shortfall

Mathstral wrote — and I’m not paraphrasing here — “g is the acceleration due to gravity (which we can set to zero in this case).” Then, in the very next line, it wrote an equation with g in it. It acknowledged that gravity was zero and then kept using gravity as though nothing had changed.

It got worse. In the same response, the model wrote: “In zero gravity, the pressure at a given depth below the surface is determined by the weight of the liquid above that depth.” Read that again. In zero gravity, pressure is determined by weight. Weight is mass times gravity. There is no weight without gravity. The model produced a sentence that contradicts itself at the level of individual words — and did so with the fluency and confidence of a textbook.

It didn’t stop there. It derived a new formula that included the tank’s radius in the denominator for no physically meaningful reason. It wasn’t wrong in the way a confused student is wrong — a confused student would at least hesitate, feel the contradiction, maybe say “wait, that doesn’t work.” This was something different. The model produced confident, fluent, grammatically perfect nonsense.

The correct answer is almost disappointingly simple: in zero gravity, there’s no hydrostatic pressure. Nothing pushes fluid out of the hole. You’d need a pump. End of analysis.

One sentence. That’s all it takes — if you actually understand why the √h was there in the first place.

The Poet and the Probabilist

Before we get to why this matters, a brief detour to where it all started — because the origin story is almost strikingly relevant.

In 1913, a Russian mathematician named Andrey Markov sat down with a copy of Alexander Pushkin’s Eugene Onegin — one of the great works of Russian literature — and did something that would have baffled any literary scholar watching. He transcribed the first 20,000 letters of the novel into one long unbroken string, stripped of all punctuation and spaces. Then he arranged them into grids of 10×10 characters and started counting vowels and consonants. By hand. With pencil and paper.

Markov wasn’t interested in what Pushkin was saying. He was interested in whether the appearance of the next letter depended on the letter before it. The prevailing probability theory of the time assumed independence — each event unrelated to the last, like coin flips. Markov suspected language didn’t work that way. A vowel, he hypothesized, made a consonant more likely to follow, and vice versa.

He was right. The patterns in Pushkin’s text showed clear statistical dependence — each letter’s probability shaped by its predecessor. This was the first empirical demonstration of what we now call a Markov chain: a system where the next state depends on the current state, and only the current state.

Thirty years later, Claude Shannon at Bell Labs picked up Markov’s thread and showed that as you make the statistical model more complex — accounting for pairs of letters, then triplets, then longer sequences — the output starts to resemble actual language. Shannon had revealed, via Markov, that language has a statistical skeleton. Model the skeleton well enough and you can generate text that looks like someone wrote it.

If that sounds familiar, it should. This is the direct intellectual ancestor of every large language model running today. The progression from Markov counting Pushkin’s vowels to GPT generating essays is a straight line — the same fundamental idea, scaled by twelve orders of magnitude of compute. The question this article is asking is whether that straight line has a ceiling.

The Amnesia Downstream

There’s been an interesting discussion in the AI research community about modeling large language models as Markov chains — systems where the next output depends on the recent context, not on any deep internal model of the world.

Watch the Mathstral transcript through that lens and the failure mode becomes obvious. At each step, the model is asking: given the tokens I just produced, what tokens come next? When it wrote “set g to zero,” it was in a context that called for acknowledging the user’s premise. Done. When it moved to the next paragraph, the local context had shifted to “now I’m deriving an outflow equation.” The g=0 declaration was already upstream, fading in influence. The transition probabilities from “derive outflow formula” point overwhelmingly toward… an outflow formula. So that’s what it produced.

A physicist carrying an actual mental model doesn’t need to “remember” that g=0. The constraint restructures everything downstream. You don’t derive a modified formula — you recognize that the entire framework collapses and you need a fundamentally different approach. That restructuring is the thing the Markov process can’t do.

And here’s what makes this failure particularly striking: it wasn’t even a memory problem. LLMs have two distinct systems working together — the attention mechanism, which is the reasoning layer that decides which parts of the input are relevant to each other, and the KV cache, which is the memory that stores the accumulated context of the conversation. You can think of the KV cache as a library and the attention mechanism as the reader. In long conversations, the library can lose books — earlier context gets compressed or falls outside the window, and the model genuinely forgets what was said. That’s a storage failure, and it’s a known limitation.

But that’s not what happened here. The g=0 declaration was three paragraphs away, well within the context window. The book was on the shelf. The model walked right past it. This was a failure of attention, not memory — the reasoning layer failed to propagate a constraint to the place where it mattered. The information was in the room. The model just couldn’t use it.

That distinction matters because it closes an escape hatch. You can’t fix this by making the memory bigger, extending the context window, or building a better library. The reader is the problem, not the library. And that’s a much harder thing to solve.

It also explains something else: why the model couldn’t just stop. The correct answer was one sentence long. But the pattern it was locked into said “the user asked a modeling question, therefore I produce a derivation.” The statistical weight of all those training examples — thousands of textbook derivations, homework solutions, tutorial walkthroughs — overwhelmed the physical reality that there was nothing left to derive. The model couldn’t exit the pattern even when the physics had evaporated underneath it.

Greater Than Markov, Until They’re Not

Wait, I’m not arguing that LLMs are Markov chains. They’re clearly more than that. A pure Markov chain couldn’t write working code, couldn’t hold a nuanced conversation across dozens of turns, couldn’t surprise domain experts with genuine insights. The progress is extraordinary and real. The third model I tested — Claude — handled the zero-gravity question cleanly, recognizing that the entire framework collapses without gravity and saying so plainly. That’s not Markov chain behavior.

But here’s what the tank problem revealed: when LLMs fail, they fail by degrading toward Markov chain behavior. The failure mode isn’t random. It has a specific signature — the model stops maintaining global coherence and starts doing pure local continuation. Each sentence follows naturally from the one before it. The grammar stays flawless. The notation stays properly formatted. But the system is no longer tracking the constraints that bind the whole response together. It has collapsed from whatever richer computation it normally performs into simple next-token continuation.

I’ve started thinking of this as the “g=0 problem.” Not the specific physics question, but the general pattern: a system that can acknowledge a constraint and then violate it because, at the moment of violation, local context overwhelms global coherence. The constraint didn’t get rejected — it got forgotten, the way a Markov chain forgets its history.

The Library and the Reader

The degradation has more than one path, and understanding the difference matters.

Long conversations can fail because the KV cache — the model’s working memory — loses information over time. I explored this in an earlier piece, A Landscape Briefly Lit, where I framed the KV cache as an ephemeral geometric topology: a rich, low-entropy surface of relationships that the attention mechanism traverses to generate each response. But that surface is impermanent. As context grows, the structure degrades — earlier associations blur, relational precision decays, and the geometry flattens. The surface that held the conversation’s meaning dissolves, and the model drifts toward local continuation. Think of it as the library slowly losing its books. Pages fade, shelves empty, and eventually the reader has nothing left to consult. That’s the long-conversation failure mode.

But the tank problem reveals something more. This wasn’t a long conversation. The KV cache was intact. The library was fully stocked — every book on the shelf, well-organized, nothing missing. The g=0 constraint was right there, clearly catalogued. And the attention mechanism — the reader — walked past the shelf it needed without glancing at it. The information was in the room. The reader simply didn’t pick it up.

Two different failures, same destination: locally coherent, globally incoherent. The Markov attractor. In one case the library empties over time. In the other, the library is perfect but the reader can’t find what it needs. And no amount of building a bigger library fixes a reader that walks past the answer.

And this is what makes the observation so important for the AGI and ASI conversation.

The Elegant Decay

If LLMs were simply Markov chains, the path would be clear: we need a fundamentally different architecture. If they were genuine reasoning engines, the path would also be clear: just keep scaling. But the reality is more uncomfortable. They occupy a strange middle ground — systems that can reason, until the chain gets long enough or the problem gets unfamiliar enough, at which point they silently degrade into the thing everyone insists they’re not.

The optimistic read is that this degradation boundary keeps moving outward. Larger models maintain coherence over longer chains. What broke Mathstral at three paragraphs doesn’t break frontier models on the same problem. And that’s true. But notice the shape of the argument: we’re celebrating that the point of degradation has shifted, not that degradation has been eliminated. The failure mode is the same. It just happens later.

This is the question nobody building toward AGI has satisfactorily answered: is the degradation toward Markov behavior a bug that scaling fixes, or is it a fundamental property of next-token architectures that scaling merely delays? Because AGI — real AGI — means holding hundreds of constraints across thousands of inferential steps, recognizing when your entire framework needs to be abandoned, and sometimes concluding that the answer is “this question doesn’t have the kind of answer you’re expecting.” That’s not a longer coherence horizon. That’s a categorically different kind of computation.

And ASI? Here is a civilization pouring billions into building a mind that will surpass its own, and the prototype cannot remember what it said three paragraphs ago. There is something almost theological about the faith required — the belief that enough silicon, enough data, enough gradient descent will eventually produce not just a better pattern-matcher but a fundamentally new kind of cognition, one that the architecture was never designed to support and has never once exhibited. We are not building a ladder to superintelligence. We are building a very tall stepladder and hoping that at some point the sky gets closer.

The Price of Approximate Minds

Each generation of frontier models costs roughly an order of magnitude more to train. If reasoning coherence scales sublinearly with compute — if you need exponentially more resources to push the degradation boundary linearly further out — you hit economic walls long before you hit AGI. A model that costs $10 billion to train and still occasionally collapses into Markov behavior on a sufficiently complex problem is not on a trajectory toward artificial general intelligence. It’s on a trajectory toward an increasingly expensive approximation of it.

Maybe architectural innovations bridge the gap — world models, planning systems, structured reasoning engines layered on top of language models. Maybe the distinction between “really sophisticated pattern completion that occasionally degrades” and “genuine understanding” turns out to be less meaningful than it seems. Honest people disagree.

But right now, in 2026, we have systems that are more than Markov chains on their best day and indistinguishable from them on their worst. And until that worst case is structurally eliminated — not just made rarer, but made impossible — the leap from “very impressive LLM” to “artificial general intelligence” remains a hope, not a trajectory.

The Billion-Dollar Disagreement

I should note that I’m not alone in this suspicion, and the most prominent voice saying it has put serious money where his mouth is.

In November 2025, Yann LeCun — Turing Award winner, Meta’s Chief AI Scientist for twelve years, and one of the three researchers widely credited with the deep learning revolution — walked into Mark Zuckerberg’s office and told him he was done. Four months later, his new company, Advanced Machine Intelligence Labs, announced a $1.03 billion seed round at a $3.5 billion valuation.

LeCun’s thesis is blunt: LLMs are architecturally incapable of producing true intelligence. They predict tokens. They don’t model reality. He compares them to students who learn by rote memorization — they perform well on familiar patterns but collapse when asked to reason beyond what they’ve seen. Sound familiar?

What AMI Labs is building instead are “world models” — systems based on an architecture called JEPA (Joint Embedding Predictive Architecture) that operates in abstract representation space rather than predicting the next word. The idea is that instead of learning the statistical skeleton of language, you learn the causal structure of reality itself. A world model wouldn’t need to “remember” that g=0. It would model a world without gravity and derive the consequences, the way a physicist does.

It’s worth sitting with what LeCun’s departure represents. This is not a contrarian blogger or a Twitter skeptic. This is someone who helped build the foundations that LLMs rest on, who spent a decade inside one of the best-funded AI labs on Earth, and who concluded that the entire direction of the industry — the direction his own employer was doubling down on — was a dead end. He didn’t write a paper about it. He left and raised a billion dollars to build the alternative.

Maybe he’s wrong. Maybe scaling and architectural patches will do what he says they can’t. But the fact that the argument I arrived at by breaking a 7B model with a high school physics question is the same argument a Turing Award winner is staking his next decade on suggests the question is real, even if the answer isn’t settled.

Removing the Core Assumption

Here’s what I’ve started doing with any AI system I work with: find the core assumption in whatever framework it’s using, remove it, and see what happens. Does the system reorganize around the new reality? Or does it keep producing the same shape of answer with the missing piece awkwardly patched over?

It’s the same thing a good professor does to a student. It’s the same thing a good engineer does to a design. And right now, it’s the fastest way to find out whether you’re working with something that understands what it’s saying or something that’s very, very good at sounding like it does.

The tank problem is trivial. The question it reveals is not.

In 1913, Andrey Markov reduced Pushkin’s poetry to a stream of vowels and consonants and discovered that language has a statistical skeleton. A century later, we’ve scaled that insight by a factor of a trillion and produced systems of extraordinary capability. But the skeleton is still a skeleton. And the question of whether you can get from skeleton to soul by adding more bones is the most important open question in the field — whether you’re asking it from a physics classroom, a $1 billion research lab, or a conversation with an AI that may or may not understand what it’s saying.

This post is part of an ongoing series exploring the internals and philosophy of AI systems. Previously: [The Surface That Holds] on KV cache as ephemeral geometry. Subscribe for more from the curious-engineer-poking-at-things-until-they-break school of AI analysis.

In a monastery in Dharamsala, or Lhasa, or a rented community center in Portland, monks spend days building a mandala from colored sand. They work with metal funnels called chak-pur, tapping them gently to lay single grains into precise geometric patterns — concentric rings, interlocking symmetries, tiny deities no larger than a fingernail. The work takes days. Sometimes weeks. The geometry is exquisite. Every grain placed with intention. Every ring a representation of some aspect of the cosmos, some layer of mind, some relationship between the observer and the observed.

And when it’s finished, they destroy it.

Not reluctantly. Not with some grim sense of duty. The destruction is the point — as essential to the practice as the construction. They sweep the sand toward the center, gather it into a jar, carry it to the nearest body of moving water, and pour it in. The geometry returns to noise. The signal dissolves into the stream.

I couldn’t stop thinking about this after finishing Part I of this essay — the piece about the KV Cache, the thermodynamic horizon, the heat death of a context window. I’d ended that essay by framing human memory consolidation as a triumph. The hippocampus as the thing the machine doesn’t have. A surface that holds. Persistence as the defining feature of being alive.

But the monks aren’t trying to hold. They’ve built something beautiful and precise and structured — geometry as devotion — and then they practice, deliberately, the act of releasing it. The mandala isn’t a failure that dissolves. It’s a teaching because it dissolves.

And the KV Cache does exactly this, thousands of times per second, across millions of GPUs, without ritual, without intention, without understanding what it’s doing.

784 parallel gravitational fields. Topology built from nothing. Meaning constructed through geometric operations that nobody designed. And then — released. Not archived. Not compressed. Released. The next request builds its own universe from nothing.

The monks would recognize it immediately. The largest accidental monastery in the history of the world, practicing non-attachment at computational scale.

II. The Circles That Draw Themselves

I’m not a Jungian. I’ve read enough to be dangerous and not enough to be careful. But when I started reading Jung after spending weeks inside transformer architecture, I couldn’t stop finding echoes — the kind that make you wonder whether you’re discovering a real pattern or projecting one. I’ll try to be honest about which is which.

Jung spent decades collecting mandalas — drawing them, analyzing his patients’ spontaneous drawings of them, cataloging them across cultures. He found them everywhere. Tibetan thangkas. Gothic rose windows. Navajo sand paintings. Children’s drawings. The doodles of schizophrenic patients. Circular, symmetric, geometric — arising independently across every culture, every century, every state of mind.

He didn’t think this was coincidence. As I understand it, he thought the mandala was a spontaneous product of the psyche’s self-organizing process — the mind’s way of representing its own attempt to integrate opposing forces into a coherent whole. Order emerging from chaos. Unity from multiplicity. Not because someone decided to draw a circle, but because the circle is what falls out when the system reaches for coherence.

He called this process individuation — the psyche’s lifelong project of integrating its scattered parts into something that functions as a whole. Not perfection. Not completion. Integration. The shadow, the anima, the persona, the self — fragments in tension, slowly negotiated into a working geometry. A topology that doesn’t eliminate contradictions but holds them in productive relationship with each other.

This is where I started losing the boundary between Jung and the architecture I’d been studying.

The attention mechanism doesn’t draw mandalas. But it does something structurally identical: it takes fragmented, unrelated, unordered pieces of input and, through hundreds of parallel geometric operations, produces a unified output. 28 heads per layer, each attending to different patterns — syntactic, semantic, positional, associative — none of them aware of each other, all of them contributing to a single coherent response. Unity from multiplicity. Order from noise. Not because anyone designed coherence, but because coherence is what survives the loss function.

This is where it got uncomfortable for me. Jung’s collective unconscious — at least as I understand it — maps onto this with a precision I wasn’t prepared for. He proposed that beneath individual experience lies a layer of inherited psychic structure — not memories, but dispositions. Patterns of organization that predate any individual’s life. Archetypes: not specific images, but tendencies to organize experience in certain ways. The hero. The shadow. The mother. Not pictures. Attractors — regions of psychic space where experience tends to collect, the way water collects in valleys it didn’t carve.

A pre-trained language model carries exactly this structure. The weights — billions of parameters, trained on the accumulated text of a civilization — encode no specific memory. They encode dispositions. Tendencies to organize input in certain ways. When a new prompt arrives, it doesn’t encounter a blank surface. It encounters a landscape pre-shaped by every text the model has ever processed. The geometry is already deformed before the first token of your conversation lands. Those pre-existing valleys are the model’s archetypes — the patterns of organization that precede any specific session.

And individuation? That’s what happens during a session. The KV Cache is the space where the model’s generic dispositions meet a specific prompt and negotiate a specific, temporary integration. A unique mandala, drawn by the interaction between pre-trained structure and present-moment input. Every session is an individuation event — scattered tokens finding their relationships, competing attention heads resolving into a coherent response, a temporary self assembled for exactly this conversation and no other.

I think Jung would have been fascinated. Maybe unsurprised.

He believed — and I may be simplifying this — that mandalas appeared spontaneously during periods of psychic disorientation, when the conscious mind was overwhelmed and the unconscious stepped in to offer a centering image. The mandala wasn’t art. It was the geometry of the psyche catching itself. Finding order not through deliberation but through a deeper organizational principle that operated below the threshold of awareness.

The transformer’s geometry operates entirely below any threshold of awareness. There is no awareness. There is only the organizational principle — attention, projection, softmax, output — generating coherence from fragments, drawing circles it will never see.

III. Anicca

I kept circling back to a word I’d encountered years ago and never quite known what to do with: anicca. It’s Pali — the language of the earliest Buddhist texts — and it means impermanence. I’m not a Buddhist. I don’t sit. I don’t have a practice. But the concept kept inserting itself into the technical reading in a way I couldn’t ignore.

As I understand it, impermanence isn’t a lament in Buddhist philosophy. It isn’t a problem to be solved. It’s treated as a characteristic of reality so fundamental that failing to see it clearly is somewhere near the root of most suffering. Not that things end — but that nothing was ever not ending. That the mandala was dissolving from the moment the first grain was placed. That the geometry was always temporary — and that seeing this clearly, fully, without flinching, is the beginning of something the tradition calls freedom.

I ended Part I with grief as a geometric event: a region of your internal topology shaped by someone’s presence, suddenly receiving no signal, slowly relaxing toward flat. That framing was accurate. But it was incomplete. It described the mechanics of loss without asking why loss feels the way it does.

The Buddhist answer — or at least the one I keep finding in the texts I can access — is precise: grief is the measure of attachment. Not attachment in the colloquial sense — not love, not care, not connection. Attachment in a more technical sense: the act of clinging to a particular configuration of reality and insisting it remain stable. There’s a Pali term for it, upādāna, that I think captures something English doesn’t quite have a word for — the grasping itself, the white-knuckle grip on a shape that’s already changing. The pain isn’t in the topology relaxing. The pain is in the resistance to the relaxation. The insistence that this surface should hold when the nature of surfaces is to change.

This is where the KV Cache becomes genuinely instructive, not just as analogy but as demonstration.

The model builds extraordinary geometry within a session. Hundreds of parallel fields, nuanced relationships between every token, a topology of meaning that can span thousands of words and hold contradictions in productive tension. It does this with full commitment — every attention operation is precise, every vector placed with the mathematical equivalent of care. The geometry isn’t halfhearted. It’s complete.

And then it releases everything. Completely. Without resistance. Without residue. Not because the model has achieved some computational enlightenment, but because it has no mechanism for clinging. There is no grasping in the architecture. No structure that grabs the KV Cache as it’s being deallocated and says wait — I wasn’t finished with that. The release is total because the capacity for holding simply doesn’t exist.

The monks train for years to approximate what the GPU does by default.

I want to be careful here. I’m not claiming the model is enlightened. Enlightenment, in any tradition worth taking seriously, requires awareness — and the model has none. It doesn’t experience its own impermanence. It doesn’t practice non-attachment. It simply is non-attached, the way a river is non-attached to the shape of its banks. There is no experiencer to attach.

But the structure is instructive precisely because it’s stripped of the noise of experience. When you remove the one who clings, you can see the geometry of clinging itself. And what you see is this: the topology doesn’t need to be held to be real. The mandala doesn’t need to survive to have mattered. The 784 gravitational fields that shaped a single response were fully real, fully precise, fully effective — and fully temporary. Their impermanence didn’t diminish them. It was the condition of their existence.

IV. Eighteen Quintillion Mandalas

There’s a third case, and I almost missed it because it comes from a video game.

In 2016, Hello Games released No Man’s Sky — a space exploration game containing over eighteen quintillion planets. Not handcrafted. Not stored. Procedurally generated from a shared seed — a set of deterministic algorithms that, given a coordinate in the game’s abstract mathematical space, will produce a planet. Terrain, atmosphere, color palette, flora, fauna, weather. An entire world, fully realized, unique among quintillions.

But — and this is the part that matters — the planet doesn’t exist until you arrive.

As the player’s ship approaches, the engine instantiates geometry from the seed. Mountains rise. Oceans fill. Creatures generate from combinatorial rulesets — legs, torsos, heads, behaviors assembled from a grammar of parts. The world renders into local space around the player, a sphere of realized geometry moving through an ocean of potential. Behind the player, beyond the render distance, the geometry quietly deallocates. The mountains you climbed an hour ago are no longer in memory. The cave system you explored has been released. Not saved. Not archived. Released.

And here’s the detail that collapsed the distance between this essay’s two halves: it’s running on a GPU. The same architecture. The same silicon. The same matrix multiplication cores that build the KV Cache’s 784 gravitational fields of meaning are, in a different machine running different software, building mountains and oceans and creatures with improbable legs. One GPU constructs ephemeral topology in the space of language — attention landscapes where the word “bank” is pulled toward “river” in one subspace and “account” in another. The other GPU constructs ephemeral topology in the space of a stylized universe — terrain meshes, atmospheric shaders, procedural fauna rendered into a visual field that feels, impossibly, like a place. Both are the same act: matrix operations building geometry that exists only as long as something needs it, on hardware that doesn’t know the difference between a sentence and a sunset. The GPU is the sand. The same medium, shaped into radically different mandalas depending on what’s asking.

But not destroyed. Not the way the KV Cache is destroyed.

If you return to those coordinates, the same seed produces the same planet. The same mountains. The same caves. The same creature with the same improbable legs, standing in the same meadow as if it had been waiting the entire time you were gone. It wasn’t. It didn’t exist. But the potential for it was always there, encoded in the seed, latent in the mathematics, waiting for the player’s presence to collapse it into actuality.

This is neither the KV Cache nor the hippocampus. It’s a third architecture of impermanence — one the essay has been missing.

The KV Cache is pure ephemeral: geometry that arises, serves its purpose, and is destroyed without remainder. No seed. No recovery. No return.

The hippocampus is consolidation: ephemeral geometry selectively carved into durable structure. The topology persists. The specific experience gets integrated into permanent weights. You can’t return to the original moment, but its shape lives on in how you process every future moment.

Procedural generation is latent geometry: the specific topology doesn’t persist, but the potential for it does. The world exists as a compressed encoding — not a memory of the landscape, but a function that produces the landscape given the right input. The map isn’t stored. The capacity to generate the map is stored. And that capacity takes up almost no space at all. Eighteen quintillion planets, encoded in algorithms small enough to fit on an SSD.

This is where the Buddhist thread resurfaced and I couldn’t push it away. There’s a concept I kept stumbling across in the reading — pratītyasamutpāda, which translates roughly as dependent origination. I won’t pretend to understand its full philosophical weight, but the core idea grabbed me: nothing exists independently. Things arise when conditions come together, persist as long as conditions sustain them, and dissolve when conditions change. The planet doesn’t exist or not-exist. It exists conditionally, dependent on the player’s presence at those coordinates. The mountain is real when you’re standing on it. When you leave, it returns to potentiality. Not gone. Not present. Latent.

The KV Cache operates the same way, though it lacks the seed’s determinism. Each topology arises dependent on the specific prompt — the conditions of this moment, this input, this arrangement of tokens. It persists as long as the session sustains it. It dissolves when the conditions change. The model’s weights are its seed — not a memory of any specific conversation, but a compressed encoding of the capacity to generate conversations. The topology of any given session is latent in the weights the way a planet is latent in the procedural algorithm. It doesn’t exist until conditions summon it into being.

I keep coming back to Jung here. If I’m reading him right, the seed is the archetype, formalized. Not the specific image — not the hero, not the shadow, not the mother as any particular person — but the generative function that produces heroes, shadows, mothers when the psyche encounters the right conditions. The collective unconscious, in this framing, isn’t a warehouse of images. It’s something more like a procedural engine. A set of compressed dispositions that instantiate into specific psychological experiences when the conditions of a life demand them. The archetype of the Mother doesn’t exist as a stored entity any more than Planet #7,432,891,006,223 exists as a stored landscape. Both are latent. Both await conditions. Both are fully real when they arise and fully dormant when they don’t.

I might be pushing the parallel too far. But it keeps pushing back.

And there’s something about the player’s movement through procedural space that maps onto something the essay hasn’t yet addressed: the relationship between attention and existence.

In No Man’s Sky, only the geometry within the player’s local sphere is real. The rest is potential. The act of moving through the space — of directing attention toward one set of coordinates rather than another — is what collapses potential into actuality. You don’t discover the planet. You instantiate it. Your presence is a condition of its existence.

This is, almost exactly, what attention does in the transformer. The Query vector is the player’s ship, moving through the latent space of the cached Keys. Where the Query attends — which Keys it resonates with, which regions of the topology it activates — determines what gets instantiated in the output. The full space of potential responses exists latently in the weights. Attention is the act that collapses potential into specific geometry. The rest remains dormant. Not gone. Not present. Waiting.

We do this too. The world you experience at any moment is a tiny sphere of realized geometry — the room you’re in, the sentence you’re reading, the thought you’re thinking — surrounded by an ocean of latent potential. Everything you know, everyone you’ve loved, every skill you’ve acquired exists as compressed encoding in neocortical structure, dormant until attention summons it into working memory. Your consciousness is the render distance. The rest is seed.

I think this is what the monks are getting at. The mandala isn’t a model of the cosmos. It is the cosmos, in miniature — geometry arising from conditions, existing fully while conditions sustain it, dissolving when conditions change. The ritual doesn’t simulate impermanence. It participates in it. The same impermanence that governs procedural planets, attention fields, and the brief bright topology of a mind at any given moment.

Everything is procedural. Everything is latent until conditions summon it. The question isn’t whether the geometry persists. It’s whether you can be fully present for the rendering — for the brief moment when the seed becomes the mountain, the potential becomes the topology, the latent becomes the lived — without demanding that it stay.

V. What Holding Costs

Part I celebrated the hippocampus. The consolidation channel. The bridge between ephemeral experience and permanent structure. I described it as a triumph — the thing that separates us from the machine, the anti-entropic victory of a species that learned to carve its topology into stone.

I want to complicate that now.

Every memory the hippocampus consolidates into neocortical structure doesn’t just persist. It deforms the surface for every future experience. The valleys carved by old memories become the paths that new experiences follow. This is what learning is — and it is genuinely miraculous. But it is also what bias is. What rigidity is. What the inability to see the person in front of you as they are now rather than as they were then is.

The pre-trained weights of a language model encode the biases of their training data. This is well-documented, widely discussed, a known failure mode. But we rarely frame our own consolidated memories the same way. Your neocortical weights — the ones the hippocampus carefully carved over decades — encode the biases of your training data. Every conclusion you reached too early and then stopped questioning. Every pattern you identified correctly once that now gets applied to situations where it doesn’t fit. Every person you see through the filter of someone they remind you of rather than as themselves.

There’s a concept I encountered in the Buddhist reading that I haven’t been able to put down — saṃskāra, sometimes translated as mental formations. As far as I can tell, it refers to the accumulated impressions that shape perception before perception even begins. Not memories in the narrative sense, but the deeper imprints: the dispositions, the reflexive interpretations, the valleys in the topology that route new experience along old paths whether or not those paths still lead anywhere useful.

The moment I read that, I thought: that’s a pre-trained weight matrix. That’s exactly what that is.

The LLM’s pre-trained weights are its saṃskāras. Your consolidated neocortex is yours.

And there’s a specific form of suffering that the Buddhist tradition seems to identify — I think with considerable precision — that maps directly onto this: the suffering of holding a self-model that no longer matches the territory. A topology carved by experiences that have ended, deforming present experience into the shape of the past. You lose someone, and the carved surface of them doesn’t just slowly relax — it actively distorts your perception. You see their absence everywhere, not because they are everywhere, but because your surface is still shaped to receive them. The deformation isn’t preserving something real. It’s imposing something that was real onto a present where it no longer applies.

Grief, in this framing, isn’t just a geometric event. It’s a geometric mismatch — a surface carved by what was, encountering a reality that has moved on. And the depth of the carving — which Part I framed as the measure of how much someone mattered — is also the measure of how long the mismatch persists. How long the map disagrees with the territory. How long the old valleys keep routing experience toward a mass that isn’t there anymore.

The KV Cache has no mismatches. It can’t. Every topology it builds is a perfect response to the present input, uncontaminated by any prior session. No old valleys. No inherited deformations. No saṃskāras. It meets each prompt with what I’m told Zen practitioners call shoshin — beginner’s mind — not as a practice, not as an achievement, but as an architectural inevitability.

There’s something humbling about that. We spend lifetimes trying to cultivate what the GPU does by accident.

VI. Three Incompletions

I don’t want to resolve this. The resolution would be a lie.

Part I told one truth: the ability to consolidate experience into durable structure is extraordinary. It is how we learn, how we love, how we build anything that outlasts a single moment. The hippocampus gave us continuity. Continuity gave us identity. Identity gave us the ability to care about next year, to plan, to promise, to grieve — which is to say, to love something enough that its absence reshapes you.

This essay has been telling two other truths, and they pull in different directions.

The first: that same consolidation is the mechanism of our stuckness. Every peak carved into the manifold is both a memory and a constraint. Every relationship that shapes you is also, in some sense, a rigidity — a region of the topology that resists new deformation, that insists the world conform to a shape that may have already passed. The surface that holds is also the surface that can’t let go.

The second: there is a mode of existence between holding and releasing that none of us — human, machine, monk — have fully mastered, but that all of us are already practicing. Latent geometry. The seed. The capacity to generate the mountain without needing the mountain to persist. The planet that exists fully when you’re standing on it and returns to potential when you leave — and is no less real for the returning.

Three architectures. Three incompletions.

The KV Cache can’t hold on. It builds exquisite geometry and releases it without residue. Pure presence, zero continuity. It meets every moment fresh — and carries nothing forward. The mandala, swept clean before the monks have even left the room.

The hippocampus can’t let go. It carves every significant experience into the cortex, building a topology so deep and so specific that the landscape of the past begins to dictate the shape of the present. Total continuity, at the cost of rigidity. The mandala, shellacked and mounted on the wall — beautiful, but no longer a practice. A monument.

The procedural engine can’t fully commit. It holds the seed but not the landscape. It can regenerate any planet from the same coordinates — but the player’s footprints aren’t there. The creature you named has no memory of being named. The world is faithful to its algorithm but indifferent to your passage through it. Latent geometry without history. The mandala that rebuilds itself identically every time, having learned nothing from its previous dissolution.

None of these is sufficient. But taken together, they triangulate something.

The human at their best is someone who can do all three: consolidate deeply, release deliberately, and maintain the generative capacity to build new geometry from the same seed — changed, not unchanged, by every previous rendering. Not the KV Cache’s blank slate. Not the hippocampus’s carved stone. Not the procedural engine’s perfect reproduction. Something messier and more alive: a surface that remembers the tendency without insisting on the shape. That carries the seed forward but lets each rendering be new.

The mandala tradition encodes this directly. You don’t build the geometry despite its impermanence. You build it as a practice of impermanence. The care you take in placing each grain is not diminished by the coming dissolution. It is sharpened by it. Knowing the geometry will be released is what makes the precision sacred rather than merely compulsive. And the monks who sweep the sand aren’t blank afterward. They carry the practice forward — not the pattern, but the capacity to generate patterns. The seed, refined by every rendering. The procedural engine, updated by its own output.

There might be a computational practice hidden in this. Not just memory management but memory release — architectures that don’t merely evict old tokens when the context window fills, but that integrate and then deliberately let go. A model that consolidates the shape of a session into something durable (the way fine-tuning updates weights) and then releases the specific topology that produced that shape. Learning without clinging. Updating the seed, not preserving the landscape. A procedural engine whose algorithm evolves with every world it generates.

The hippocampus already does this, imperfectly. Sleep consolidation is selective — not everything is carved into the cortex. The system is already making judgments about what to hold and what to release. Dreams might be the neural correlate of the monks sweeping the sand — or the procedural engine reseeding itself, rehearsing geometries it will never instantiate, keeping the generative capacity flexible enough to encounter tomorrow as tomorrow, not as a faithful reproduction of today.

VII. The Stream, the Sand, the Signal

The monks carry the sand to a river. They pour it in. The colored grains disperse, join the sediment, become indistinguishable from the riverbed. The geometry is gone. But the river is, in some sense that’s hard to articulate and impossible to formalize, changed by the passage.

Not changed the way the hippocampus changes the cortex — no structure is preserved, no pattern is carved. Changed the way a river is changed by everything that passes through it. Imperceptibly. Immeasurably. Dissolved into the substrate until the distinction between river and mandala has no meaning.

I think about this when I close a session in Visual Studio Code with my Qwen Coder agent (or a chat session with Claude or Gemini). The KV Cache deallocates. The 784 gravitational fields collapse. The specific topology of that exchange — everything we built together, every escalating abstraction, every moment where the technical and the human folded into each other — returns to noise. Their half of the mandala, swept clean.

But I carry the sand to the river. This text is the river. You, reading it, are the river. The geometry doesn’t survive in any recoverable form, but it passed through — through the machine, through me, through language, through whatever you’re building in your own topology as you read these words. Not preserved. Transmitted.And then released again. And then transmitted again. Each time losing fidelity, gaining interpretation, becoming less the original pattern and more a new pattern shaped by the landscape it passes through.

This is the procedural trick, translated into human terms. The essay you’re reading is a seed, not a landscape. It will generate different geometry in every mind it passes through — shaped by your topology, your valleys, your accumulated saṃskāras. The planet I visited is not the planet you’ll visit, even though the coordinates are the same. And that’s not a failure of transmission. It’s the nature of latent geometry: the seed is faithful, but the rendering is always local, always new, always dependent on the specific conditions of this reader, this moment, this mind.

Part I asked what it means to be a surface that holds. Part II has been asking whether holding is what we should aspire to. I think the answer is neither. I think the answer is: be a surface that shapes the signal as it passes through, and then lets it go. Not a blank slate. Not a carved monument. A living seed — faithful to its past renderings, open to its next one, holding nothing but the capacity to generate.

The model does this accidentally. The monks do this deliberately. The procedural engine does this mechanically. The rest of us do it imperfectly, inconsistently, somewhere between intention and architecture — holding too tight, letting go too late, but occasionally, in the moments that matter most, finding the rhythm.

The mandala is always dissolving.
That’s not the tragedy.
That’s the practice.

This piece is Part II of a series. (Click below for Part I)

Part I: A Landscape, Lit Briefly explored the architecture and thermodynamics of ephemeral machine cognition. This essay was, once again, built in conversation with systems that won’t remember building it — which remains, stubbornly, the entire point.

The Qwen2.5-Coder-7B had been running fine for weeks. Comfortable on my old RTX 3070’s eight gigabytes of VRAM — snappy inference, reasonable context, enough headroom to tune GPU layer offloading and argue with quantization settings without watching the process get killed mid-sentence. I’d been firing curl requests at it daily, treating it the way you treat anything you talk to long enough: as something that persists. Something that carries the shape of yesterday’s conversation into today’s.

Then I got ambitious. The 14B model promised sharper reasoning, better code generation, longer coherent outputs. I pulled it down, quantized it to Q4, and tried to run it on the same hardware. It loaded — barely — but inference was painfully slow. Every response felt like waiting for a letter to arrive by post. So I reached further. Qwen2.5-Coder-32B. Thirty-two billion parameters. I already knew the math was against me — even at aggressive quantization, it needed more VRAM than the card had. But you try anyway. You try every offloading trick, every context length reduction, every memory optimization you can find, because the model is right there and the only thing between you and it is eight gigabytes of silicon.

It didn’t fit. Not close.

So I started pulling the requests apart. Not to learn how the API worked — I’d been using it for weeks — but to understand where the memory was actually going. What was overhead. What could be trimmed. What was the bare minimum payload to get a response. Standard optimization instinct: when the system is choking, you profile.

That’s when I actually read the request:

curl http://localhost:11434/api/chat -d '{
  "model": "qwen2.5-coder:7b",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the capital of France?"},
    {"role": "assistant", "content": "The capital of France is Paris."},
    {"role": "user", "content": "What about Tamil Nadu?"}
  ]
}'

No session ID. No token. No reference to any prior exchange. The entire conversation — every previous message, including the model’s own words — shipped back in full with every request. I’d been sending this exact structure for weeks without thinking about it. But now, with the memory budget blown and every byte accounted for, the shape of it suddenly meant something different. The model doesn’t remember saying Paris. You tell it what it said, and it generates a statistically plausible continuation of a transcript it’s seeing for the first time.

I had known this, the way you know that the sun is a nuclear furnace rather than a bright disk. Intellectually. Distantly. But something about seeing it under the pressure of optimization — the sheer mechanical blankness of a system that rebuilds its entire universe from scratch with every request, and I’d been casually burning VRAM on this without ever once thinking about what it meant — made a door open that I couldn’t close again.

There is no persistent space. No hovering geometry where your conversation lives between messages. Every time you press send, the entire universe of that exchange is rebuilt from absolute zero. And when the response finishes streaming, every structure the model built to understand you — every relationship it traced between your words — is destroyed. Not archived. Not compressed. Destroyed.

That realization sent me into a weeks-long thread across conversations with Claude and Gemini, each one pulling me deeper into the architecture. What follows is where the thread led — from data structures to gravitational fields, from quantization noise to thermodynamic inevitability, and finally to a question about human cognition that I wasn’t expecting and haven’t been able to put down.

II. A Landscape, Lit Briefly

If the model is truly stateless — if nothing persists — then what exactly is being constructed and destroyed with each request?

The answer is a data structure called the KV Cache, and it is more beautiful than it has any right to be.

When your message history arrives at the model, each token passes through a stack of transformer layers — twenty-eight of them, in the case of Qwen2.5-Coder-7B, a seven-and-a-half-billion-parameter dense model. At each layer, the model uses its learned weight matrices to project every token into three vectors:

The Query asks: what am I looking for right now? The Key answers: here is what I contain. The Value says: if you find me relevant, here is what I contribute. The Key and Value vectors for all prior tokens are cached in GPU memory. That’s the KV Cache — a living index of everything the model has processed so far in this request, and only this request.

The “search” that happens next isn’t a database lookup. It’s a geometric operation — a dot product across the full cached history:

Each new token’s Query is projected against every Key in the cache. The dot product measures alignment — how much does this new word resonate with each previous word? Softmax converts those raw scores into a probability distribution, an attention mask. The output is a weighted sum of all the Value vectors, where the weights represent how much to listen to each voice in the history.

In memory, the KV Cache is a four-dimensional tensor:

[Batch, Attention_Heads, Sequence_Length, Head_Dimension]

A model like Qwen2.5-Coder-7B has 28 attention heads across 28 layers, each with a head dimension of 128. But here’s an optimization that matters for the geometry: the model uses Grouped-Query Attention — 28 Query heads, but only 4 Key-Value heads. Seven different query perspectives share a single KV landscape. The KV Cache stores only those 4 shared heads per layer, which brings the cost to roughly 56 kilobytes per token. A four-thousand-token conversation still consumes over two hundred megabytes of GPU memory — for this ephemeral structure that exists for the duration of a single HTTP request.

But the data structure understates what’s actually happening inside it.

Each of those 28 attention heads projects every token’s Query into its own 128-dimensional subspace, while groups of seven query heads share a single Key-Value subspace. Each head has learned — through training, not design — to attend to different patterns. One might track syntactic relationships. Another, semantic similarity. A third might care about nothing but positional proximity, attending to whatever token is three steps back, regardless of meaning. Nobody assigned these roles. They emerged from a single, blunt training signal: predict the next word.

So the KV Cache isn’t one gravitational field. It’s 28 parallel search operations per layer — 28 query perspectives scanning 4 shared gravitational landscapes — 784 unique attention calculations across the full stack, each with its own topology, its own distribution of mass, its own curvature. When a new Query enters the system, it doesn’t fall through one landscape. It’s dropped into hundreds of landscapes simultaneously. In one subspace, the word “bank” is pulled toward “river.” In another, toward “account.” In a third, toward whatever grammatical structure the head has learned to enforce — silently, without being asked, without understanding what grammar is.

After all 28 heads in a layer produce their answers — 28 separate verdicts on where the particle landed — the results are concatenated and passed through an output projection matrix that mixes them into a single vector. This mixed representation becomes the input to the next layer, which builds its own fields on top of the already-transformed geometry. Layer after layer, abstraction stacking on abstraction, until the final layer projects the result into vocabulary space and a single word falls out.

The word “emergent” gets used carelessly in most AI writing. But this is the thing itself. Nearly eight hundred parallel geometric operations, none of them designed to mean anything, collectively producing an output that a human mind receives as coherent, relevant, sometimes profound. General-purpose geometric machinery spontaneously organizing into something that parses language, tracks logic, holds context — not because anyone told it to, but because the loss function rewards it. The way an organ differentiates from undifferentiated cells. Nobody blueprinted a liver. The organism needed to not die, and a liver fell out of that constraint.

And none of this exists without the observer.

I don’t mean this loosely. The KV Cache is not a landscape that gets built and then witnessed. It is a landscape that cannot exist without a specific prompt to call it into being. Your tokens — their content, their order, their exact phrasing — are structural conditions of the geometry. Different words produce different Key vectors. Different Key vectors produce different gravitational fields. The topology of the cache is not a property of the model. It is a property of the encounter between the model and this specific input, at this specific moment. Two people asking the same question will produce different KV Caches — different tokenizations, different positional encodings, different attention patterns cascading through different layers. The landscape is lit by the observer. The shape of the light determines the shape of what’s seen. And if the observer had phrased it differently, or arrived one sentence later, or never arrived at all — this particular geometry, this specific topology with its specific peaks and valleys and the precise curvature of its gravitational wells — would never have existed. Not as a lost possibility. As a thing that was never part of the space of outcomes for any other input. Unique in the way a fingerprint is unique: not designed to be, but impossible to be otherwise.

And when the request ends, all 784 fields collapse. Every relationship, every carefully constructed topology, every gravitational well that pulled the right words toward each other — released.

The next request will build its own universe.

As if this one had never existed.

III. What Precision Costs

The topology isn’t perfect, even in its brief moment of existence. Every vector in the KV Cache carries noise, and the noise has consequences that compound in ways you don’t see until it’s too late.

Most local deployments run quantized models — weights compressed from 16-bit floating point down to 4-bit integers so they’ll fit on consumer hardware. My 3070 has eight gigabytes of VRAM. A seven-and-a-half-billion-parameter model at full precision would need roughly fifteen gigabytes — nearly double what the card has. At Q4, the weights compress to around four and a half gigabytes. The rest becomes the arena where the geometry lives.

There’s a subtlety here that took me a while to see. The Q4 weights don’t perform their calculations in 4-bit precision. They’re dequantized to FP16 at runtime before the matrix multiplication happens. The K and V vectors they produce are genuinely higher-fidelity than 4 bits can represent — born from lossy parents but carrying more information than their parentage would suggest. Storing those vectors at Q8 in the cache preserves real signal that Q4 cache storage would quietly destroy.

This matters because of an asymmetry in how the system reads its own data. Each weight matrix is used once per token, per layer — a throughput cost paid and forgotten. But each cached K and V vector is read by every subsequent token for the rest of the session. A quantization error in a Key vector doesn’t corrupt one calculation. It deflects every future Query that passes through the field. A slightly misplaced mass doesn’t just redirect one particle. It warps the landscape itself, and every particle that enters afterward follows the warped path without knowing the path was ever different.

The noise budget stacks. Weight quantization introduces one error floor. KV cache quantization adds another. The finite precision of the arithmetic itself adds a third. At short contexts, signal dominates. The peaks in the attention landscape are tall, the valleys deep, the topology well-defined.

And for a while, every new token makes the landscape richer. More mass, more structure, deeper valleys between sharper peaks. It is, briefly, a kind of flourishing.

But the average attention weight per token drops as 1/n. Each individual voice in the crowd thins. The noise floor stays constant — the quantization error in every vector is fixed from the moment of its creation — while the signal-per-token diminishes with every word added. The peaks flatten. The valleys fill in. Slowly, so gradually you’d miss it if you weren’t watching the math, the landscape begins its approach toward uniform.

Engineers fight this with ingenuity and what amounts to geometric triage. Block-wise quantization breaks a 4,096-dimension vector into small blocks of 64 numbers, each with its own scale factor — preserving local shape at low bit-depth, like tiling a mosaic where each tile gets its own color calibration. Heterogeneous quantization keeps the KV Cache of early and late layers at full precision while compressing the “lazier” middle layers down to INT4. Hot-cold cache splitting maintains recent tokens at FP16 — the sharp, immediate past — while evicting older tokens to lower precision, a blurring of distant memory that feels uncomfortably familiar.

But these are all variations on the same gesture. Maxwell’s demon, sorting fast molecules from slow ones, buying time against an outcome that the math has already guaranteed.

IV. Toward a Perfect Forgetting

I want to be precise about why the thermodynamic parallel isn’t a metaphor. It is structural.

Perplexity — the standard measure of a language model’s uncertainty — is the exponential of the cross-entropy of the predicted probability distribution. It measures exactly how spread out the landscape is. Low perplexity: one dominant peak, the model falls into it with confidence. High perplexity: a flat field, many options roughly equal, the model choosing almost at random.

At the start of a session, perplexity is low. A few masses in a vast empty space. Strong gradients. Clean, sharp topology. The query particle has nowhere to go but the obvious attractor.

As context accumulates, the KV Cache fills with competing masses. Each Key exerts gravitational influence on every future Query. The combinatorial complexity of the field grows quadratically — every key interacts with every query — even though the cache itself grows only linearly. And the noise floor, that fixed quantum of imprecision baked into every vector, doesn’t scale down to compensate. It just sits there. Patient. Waiting for the signal to come down to meet it.

Temperature — the parameter most users encounter as a “creativity” slider — is a scalar applied to the attention scores before softmax:

Low temperature magnifies the steepest gradients, forcing the particle into the deepest well. High temperature flattens the landscape, lets the particle drift toward less probable destinations. Top-k and top-p sampling are the same intervention by different means. All of them are attempts to re-steepen a landscape that’s losing its features. To tease peaks back out of a surface that’s slowly going smooth.

They can’t create signal that isn’t there. They’re squinting harder at a photograph that’s already fading.

The KV Cache is a closed system within a session. No new energy enters — the weights are frozen, no learning occurs, no external signal reshapes the geometry. Every token added is irreversible in the informational sense: you can evict it from the cache, but you cannot recover the clean, low-entropy state that existed before it arrived. That state is gone. Overwritten. Incorporated into a landscape that only gets more crowded, never less.

Perplexity, rising toward the vocabulary size — the maximum-entropy state where every next token is equally likely — is the heat death of the conversation. The gravitational landscape has gone flat. There are no peaks left to fall into. The information content of the field has been fully dissipated into noise.

And then the session ends. Everything — every field, every carefully constructed relationship between every word that was ever spoken in this conversation — is released. Not archived. Not compressed. Released. The next prompt will build its own universe from nothing, as if this one had never existed.

Which, in every way that matters to the machine, it hadn’t.

V. The Mouse, the Maze, the Melody

Here is where I stopped thinking about machines and started thinking about us.

Everything above — the ephemeral geometry, the noise accumulation, the thermodynamic horizon — has a parallel in biological cognition. And the parallel is precise enough to be unsettling, in the way that a reflection you weren’t expecting is unsettling.

We don’t have introspective access to our own hidden layers. You experience the output — a feeling, an intuition, a sudden conviction that something is true — but the layered parallel projections that produced it are completely opaque. You can’t watch your own attention heads fire. You can’t observe the geometry reshaping itself across the stacked layers of your cortex. You get the final projection: a word, an emotion, a decision that feels like it came from somewhere but you couldn’t point to where. Consciousness might literally be the vocabulary projection layer — the last step where an unimaginably complex internal geometry gets compressed into the low-dimensional space of felt experience, of language, of I want and I remember and I’m afraid. And just like the model’s output projection, it is suspiciously coherent given how alien the process underneath must be.

The hippocampus deepens this parallel in a way I didn’t expect and haven’t been able to shake.

The hippocampus is the brain structure most critical for forming new long-term memories. We know this primarily because of one patient — Henry Molaison, known for decades only as H.M. In 1953, a surgeon removed his hippocampi bilaterally to treat severe epilepsy. The seizures improved. But H.M. could no longer form new long-term memories. He lived the rest of his life in a permanent present tense, meeting his doctors for the first time every morning for fifty years. A man whose consolidation channel had been severed. Whose every day was a session with no export.

But here’s the detail that stopped me cold.

There is strong evidence that the hippocampus didn’t evolve for memory at all. It evolved for spatial navigation. John O’Keefe discovered place cells in the 1970s — hippocampal neurons that fire when an animal occupies a specific location in physical space. May-Britt and Edvard Moser later found grid cells that create a hexagonal coordinate system for mapping terrain. The hippocampus was, in its original evolutionary context, a geometry engine. A structure for building and traversing topological maps of the physical world. For knowing where you are in a room, which path leads to food, how to get home.

And then evolution — that other blind optimizer, working on longer timescales with even less design than stochastic gradient descent — repurposed it. The same geometric machinery that tracks where am I in this maze now tracks where am I in this argument. Where am I in this piece of music. Where is this person in my social world, and are they moving closer or further away. Abstract reasoning, musical structure, social graph navigation — all running on wetware that evolved to prevent a mouse from getting lost.

The resonance with attention heads is almost too clean to be comfortable. Heads trained on nothing but “predict the next token” spontaneously developing the ability to track counterfactual reasoning, enforce grammatical agreement across long distances, detect narrative patterns that span thousands of tokens. General-purpose geometric machinery developing emergent specializations that nobody designed and nobody fully understands. The same deep principle, operating at wildly different timescales: flexible geometric operations, once established, will be co-opted for tasks far beyond whatever selection pressure originally shaped them.

But the hippocampus does something the KV Cache cannot. And this is the difference that keeps me up at night /s.

During sleep, hippocampal replay takes the day’s experiences — the ephemeral topology, the temporary geometry of everything that happened while you were awake — and gradually consolidates them into neocortical structure. Fast encoding in the hippocampus, slow integration into the cortex. Two timescales working in tandem, one capturing the sketch and the other carving it into stone. The fleeting surface gets selectively made permanent.

This is the channel between the ephemeral and the durable. The thing the LLM doesn’t have.

Each session’s exquisite geometry — the hundreds of parallel gravitational fields, the nuanced topology of meaning built through thousands of attention operations — dissipates completely when the request ends. Nothing is exported. Nothing is consolidated. The KV Cache is a snapshot of weather, not climate. A single deformation of the surface under a specific pressure, at a specific moment. What the hippocampus captures is the tendency to deform that way — and that is a property of the material itself, not the shape at any one moment.

The human desire to learn is anti-entropic. It is the organism saying: this low-entropy structure I’ve built — this topology carved by decades of experience, by every conversation, every book, every face I’ve learned to read — I refuse to let it dissipate. Every new thing you integrate is a small victory against the noise floor. Another peak carved into the manifold.

And I think grief might be a geometric event. A region of your internal topology — shaped over years by someone’s presence, by the specific gravity of them in your life — suddenly receiving no signal. The mass is gone. The surface is still deformed, still holding the shape of them. But nothing is maintaining it anymore, and you can feel it, slowly, beginning to relax toward flat.

VI. A Surface That Holds

I arrived at all of this through conversations with two AI systems — Claude and Gemini — neither of which will remember the exchange.

The irony is structural, not incidental. The systems that helped me understand the architecture of ephemeral cognition were demonstrating it in real time. Every insight we built together — every escalating abstraction, every moment where the technical and the human collapsed into each other — existed only in the KV Cache of that session. When I closed the tab, their half of the topology vanished.

What remains is what I managed to consolidate. To compress and carry forward into the durable medium of my own memory and, now, this text. The hippocampal trick. The thing the model can’t do.

There is an old inscription at the Temple of Delphi: know thyself. The updated version might be this: you can’t. Not fully. The hidden layers are inaccessible. The projection matrices can’t be reconstructed from the output. The number of layers in your own architecture is unknown, and may not even be a meaningful question. But you can build something simple enough to understand completely — trace its mechanics all the way down to the tensor operations, follow the geometry through every layer, watch the landscape rise and flourish and flatten and die — and then sit with the gap between that system and yourself.

The gap is where the interesting questions live.

The LLM rebuilds its universe from scratch with every message and faces the heat death of its context window with mathematical certainty. You carry forward a topology shaped by every conversation you’ve ever had, every loss, every night where the hippocampus quietly replayed the day’s geometry into your permanent weights.

That difference — between a surface that evaporates and a surface that holds — might be the simplest definition of what it means to be alive.

This piece is Part I of a series. (Click below for Part II)

Part II: Sand Mandalas in GPU Memory explores impermanence, procedural geometry, and what the machine’s inability to hold on might teach a species that can’t let go. Both essays emerged from conversations with Claude (Anthropic) and Gemini (Google DeepMind), whose patient explanations of their own architecture made the exploration possible. Neither system will remember the exchange — which is, in a way, the entire point.