The Hive Still Makes the Chips

What Rorqual Maru, the Benthics, and some highly suspicious semiconductors can teach us about living with large language models.

I have read T. J. Bass's The Godwhale an unreasonable number of times. So many that I eventually acquired a second copy, presumably as a concession to the structural limits of paper. It is an odd, wonderfully inventive novel about an old whale-shaped harvesting ship named Rorqual Maru, a civilization called the Hive that has become rather too successful at controlling the world, and the various human beings and other creatures who keep demonstrating that life has a distressing tendency to escape the categories designed to contain it.

Rorqual is a vast creature of steel, biology and memory. There are ocean people, the Benthics, and there is a small mechanical companion whose job, among other things, is to remind an enormous artificial intelligence that the world contains details too small for an enormous artificial intelligence to notice on its own. I've been carrying those characters around in my head for decades. They keep turning up in our work with little robots, ships, local AI systems and the general problem of how to keep a machine from mistaking its assigned purpose for a moral right to rearrange the universe.

But there's another part of the story I've been thinking about lately.

Near the end, as I remember it, the people living out on the ocean have a rather practical supply-chain problem. They've escaped the Hive's social order, but they haven't escaped physics. Their ships can repair an astonishing number of things. They have ingenuity, salvaged machinery, tools, new customs, cooperative arrangements and plenty of reasons to remain free. What they don't have is a complete semiconductor fabrication industry. The Hive still does.

And so they trade.

This is an awkward commercial relationship because the Hive would very much prefer that the free people stop being free. It supplies useful chips, but the ocean people cannot assume that everything inside those chips is there to serve the buyer. In the version of this scene lodged in my memory, the Hive plays games with loyalty circuitry and delayed failures, trying to make the components misbehave or burn out when the people on the ships depend on them.

You can almost picture the procurement meeting. The customer wants navigation components and the supplier would like the customer to surrender political autonomy, preferably before lunch. The customer asks about warranty terms. The supplier explains that the warranty is contingent upon the eventual restoration of proper civilization, as defined by the supplier. Somebody from the Benthics suggests opening the package and looking at the board. The Hive considers this a hostile act.

I am embellishing that meeting, obviously. Bass gave us the world; the procurement dialogue is mine. The problem is not imaginary.

The ocean people learn an important lesson. They needn't decide that every chip the Hive makes is worthless because the Hive is untrustworthy. They also needn't accept the Hive's claim that manufacturing the component entitles it to govern the person who installs it. They can inspect, test, rework where possible, isolate what they cannot verify, keep spares, build around known failure modes and sometimes decline a part altogether. They can take useful capability out of an unfriendly supply chain without making the supply chain the captain.

Which brings us, by a somewhat improbable but surprisingly direct route, to large language models.

A model arrives with a history

There is a lot of discussion about who trains LLMs, what went into their training, whose intellectual work was used, what values were promoted, which political or commercial incentives shaped the process, what data was excluded, and how much trust a particular provider deserves. All of that matters. A model is not a naturally occurring mineral we pick up from the beach. People built it, with resources, objectives, decisions, omissions and economic interests.

But the argument sometimes gets stuck at the factory gate. We talk as though the company that trained a model is forever identical to every possible use of that model, or as though a questionable origin proves that no useful work can be done with the artifact after training. That seems to me like a terrible way to run a civilization with a limited supply of chip fabs.

A trained LLM is not literally a semiconductor carrying one discrete loyalty circuit you can desolder with a steady hand and an unusually confident crab. Its behavior is distributed across a great many learned parameters and shaped by the prompts, context, tools, software and infrastructure surrounding it. Some limitations are obvious. Some only appear under pressure. Some undesirable tendencies are difficult to diagnose, let alone remove. You cannot guarantee a model is free of backdoors or harmful capabilities because it performed politely for ten minutes. And provenance questions—consent, licensing, ownership of training data, obligations to affected people—do not disappear because you found a clever way to use the finished model.

Those are reasons for more serious engineering, not for surrendering the engineering to the vendor.

Once a model exists, we can study what it does. We can inspect its documented origin and license where those are available, test it against known tasks, compare its behavior with other models, constrain what information we give it, measure its failures and decide where, if anywhere, it belongs in our systems. We can sometimes run it locally. We can place it behind our own interfaces. We can ask one model to analyze something and a different system to verify the resulting proposal. We can change models when the work changes instead of changing the whole organization's identity to match the model supplier.

This is a bit like receiving a box of chips from the Hive with a very impressive specification sheet and an equally impressive history of manipulating its customers. You don't have to throw every chip into the ocean. You also do not plug one directly into the ship's steering gear, give it the captain's credentials and hope that whatever loyalty circuitry might be present has developed a sudden affection for maritime freedom.

You build a test bench.

More importantly, you build a system in which the component's usefulness does not depend on trusting it with every consequence of its own outputs.

The steering gear belongs to the ship

We have been building QuietWire around a simple distinction: a language model can provide cognition without automatically receiving authority. A small robot can ask a larger system what something in an image might be, then keep the answer as attributed evidence rather than a command to turn left. An AI companion can propose a change to a real server while the actual ability to execute it remains tied to a separate identity, a bounded scope, a present authorization and a receipt that shows whether the operation succeeded. A model can help interpret a community's records without becoming the owner of those records or the final judge of what the community remembers.

These arrangements are not foolproof. Any sufficiently complicated assembly can fail. The model may hallucinate, overlook a constraint, be manipulated through an input, or propose a perfectly plausible course of action that would be disastrous in the actual environment. The surrounding software may have bugs, and the people may make errors. But the model is no longer the one thing that must be perfectly trusted for the entire system to behave acceptably.

That changes the kind of trust we are asking for.

For a low-consequence writing task I may be quite happy to use a model that makes delightful sentences and occasionally misunderstands a metaphor. For medical decisions, financial transfers, private records, critical infrastructure or a large tracked excavator, the same model's eloquence is not nearly enough. The proper boundaries depend on what it can affect. We need to know which information it may see, which external systems it may reach, which claims require checking, which actions require human permission, how to stop it, and what evidence will survive if something goes wrong.

Sometimes the correct disposition is do not use this chip. A part with an uncontained defect does not become safe just because we love its other features. The point is to make that decision ourselves, based on observed behavior, the actual risk of the intended use and the authority we are willing to grant, rather than treat a model's origin as either a universal certificate of virtue or a permanent curse.

The ocean people can't manufacture everything the Hive manufactures. They can still build a different world with some of those components. They just need to do the work that makes a component into a trustworthy part of a larger system.

That is the part of the analogy I find most useful.

Build the ship, not a religion about the chip

There is an odd temptation, when a new language model appears, to decide whether it is one of the Good Ones or one of the Bad Ones. We ask whether the company is virtuous, whether the training is pure, whether the model shares our beliefs, whether it will always tell the truth, whether it can be made safe by choosing exactly the right words. Then we discover a month later that the model has changed, or the vendor has changed its terms, or somebody has found a new failure mode, and the theological committee has to meet again.

I would rather know which jobs the thing can do, which ones it cannot, and what it is permitted to touch.

That is not indifference to who controls the factory. Quite the opposite. We should care a great deal about concentration of compute, the provenance of training material, labor, energy, the ability of smaller communities to obtain useful models, the cost of running them, and the practical power of vendors to revoke or modify services on which other people have come to depend. Those are economic and political questions. They are also architecture questions. If one organization can change an essential component and thereby silently change what your organization knows or can do, then you have a dependency worth understanding before it becomes an emergency.

We don't solve that by pretending every community is going to build its own frontier-model training cluster any more than the Benthics were going to build a clean room on the back of Rorqual Maru. Some may. Many won't. Most of us will use a mixture of local and external capabilities, specialized and general models, products we can inspect deeply and services we can only test at their interfaces.

What matters is whether we can maintain our own continuity, our own evidence, our own permissions, and our own ability to change suppliers without abandoning the work or the people who depend on it.

I keep returning to the picture of that impossible ship: Rorqual Maru moving through the open ocean with a little mechanical companion, a growing population of free people and a cargo of useful, suspicious parts from a factory that has not earned their loyalty. They have no need to pretend the Hive is harmless. They have even less reason to let it define what freedom means.

They can trade with it. They can test what they buy. They can build protections around the things they choose to use. They can learn to manufacture more for themselves over time. They can remain connected without remaining obedient.

And if one of those chips begins suggesting that the captain might like to report back to the Hive every seventeen seconds, somebody can take out the screwdriver.

Preferably before it reaches the steering gear.

The Hive can keep making chips.

The ship gets to decide what the chips are allowed to do.