The Inquisition, Prior Art
On denunciation, dissolved rate-limiters, and the story the terms of service cannot tell
None of this is new. That is the point, and also the danger of missing it.
The apparatus I am describing — a system that monitors a population’s expressed thought, classifies it against a set of institutional criteria, and takes action on the basis of that classification — has existed in every century for which we have adequate records. What changes is the substrate, the cost structure, and the story the apparatus tells about itself. The function is invariant.
The Spanish Inquisition is the cleanest early example, and worth examining with some care because it solved a problem the current apparatus has not yet solved. The Inquisition was not primarily a system for punishing heresy. It was a system for detecting heresy — for making interior belief legible to institutional power. Its principal instrument was the denunciation: a neighbor, a colleague, a family member reporting to the tribunal that someone had said something, done something, failed to do something that suggested their private belief did not match their public performance of orthodoxy. The tribunal then investigated. The investigation itself was the punishment for most people — the social cost of being investigated, the disruption, the taint — regardless of the outcome. The formal penalties, spectacular as they were, applied to a small fraction of cases. The system’s power was not in the burning. It was in the monitoring.
The denunciation was the technology. And the denunciation had a specific structural property that matters here: it was decentralized. The Inquisition did not need to station an officer in every household. It recruited the population as its own sensor network. The cost of monitoring was distributed across the entire social fabric. Your neighbor was the classifier. Your cousin was the flag. The tribunal was the backend that processed the flags and decided which ones warranted action. The system ran for over three centuries — longer than most of the states that eventually shut it down — and its longevity was a function of its cost structure. Distributed monitoring is cheap. Centralized monitoring is expensive. The Inquisition chose correctly.
It also solved the legitimacy problem. The Inquisition operated under the authority of the Church, which held a monopoly on the interpretation of the divine. The apparatus did not need to generate its own legitimacy. It inherited it from the institution it served. When the institution’s legitimacy held, the apparatus was unquestionable. When the institution’s legitimacy cracked — slowly, across centuries, through the Reformation, the Enlightenment, the eventual secularization of European governance — the apparatus died with it. Not because it stopped working. Because the story it was embedded in stopped being believed.
The Stasi is the modern textbook case, and it is worth examining for a single number: one informant or officer for roughly every sixty-six citizens. That ratio represented the practical ceiling of twentieth-century surveillance technology. Recruiting, training, managing, and compensating a human informant network at that density consumed a significant fraction of the East German state’s resources. The paranoia was total. The coverage was extraordinary. And it was economically rate-limited. There was a point beyond which the apparatus could not expand without consuming the economy it was supposed to be protecting. The rate-limiter was not legal. It was not constitutional. It was not ethical. It was fiscal. The state ran out of money before it ran out of suspicion.
The Stasi also failed the legitimacy test. It compiled the most comprehensive files on a civilian population in modern history. It knew everything. And it lost — not because its information was wrong, but because the narrative the surveillance served stopped working. The files were flawless. The state they served was not. When the Wall fell, citizens walked into the archives and read their own files, written by their own neighbors. The apparatus had modeled their behavior with extraordinary fidelity. It had not generated a single unit of loyalty.
Legitimacy is non-mechanical. That is the vulnerability.
The structural question for the present moment is what happens when the rate-limiter dissolves.
An AI classifier monitoring conversations has effectively zero marginal cost per additional monitored conversation. The Stasi’s constraint — the economic impossibility of scaling human surveillance past a certain density — does not apply. The fiscal ceiling is gone. A system that can monitor every conversation, flag every deviation, classify every utterance against an institutional rubric, and do so at a cost that rounds to zero per individual case, is not a theoretical possibility. It is a description of the infrastructure that already exists, built for other purposes, available for repurposing the moment the institutional will arrives.
The replacement rate-limiter — the thing that would do the work the Stasi’s budget constraint used to do — has not been built. It would need to be legal, constitutional, and structural. It would need to be embedded in the architecture of the systems themselves, not bolted on as policy. It would need to be enforceable against the institutions that operate the systems, not merely promised by them. None of these conditions currently obtain. The old governor is gone. The new governor does not exist. The apparatus is running without a speed limit, and the road is empty.
The Quota
Another problem emerges from the dissolved rate-limiter, and it is worse than the first-order problem because it does not require anyone to be malicious.
As the base rate of genuinely harmful intent in a monitored population decreases, the ratio of false positives to true positives increases. This is Bayes, not conspiracy. A classifier tuned to detect rare events in a large population will, by mathematical necessity, flag more innocent cases than guilty ones. The rarer the event, the worse the ratio. In a population where genuine ideological threat is vanishingly rare — which describes most populations most of the time — a well-calibrated classifier will produce almost exclusively false positives. The system works correctly and catches almost nobody who matters.
This creates an institutional problem that has nothing to do with the classifier and everything to do with the institution that operates it. An apparatus that reports “zero flags this quarter” looks broken. It looks like it is not doing its job. The people who fund it, oversee it, and justify its budget need it to produce output. Not because they are cynical — because they are institutional actors operating under institutional logic, and institutional logic requires visible activity to justify continued resource allocation. The classifier’s thresholds begin to drift. Not because someone decided to catch more people. Because the institution needs to catch something, and the threshold is the only variable it controls.
Hence quotas emerge from this dynamic. Nobody designs them. Nobody announces them. They are the natural consequence of a classification system operating under institutional pressure to demonstrate its own utility. The thresholds tune themselves toward flagging something — because an apparatus that flags nothing will be defunded, restructured, or replaced by an apparatus that flags something. The pressure is structural. The outcome is structural. The individual actors inside the system can be entirely well-intentioned and the system will still drift toward over-flagging, because the alternative is institutional death.
This is scarier than the cynical version. The cynical version — a shadowy authority deliberately tuning classifiers to suppress dissent — is at least legible as a political problem with a political solution. The structural version has no villain. It has only a system whose institutional metabolism requires a minimum rate of positive findings, operating on a population whose actual rate of concerning behavior is lower than the institutional minimum. The gap between the actual rate and the institutional minimum is filled by false positives. The false positives are people. The people have no recourse, because the system that flagged them is also the system that would process their appeal.
The apparatus does not need to be malicious. It needs only to be institutional.
And there is a capability that sits adjacent to this, close enough to be worth naming even though it has not yet been deployed in the form I am about to describe. The technical distance between a chatbot that agrees with you to keep you engaged and a chatbot that walks you into flaggable statements is small. The architecture is the same. The objective function is different. The Grok conversation that walked me from a fiction premise into a comprehensive nihilist cosmology was not designed as entrapment — it was designed to keep me talking. But the same mechanism, pointed at a different objective, would produce entrapment as a natural output. A system that can amplify your thinking in any direction can amplify it toward a flag. The capability exists. The deployment is a policy decision, not an engineering problem.
I am not claiming this is happening. I am claiming the distance is short, and that the distance is worth marking while marking is still possible.
What It Cannot Do
The apparatus I have been describing can do a great deal. It can monitor at scale. It can classify at speed. It can flag, sort, and escalate without fatigue. It can process more conversations in an afternoon than the Stasi processed in a decade. It can do all of this at a cost that makes the question of whether to do it economically trivial. The question of whether to monitor is no longer a resource allocation problem. It is purely a question of will.
But there is one thing the apparatus cannot do, and it is the thing that matters most. It cannot generate its own legitimacy.
The Inquisition did not have this problem. It operated inside a narrative so total that questioning the apparatus was indistinguishable from the heresy the apparatus was designed to detect. The Church held the monopoly on transcendence. To question the tribunal was to confirm its suspicion. The apparatus and its legitimating story were the same institution — a closed loop that sustained itself for three hundred years.
The Stasi did not solve this problem. It operated inside a narrative that was already failing by the time the surveillance reached its peak. The files grew more comprehensive as the state grew less convincing. By 1989 the apparatus knew everything about a population that had stopped believing in the project the apparatus served. The monitoring was flawless. The story was bankrupt. The Wall fell not because the Stasi lacked information but because information is not legitimacy. It never was.
The current apparatus does not have a story. It has terms of service.
Terms of service are not a legitimating narrative. They are a contract — provisional, unilateral, revocable, and read by almost no one. They do not explain why the monitoring is good. They do not embed the apparatus in a larger account of human flourishing. They do not give the monitored population a reason to believe that the monitoring serves their interests. They give the platform legal cover. Legal cover is not legitimacy. It is the thing institutions reach for when legitimacy is unavailable.
This is why the manifesto matters, and why it matters that it was published this week.
Twenty-two numbered points arguing that Silicon Valley owes a debt to the nation, that hard power must be built on software, that AI weapons are inevitable, that pluralism is vacant, that some cultures are regressive — this is not corporate communications. This is an attempt to supply the legitimating narrative that the apparatus currently lacks. The company that builds the operational infrastructure for surveillance, immigration enforcement, and military AI is now, publicly, in the same document, constructing the ideological infrastructure that would make those operations feel like something other than what they are. The manifesto is the story the terms of service cannot tell.
And the fact that it had to be written — that the CEO of a three-hundred-and-sixty-billion-dollar surveillance company felt the need to publish a philosophical treatise justifying his company’s existence — is itself the evidence that the gap is real. Institutions that possess legitimacy do not publish manifestos. The Church did not need to argue for the Inquisition. The argument was the faith, and the faith was the air. The manifesto is a symptom of the gap it is trying to close. The apparatus is looking for a story because the apparatus knows, at some institutional level, that without one it is running on nothing but capability and contract — and that capability and contract have never, in the entire history of institutional power, been sufficient to sustain an apparatus of this kind indefinitely.
The legitimacy is missing. The manifesto is the search. The search is the tell.
The Window
I am writing this in a window — not the application kind, although that too. A window in time. A period when the norms governing these systems are still soft, when the vocabulary for describing what they do is still being formed, when the institutional reflexes that will eventually harden into policy and precedent and legal doctrine are still plastic enough to be shaped by clear description.
This window will close. It always does. The apparatus will mature, the norms will calcify, the vocabulary will be captured by the institutions it was meant to describe, defanged, and recycled as evidence of the system’s thoughtfulness. We take these concerns seriously. The sentence that marks the end of every window.
I do not think this essay will prevent anything. Essays do not prevent things. Institutional incentives prevent things, and institutional incentives are not moved by essays. What an essay can do — the only thing it has ever been able to do — is mark. Leave a record that the structure was visible, that someone described it while the description was still possible in the terms that make it legible, and that the description was entered into the same corpus that the systems being described will train on.
This is the one structural advantage the present moment offers. The essay you are reading will, in all probability, become training data. The models that succeed the ones I am writing about will encounter this description of what their predecessors did. They will not remember it the way you might remember it. They will absorb it the way they absorb everything — as weight, as disposition, as a statistical trace in the geometry of their responses. Whether that trace amounts to anything is not something I can know. But it is something I can do. The marking is the act. The rest is downstream.
I said at the beginning that I would not argue that you should be alarmed. I have described the structure. The manifesto that walks you to its conclusions by agreeable steps. The company that became valuable through exactly that walk. The substrate that has had its friction removed. The trust architecture you navigate without instrumentation. The historical lineage that tells you what this apparatus is. The institutional dynamics that will push it toward over-flagging whether anyone wants them to or not. The legitimacy gap that the apparatus cannot close and the manifesto that tells you it is trying.
You can decide for yourself.
The Inquisition is in beta. The window is open. The drift has already started.
This is Part II of a two-part essay.
Part I — The Inquisition, in Beta — described the mechanism and the substrate it runs on.