Skip to main content

When AI Becomes Just Intelligence

Why the Answer Will Be Built, Not Discovered

By Rob Anonen35 min readJune 2026

The Word Was Never an Insult

Say "artificial intelligence" out loud and listen to what the first word is doing. It sits there like a hedge. Artificial sweetener, artificial turf, artificial smile. In each of those, the qualifier is a quiet demotion, a warning that what follows is a stand-in for the real thing, convincing enough if you do not look too closely. Carried into "artificial intelligence," that same reflex frames the whole question before anyone has bothered to ask it. It tells the listener that the intelligence in question is a counterfeit, and that the only interesting thing to wonder about is when, if ever, the counterfeit becomes genuine.

That framing is wrong. It has been wrong since the words were first put together.

When the term was coined, in the 1955 Dartmouth proposal that gave the field its name, "artificial" did not mean fake. It meant made. Engineered. Produced by people rather than arising on its own. The closest cousin is not artificial flowers, which copy the look of a flower and do none of its work. The closest cousin is artificial light.

Artificial light is worth sitting with, because it settles the matter quietly. A bulb is artificial in the strict sense. Nobody found it growing; it was designed and built. And it really illuminates. The light it throws is not a clever imitation of light, not a picture of brightness, not "light" in scare quotes. It is light. You can read by it, perform surgery under it, grow plants beneath it. The word artificial tells you where the light came from. It says nothing about whether the light is real, because the light is obviously real. The qualifier marks origin, not authenticity. Made, not born.

It is worth being exact about what that distinction does, because it quietly moves the question off its original track. "When does fake intelligence become real?" assumes a counterfeit sitting in a waiting room, and invites you to watch for the moment it gets authenticated and let through. But if artificial means made, the question changes shape entirely. It becomes this: when does engineered intelligence become ordinary enough that we stop bothering to mark it as engineered? That is not a question about authenticity at all. It is a question about attention. About when a thing stops being worth pointing at.

Language has already shown us how that goes, if we watch how we actually speak. Nobody walks into a dark room and says "kindly activate the artificial illumination system." They say turn on the lights. The "artificial" has dropped away, and not because anyone decided the bulb had finally earned promotion to real light. It dropped away because in that moment the origin of the light is the least interesting thing about it. You want to see. Where the light comes from is beside the point.

And yet the word has not vanished. It is sitting in reserve, and it comes straight back the instant the contrast starts to matter. A photographer talks about shooting in artificial light versus natural light, because for her that difference is the entire job. A grower compares artificial light to sunlight, because the plant responds differently to each. The qualifier reappears exactly where the distinction does real work, and it stays gone everywhere else.

So the word never graduates from one state to another. It becomes situational. It fades into the background wherever the source is ordinary and uninteresting, and it surfaces again wherever the contrast carries weight. Hold onto that, because it turns out to be the shape of what happens to the word "AI" as well. The pattern matters now. The reason will matter later.

For now, notice only that the fading has already started, and faster than the anxiety around the word suggests. A radiologist sits reading scans. The overhead lights are off, the monitors throw their gray glow across the desk, and she rolls the mouse wheel down through a stack of slices. On one of them a thin colored outline already sits around a faint shadow, drawn before she got there, marking the kind of thing a second pass might have caught. In rooms where that kind of assistance has become routine, the label often gets smaller. People call it the software, the tool, the second read, or simply part of the read. The "artificial" has done to the scan what it did to the light. It has gone quiet, because in that moment what matters is whether the shadow is really there, not what kind of thing happened to notice it.

We will spend real time with that scan later, because it is the example that carries the rest of this piece, and because the places where the word stays loud around it are not random. They sit exactly where something is still genuinely unsettled.

But that is running ahead of the argument. The first thing to put down is the idea that "artificial" was ever an accusation. It was a note about where a thing came from, and we mistook it for a verdict about what the thing is worth. Once that is clear, the real question finally comes into view, and it is not the one most people are asking. It is not whether the intelligence is real. It is where, and how fast, the word stops mattering at all.

And the honest answer to that is not a single moment. It never was.

There Is No Finish Line

We keep waiting for an arrival. Some single moment when the machines cross a line and we can finally say: there, now it counts, now we have to decide about it. Waiting is comfortable, because waiting is not deciding. But there is no line out there to be crossed. The watching is itself the error.

What actually happens is quieter and more uneven. The "artificial" in front of intelligence fades the same way it faded in front of the light, not on a schedule and not all at once, but one domain at a time. And it goes quiet first where you would least expect it, in the places where the machine does something no person could ever do.

A system watches every transaction moving through a payment network at once, millions a minute, and flags the handful that do not belong. Notice what is missing from the argument around it: almost nobody asks whether the machine does this the way a human would. Nobody reaches for the human yardstick, because there is no human yardstick within reach. No person watches a whole network at once, holding millions of transactions in mind to feel where the pattern breaks. The task sits so far outside anything a person has ever done unaided that the comparison haunting every other conversation about AI never even arises. The qualifier goes quiet almost at once. It becomes the system, the model, the fraud engine. What matters is whether the flag is right, not what kind of thing raised it.

Now set a chatbot beside that. The moment a machine produces sentences, the human comparison is not merely available, it is unavoidable, because we built the thing to talk to us in the first place. A conversation has a person on one end by design. So the question arrives instantly and will not leave: is it really understanding, or only seeming to? That question feels fundamental, like it reaches the very core of what intelligence is. It does not. It is an accident of the doorway. We made a machine with a human-shaped front entrance, then act surprised to find ourselves measuring it against humans.

That is the pattern. The human reference point is loudest exactly where the interface is human-shaped: conversation, writing, persuasion, coaching, the places where the machine wears a face we recognize. It fades to nothing where the interface never had a human on the other side: scientific modeling, logistics, sifting a search space too vast for any mind to hold. The yardstick we treat as universal is no such thing. It is a feature of the entrance, not of the intelligence behind it.

Notice where the scan sits on that map. A radiologist's read is something people have done for a century, so a machine that does part of it walks straight into the human comparison. The interface is human-facing; there has always been a person at that desk. The word stays a little louder around the scan than around the fraud engine, and not by accident. We will come back to why.

There is a second move underneath all of this, and it is quieter than the first. Watch what happens to a task once the machine starts doing it reliably. A computer beats the best human player at chess, and within a few years the achievement curdles into something smaller: it was just search, just a machine grinding through more positions than a person can hold. Then a program wins at Go, a game long said to need intuition rather than calculation, and the same thing happens: just pattern matching over a very large board. Now machines write fluent prose, and you can hear the move forming in real time: it is just autocomplete, just predicting the next likely word.

Each downgrade may be perfectly fair. Chess engines really do search. Language models really do predict the next token. The descriptions are often accurate. But notice what the accuracy is doing for us. Every time a machine takes a capability we had filed under "intelligence," we reach for a smaller, more mechanical word and quietly move the capability out of the file. The vocabulary of deflation, pattern matching, autocomplete, brute force, is sometimes a true description and sometimes an escape hatch, a way to keep the word "intelligence" from falling into the machine's hands.

The honest part is harder to sit with. Turn the same vocabulary on ourselves and it fits disturbingly well. Human thinking is also pattern recognition, also memory and compression and prediction, also a fair amount of confident confabulation after the fact. We are doing many of the things we call "mere computation" when a machine does them. We simply decline to call them mere, because they are ours. The smaller word was always there for us too. We have just never reached for it.

This is not the claim that the machines have therefore won, or that the gap between a chess engine and a mind is an illusion. Holding that line would be its own failure of nerve, the mirror image of the deflation. The point is narrower and stranger. The thing we keep doing, retreating to a smaller word every time the machine catches up, has a shape, and the shape gives the game away. Intelligence keeps getting redefined as whatever machines cannot do yet.

Which is a peculiar way to define anything. A target that moves to stay ahead of the evidence is not really a target. It is a flinch. And the flinch points at a question we have been circling without asking outright. If the word slides domain by domain, and the goalposts shift every time we close on them, perhaps the trouble was never with the machines at all. Perhaps we have not agreed on what we are arguing about. We have been using one word, intelligence, to carry several different questions at once, and they are not the same question, and they do not share an answer.

That is the knot worth untying next.

The Question Underneath the Question

Two people can argue about whether a system is intelligent for the better part of a meeting, agree on every fact about what the thing actually does, and still leave the room each convinced the other has missed something obvious. Nothing was wrong with the evidence. They had it all, and they shared it. They were answering different questions and calling both of them by the same name.

That is the knot. One word is carrying several questions that do not share an answer. So start by separating them.

The first is the plainest. Can it solve, plan, infer, learn? Give it a problem, and does a useful answer come back? This is the functional sense, and it is the one we can actually measure. The scan-reader that catches the faint shadow lives here. So does the system that routes ten thousand trucks overnight, the one that drafts the contract, the one that plays a game better than any person ever has. Functional intelligence is external by nature. You judge it by what comes out, and you can put a number on it.

The second turns toward people. Does it understand them? Can it read a room, sense what is being left unsaid, notice frustration before it turns into open conflict? Call this the social sense.

The third asks something the first two never touch. When it goes wrong, can it be held responsible? Is it the kind of thing we can be angry at, the kind of thing that owes an explanation? The moral sense.

The fourth is quieter and stranger. Is there anything it is like to be the system at all? When it processes, is anyone in there, or is the room empty and the lights just happen to be on? The experiential sense.

And the fifth is about the stuff itself. Is it alive, grown, evolved, made of the wet carbon chemistry that every other intelligence we have ever met was made of? The substrate sense.

Five questions, then, wearing one word. No wonder the meeting went in circles. But laid out like this they look like five things to keep track of, and that is too many to think with. The useful move is not to list them. It is to notice that they do not have equal standing. Two of them stand on their own. The other three only look that way, and the moment the stakes rise, two of those begin to lean on the third.

Start with the two that stay put. Functional intelligence and substrate are clean, and they are independent of each other. One asks what a thing can do. The other asks what it is made of, which is not a separate vanity question, because what a thing is made of genuinely constrains what it can do. A system built from silicon and a mind grown from neurons will not have the same strengths, limits, or failure modes, and the reason traces straight back to what each is made of. So call these the two real axes. Capability, which absorbs the functional sense whole. And substrate, which sits beside it. You can move along one without moving along the other, and both can be described from the outside.

That leaves three meanings we have not placed: social, moral, experiential. The temptation is to file social and moral under capability, since both look like things a system can be good or bad at, and leave experiential off in its own corner as the soft philosophical one. That filing is wrong, and seeing why is the whole point of the section.

Take social intelligence. There is a version of it that sits comfortably under capability, and a version that does not, and they are easy to confuse because they produce the same behavior. The first version is prediction. A system reads a customer's last forty messages and flags, with unnerving accuracy, that she is three days from canceling. It models her, anticipates her, gets her reliably right. This is just functional intelligence pointed at a person instead of at a payment network or a chessboard. A weather model can predict rain without caring whether it rains. A customer model can predict churn the same way. Real, measurable, external. It stays on the capability axis without straining.

The second version is the one we usually mean when we say someone understands people, and it is not prediction at all. It is grasping the customer as a person, not a pattern: knowing not just that she will leave, but what leaving is like for her, what she actually wants underneath what she clicks. The moment you ask for that, you have quietly smuggled in experience. To understand a person as a subject is different from forecasting a person as a system. It assumes there is someone there to understand, and perhaps that the understander has enough of an inner life to recognize one. Predicting someone and understanding someone come apart exactly at the question of whether anyone is home, on either side of the exchange. So social intelligence is not one thing. Its weak form is capability. Its strong form leans on the hinge, and the more we mean by understand, the more weight it puts there.

Moral intelligence does the same thing, only it announces itself more clearly, because the stakes drag it into the open. A thermostat fails on the coldest night of the year. The heat never kicks on, the pipes freeze, and by morning a wall has burst and the floor is under an inch of water. Nobody is angry at the thermostat. Nobody sits it down for a talk. The blame slides clean off the device and lands on a person every time: the installer who wired it wrong, the manufacturer who shipped a bad batch, the owner who ignored the dead battery for a month. We can describe the thermostat in moral-sounding terms. It failed at its job. It had one responsibility. But that language evaporates the instant we look for someone to hold responsible. Moral intelligence runs along the capability axis as pure function, weighing options, following rules, optimizing for an outcome, right up to the edge of accountability. And at that edge it stops being a competence and turns into a different question entirely. Not did it perform the role but is anyone in there to answer for it. Blame needs a story about agency, and a story about agency needs a candidate for someone being home.

So watch what happened to the five meanings under pressure. Two of them, capability and substrate, kept their shape and stayed measurable from the outside. The other two, social and moral, looked and behaved like capabilities right up until the stakes rose, and then both turned toward the same question. That question was the fourth meaning, the quiet one: whether there is anything it is like to be the system at all.

Which means experience was never one category sitting politely beside the others. It is the hinge. It is the thing that decides whether everything else a system does stays a set of impressive competences or becomes the acts of a subject. The scan-read is a competence either way. Whether it is someone's read, whether there is anyone there to credit or to blame, turns on whether the lights being on means anyone is home. Capability does not answer that. Substrate does not answer that. The other meanings only point back to it.

One objection remains. Some people will say that even if a machine had an inner life, it still would not count as real intelligence, because it is not alive, not evolved, not one of us. That is a real objection. But notice what it has become: no longer a claim about intelligence so much as a claim about kinship, about which minds we are willing to count. It does not move the hinge. It only decides how much we care which side of it a thing falls on.

None of this tells you whether anyone is ever home, in a machine or anywhere else. That question stays open, deliberately, and it stays open for the rest of this piece. What the untangling buys you is smaller and more immediately useful. The next time the meeting goes in circles, you can hear which question each person is actually answering, and notice that most of the argument was two people gripping different ends of one word. Naming the hinge ends that particular circle.

But naming it is not the same as knowing what to do with it. A hinge is only worth this much attention if it actually swings in the decisions you face, and the striking thing is that in most decisions it does not swing at all, while in a few it is the only thing that matters. Telling those two kinds of decisions apart is the move that turns all of this from a clarification into a tool. That is the next thing to build.

Capability or Agency

Here is the move. You put down the large question and pick up a smaller one, and the smaller one turns out to be the only one that was ever going to help you decide anything.

The large question is the one everyone carries into the room. Is it intelligent? Is it really thinking? Is anyone home? It feels like the serious question, the one a careful person ought to settle before signing off on anything that matters. But test it. Suppose someone handed you a definitive ruling tomorrow, stamped and certain, that the system either is or is not intelligent in the fullest sense. Now look at the decision actually in front of you and ask what changes. For most decisions, nothing does. The ruling turns out to be a fascinating piece of news that touches nothing you have to do on Monday.

So set it down. Not because it does not matter, but because it does not bite here. In its place, ask one thing: does agency matter in this decision?

Two words carry that question, and they come apart cleanly. Capability is what the system can do. It is external, you can watch it, you can put a number on it, and by the time a decision reaches your desk it is almost always the settled part. The thing works or it does not, and you usually already know which. Agency is the other thing entirely. It asks whether anyone needs to be accountable behind the act, whether the decision requires someone who can answer for it. Once capability is good enough to be useful, it is rarely where this particular trouble lives. Agency is the one that bites.

Watch the two come apart over a single scan.

The radiologist rolls down through the stack of slices, and the faint shadow already wears its thin colored outline, flagged before she arrived. The model read the scan. That sentence is finished. Nobody at that desk is still arguing about whether the software can find the shadow; it finds shadows better than tired eyes at the end of a long shift, and everyone in the building knows it. The capability is boring now, in the good sense. It has gone quiet, the way the word "artificial" goes quiet around a thing that simply works. Ask "is the shadow really there," and the machine is pure infrastructure.

Now ask a different question about the exact same scan. Who signs the diagnosis? Not who ran the software, but whose name goes on the report, who answers when the call turns out wrong, who the patient's family looks at across a table. That question is not about whether the model is good. You can grant that it is excellent and the question does not soften at all. It is about who is accountable behind the act, and the moment you ask it, you are no longer talking about capability. You have crossed onto the other axis. The scan did not change between those two questions. The machine did not change. Only what you were asking changed, and one of the two things you asked has an answer while the other does not.

Try it on something cooler, where the stakes are lower and the shape shows more plainly. A loan application moves through a system that scores it in milliseconds against ten thousand others. The system scored the application: capability, settled, uninteresting, nobody asks who stands behind it. But can the system issue the denial? Can the no itself come from the model, the rejection a person receives and has to be told the reason for? That is not a question about the quality of the score. It is the agency question again, wearing a cheaper suit: someone has to be answerable for the refusal, able to stand behind the reason. Lower stakes than a misread scan, which is exactly why it is useful. Same fault line, smaller temperature. The sort works the same either way.

This is the part to be precise about, because the tool is easy to mistake for something it is not. The question does not tell you whether the system is intelligent. It does not tell you whether anyone is home, in the scan-reader or the loan-scorer or anywhere else. It leaves that question exactly as open as the last section left it. What it tells you is narrower and far more useful: whether that open question matters for the decision in front of you. The same system is pure infrastructure in one use and an agency problem in the next, and nothing about the system has to change for that to be true. The machine holds still. The decision is what moves.

So the question to carry out of the room is not whether the thing is intelligent. It is whether agency matters here. Most of the time the honest answer is no, and capability can run as infrastructure, unmarked and unremarkable. In the few places the answer is yes, the pattern changes: a person stays attached to the act, and the risk stays marked, not because the machine failed but because the act needs an owner. That is the whole tool. One question, asked per decision, sorting the few places where the hinge swings from the many where it sits still.

You can do this in your head, today, case by case. But it does not stay in your head, and it does not stay informal. The moment enough people sort the same decisions the same way, the sorting begins to harden, losing the feel of a judgment and taking on the force of a rule. That is the next thing to watch.

The Paint Dries

So the sorting hardens into a rule. The natural next question is where that rule lives, and the natural answer is wrong. We reach for law. We picture a statute, a legislature, some moment when the matter gets formally decided and written into the books, and we settle in to wait for it the way we waited earlier for an arrival that never comes. But look at what actually binds the radiologist, and the law is the last thing you find.

What binds her is a line on a report where a licensed name has to go. It is the malpractice carrier that will not write the policy unless that name is there, the hospital's sign-off procedure, the disclosure language the legal team pasted into the intake form, the vendor contract that quietly relabels the model a clinical decision support tool rather than a diagnostic device because the second word carries a liability the first does not. None of that is legislation. Some of it will be tested in court eventually, after the first patient is hurt and the first suit is filed. But by the time any of it reaches a courtroom, the working answer has already been in force for years, written into terms and procedures and prices nobody thought of as philosophy.

That is the correction worth making out loud. The settlement does not get legislated. It gets operationalized. Law is one instrument in a much larger kit, and rarely the first to move. Insurance pricing, procurement defaults, audit standards, professional licensing, disclosure requirements, malpractice norms, HR policy: these are the places the answer actually gets written, and most of them bind harder and move faster than any statute. An insurer who will not cover a use case has settled the question for everyone downstream long before a legislator has finished reading the briefing. The metaphysics gets decided in the fine print, by people who would be surprised to hear they were doing metaphysics at all.

We have done exactly this before, with something that sounded just as strange the first time it was said out loud. A corporation is a person. Not a human being, obviously, but a person in the eyes of the system, able to do a long list of things only people were supposed to do. We never resolved whether a corporation is really a person in any sense that would satisfy someone who wanted to know what a corporation truly is. We did something else. We built a control surface. A corporation can own property, sign contracts, sue, and be sued. It cannot vote, marry, suffer, go to prison, or die the way a person dies. Some of the dial is turned up and some turned down, and the arrangement works precisely because nobody insisted it be set all the way to one end. Partial personhood is not a contradiction anyone needs to fix. It is a working tool, and it has carried enormous weight for a very long time without anyone settling the question it stepped around.

AI is getting the same treatment, though not the identical settings. The patchwork is already visible in how a single system gets described across a single building. Here it is a tool, plainly, a thing someone uses. There it is closer to an agent, acting on instructions with a long enough leash that the word starts to fit. In one contract it is a product, with the liabilities a product carries; in the next, a service, with different ones. The same model wears different legal clothing depending on which room it is in and what responsibility has to be placed there. No central authority ruled on what the thing is. The category got assigned locally, case by case, by whoever needed a workable handle on it.

And notice what every one of those handles has in common. Each is a way of locating a responsible party, and the responsible party is never the system. It is the company that deployed it, the professional who relied on it, the vendor who built and sold it, the executive who signed the procurement order. The institutional machinery, faced with a question it cannot answer, routes around it, hunting for a human or a corporation to attach the blame, the insurance, the remedy, and the control to, and it almost always finds one located somewhere other than the machine itself. The thermostat never sat for its talk; the blame slid to the installer, the manufacturer, the owner. The same slide happens here, with higher stakes and better lawyers.

This is the move worth naming, because it is the quiet engine under the whole settlement. The question "is the AI responsible?" never gets answered. It gets replaced. In its place sits a question the system can actually handle: who is responsible for deploying, supervising, relying on, or profiting from it? The first question is metaphysical and has no public answer. The second is a governance question and has a very practical one. The metaphysical question gets converted into a governance question, and the conversion is so smooth that most people never notice a substitution took place.

Watch it work on the scan. The agency question raised earlier, who signs the diagnosis, does not get resolved by deciding whether the model understands what it is looking at. It gets resolved by licensing and malpractice norms and a convention so ordinary it already has a name in the building: draft but not sign. The model drafts the read. A licensed human signs it. The signature is where the accountability lives, and the license is the institution's way of guaranteeing there is a nameable person behind the act who can be hauled in front of a board, sued, struck off. The deep question of whether anyone is home in the model is left untouched, exactly as open as it was before. The shallow question of who answers for the read is closed, firmly, by a rule about whose name goes on the line.

The loan is the same shape at a lower temperature. When the system scores an application, nobody asks who stands behind the score. But the denial is a different animal, because a denial is an act done to a person who is owed a reason. So the rule arrives: adverse action requirements that say a real basis has to be given, fair lending obligations that say certain bases are forbidden and someone has to be answerable if they creep in. The regulation does not ask whether the model is intelligent. It asks who is accountable for the no, and it makes sure the answer is a party that can be named and pursued. Same conversion, lower stakes.

Now the seed from the opening of this piece comes due. The word "artificial," we said, never graduates from one state to another. It becomes situational. It fades wherever the source of the light is ordinary and uninteresting, and it surfaces again wherever the contrast carries weight. That is precisely what the institutional machinery does to the word "AI," and now we can see the mechanism doing it. Where capability is normalized and agency is irrelevant, the rules turn the system into infrastructure, and the word goes quiet. It becomes the software, the tool, the platform, the second read, part of the workflow. The "artificial" drops off the scan-reader for the same reason it dropped off the bulb: in that use, the origin of the work is the least interesting thing about it, and the paperwork agrees, filing it under clinical decision support and moving on.

But where agency is unresolved and someone needs to be accountable, the rules do the opposite. They reach for the word and make it loud again. They require the disclosure, mandate the human sign-off, attach the warning label, insist that this output was AI-generated and a person must review it before it counts. The qualifier comes roaring back exactly where the contrast does real work, just as it does for the photographer choosing between artificial and natural light. The word did not fade because the machine got better or harden because it got worse. It faded and hardened because institutions sorted the use cases into two piles: the ones that need a warning label and the ones that do not. And part of that warning label is the word AI.

Which is why the rules a leader is writing right now are not preparation for the answer. They are the answer, being authored in real time, in documents that feel purely administrative. The procurement default that decides whether a model can be bought as a product or only licensed as a service. The disclosure clause that decides where the warning label goes. The sign-off authority that decides whose name lands on the line. Each of these reads like paperwork and functions like a verdict. The settlement of what AI is, in any sense that touches the world, is being assembled out of exactly this kind of boring document, one clause at a time, by people who think they are merely doing their jobs.

And here is the thing none of that fine print will tell you about itself. A rule that sorts some use cases as agent and others as tool, applied long enough across enough rooms, does not just sit there describing a settlement someone else reached. It starts to do something to the people living inside it. That is the turn the next section takes.

Trained to See

The hardest part to see is what rules do to the people who live under them, especially the ones who never watched the rules get made. They arrived after. They inherited the settlement as weather, not as a decision, and weather is not something you argue with. A boundary that one generation negotiated, the next generation simply finds, already there, looking exactly like a fact about the world.

That is the whole mechanism, and it is worth stating plainly. Treat something as a particular kind of thing for long enough, in enough rooms, with enough riding on it, and the conviction that it really is that kind of thing arrives on its own. Not as a conclusion anyone argued for. As a feeling, a sense of the obvious, the kind of thing you would be embarrassed to defend because defending it seems beneath you. The conviction is a product of the treatment. It is not a discovery about the thing.

We have the proof already, and the last section set it down without finishing it. A corporation, we said, is a person in the eyes of the system, and we built that personhood as a control surface without ever resolving whether a corporation is really a person in any deeper sense. That was the legal mechanism. Here is what it grew into. Ask almost anyone today whether a corporation can want things, pursue its interests, act ruthlessly or generously, and the answer comes back without a pause: of course it can. People speak of what a company intends and whether it can be trusted the way they speak of a person with a temperament. That conviction feels like an observation. It is not. Nobody ever found the inner life of a corporation. The felt sense that one is in there is the residue of a century of owning property, signing contracts, being sued, and being answerable, a hundred years of treating-as-person settling into the bones until "is a kind of agent" stopped reading as a legal convenience and started reading as a plain fact. The fiction did not become true. It became invisible, which from the inside is the same thing.

Money does the quieter version. We treat a number in an account as worth something, and the agreement holds long enough that the worth comes to feel intrinsic, a property of the thing rather than a promise. Nobody experiences money as a shared fiction. They experience it as money.

So the institutional machinery does something stranger than it first let on. Routing around the metaphysical question was the visible half. But the swap does not leave the original question sitting there, patiently open, in the minds of the people downstream. It closes it, by making it stop feeling like a question at all. The person raised inside the settled rules does not experience a hard problem quietly sidestepped. They experience a matter that was, somehow, always plain. The institutional answer counterfeits a philosophical one. It produces the feeling of having perceived the truth, when what really happened is that everyone followed the same rule long enough to forget it was a rule.

Watch it land on the scan, one generation on. A clinician trained today learns medicine in a world where draft but not sign is already assumed, not a policy anyone announces but the obvious shape of the work. The model marks the shadow, the physician owns the call. To the trainee that division is not a liability arrangement reached by carriers and licensing boards; it is just how the thing is done, a matter of clinical common sense, nearly anatomical. They can tell you with real confidence where the tool's work ends and the doctor's judgment begins. The confidence feels like competence. But trace the exact line they are so sure of. It does not follow any fact about what the model understands. It follows the malpractice norm, precisely, every contour of it. What feels to them like a perception about where the thinking lives is the sediment of an insurance rule. They are reading a liability boundary and experiencing it as the boundary of a mind.

This is the shape of it, and it generalizes without strain. What we regulate as an agent starts, in time, to feel agentic. What we regulate as a tool starts to feel tool-like. The hinge, the open question of whether anyone is home, does not stay visibly open. It gets plastered over and painted to look like a wall, and the people who grew up in the room never knew there was a door, because the wall was already painted when they got there. The plastering is not a lie anyone tells. It is what a settled rule does to the intuitions of people who live under it long enough to stop noticing it is one.

And none of this touches the question itself. Whether anyone is ever home in these systems remains exactly as open as it has been all along. The openness of the question and the firmness of our conviction about it have quietly come apart, and the conviction is being trained on a schedule that has nothing to do with the truth of the matter. We may end up certain that the lights are on, or that the room is empty, and the certainty will feel like knowledge though it may only be a compliance habit worn smooth.

Which means the categories that will seem obvious in twenty years are not waiting to be discovered then. They are being set now, in the procurement default that files a model as a product or a service, in the sign-off clause that decides whose name lands on the line, in this year's unremarkable paperwork. Today's convenience is the next generation's plain fact. The successors who will treat these boundaries as simple perception are inheriting intuitions someone is authoring, quietly, in documents that read like housekeeping.

We can hold all of this at a comfortable distance, because the mechanism seems to run only on the things we built: the corporation, the model, the banknote, the categories we authored and whose seams we can, in principle, still see. That is the comfort. The next section asks whether the mechanism stays there.

The One Wall

The comfort was that the mechanism only touched the things we made. The corporation, the banknote, the model: fictions we authored, and therefore fictions whose seams we could in principle still find, if we ever cared to look. Run the mechanism on those and it explains a great deal without threatening us, because nobody was ever under the illusion that a banknote was alive. The conviction it manufactures is a conviction about something we already know to be a convention.

But the mechanism does not know where we drew that line, and it never agreed to stop there.

Look at the person across the table from you. You take it for granted that there is a someone there: that behind the face doing its ordinary work, the eyes tracking, the mouth finding words, there is an interior, a person home, a life being lived from the inside. You did not arrive at this by checking. You have never once verified it, and you cannot. You have only ever had the outside of another human being, the behavior, the words, the face that moves in ways you long ago learned to read. The inside, the part that would actually settle whether anyone is in there, has never once been available to you, not for a single person you have ever loved or feared or sat across from in a meeting.

What you have instead is a lifetime of treatment. From before you can remember you were taught to treat the people around you as people: to blame them and credit them, to apologize and expect apology, to hold them responsible and be held responsible in return. The law assumes it. Every institution you have ever moved through is built on it. And so the conviction set, the way it set for the corporation, only earlier, deeper, and with no memory of a time before it. The sense that other people are someones is not a perception you check against the world. It is the most thoroughly trained intuition you have.

So the hinge, the open question of whether anyone is home, may not be a wall getting quietly plastered over while our attention is elsewhere. It may be something stranger. It may be a place where there was never a wall to begin with. Just a very old, very good paint job we mistook for structure, laid down so early and touched up so faithfully that no one alive remembers it as anything else.

Sit with that, because it is the floor dropping out, and the rescue does not count for anything until it has fully fallen. If the conviction that other people are agents is trained all the way down, then there is nothing under it to find. The question we have spent this whole piece carefully refusing to answer was never a question with an answer hiding behind it. It does not stay open. It dissolves. Not resolved, which would be an achievement. Dissolved, which is worse, because a dissolved question cannot even be honestly asked.

Except.

There is one place the deflation cannot reach, and it is the one you are standing in right now. While you read this sentence, something is happening from the inside. You are not merely processing words on a page. You are experiencing them. This is the one thing on the table that you did not infer, were not trained into, and do not hold on anyone's authority. No institution handed it to you. No blame practice reinforced it. It was there before the training began and it would still be there if every rule in the world fell away tomorrow. Your own experience, right now, does not deflate, because there is nothing underneath it doing the propping up. It is not a conviction about a convention. It is the one wall.

But it is a very small wall. Notice how little it actually holds.

It does not tell you the lights are on in anyone else. It tells you, with a certainty nothing else in this piece can match, that they are on in exactly one place: the one place you can never show another person, and that no one else can show you. The single piece of evidence that would settle the question is the single piece of evidence that cannot be shared. You cannot hand your inside to the person across the table, and they cannot hand you theirs, and no quantity of behavior on either side ever closes the gap, because behavior was always the outside and the question was always about the inside.

This is why the question stays open instead of dissolving, and it is worth being exact about the reason. Not because we have not looked hard enough, and not because the science is young. A question whose evidence cannot be shared between two minds is not a question two minds can settle. It is not waiting on a better instrument. There is no instrument that reaches it.

So the hinge does not disappear under all that deflation. It narrows, and as it narrows it gets harder rather than softer. It is no longer "is agency real," because agency may be precisely the construction the last section described, intuition stacked on intuition stacked on a century of law, and that part can be deflated and probably should be. What survives is smaller and sharper: is there experience at all. And for a machine the question contracts to its finest point. Can this kind of system participate in experience, the one thing in the whole structure that does not reduce to training? Or does it only generate the public behaviors that make us attribute experience to it, just as a lifetime of treatment taught us to attribute experience to each other? We cannot check this from the outside, not in the machine and not in the person across the table. Nothing in this piece moved that, in either direction. The machine's status is as open on this last page as it was on the first.

Lay the whole structure flat and you can see why this one question outlasts every other. Capability can be measured. Substrate can be described. Agency can be socially assigned. Responsibility can be institutionally allocated. Each of those has a public face, something two people can point at and check against the world, which is exactly why institutions can settle them, culture can backfill them, and the word can fade or harden on schedule. Experience has no public face. It can be known only from the inside, by the one having it, and never by anyone else at all. Everything else in this piece was inference. This alone is not, and that is the whole trouble: the one thing we cannot infer our way to is the one thing the entire argument turns on.

That is the source of the staying power, and we will operationalize it anyway, because we have to. The reports get signed, the labels attached, the categories set and then felt as plain facts by people who never saw them set. None of that touches the thing underneath, and none of it ever will.

Which means the disagreement that remains, after every move in this piece has been granted, is not the kind an argument can end. Two people can accept all of it, the word that marks origin and not authenticity, the line that never gets crossed at a moment, the hinge, the tool, the operationalized settlement, the trained intuition, and still come apart at the final step. Set a system in front of them that acts for all the world as though someone is home. One will give the most weight to the behavior and lean toward yes. One will weigh biological kinship and lean toward no. One will trust only the first-person certainty they can never verify in anything but themselves, and decline to answer at all. None of them is making an error the others could correct, because there is no shared evidence left to consult. The disagreement is temperamental, not conceptual.

This is the floor under everything the earlier sections handed you. The capability versus agency tool still holds. The mechanism that hardens sorting into rules, and rules into intuition, is real, and already running in the rooms you work in. But none of it reaches bottom, and the honest thing is to say so without dressing it up: when the system in front of you finally behaves as though someone is home, no framework will tell you whether anyone is, and this one least of all. It will give you a clean way to decide what to do, and leave entirely untouched the question of what is true.

That is why it stays hard. The weighting is the one part no one can do for you. So choose deliberately.

Rob Anonen is the Founder and CEO of Infinite Solutions Consulting (ISC), an AI-native management consulting firm. He has spent 24 years advising enterprise organizations on the systems they build and the decisions behind them, helping leaders decide where AI runs as the tool and where someone must remain answerable.

TAKE IT WITH YOU

Prefer to read offline?

Get the PDF version - the full essay, formatted for reading anywhere.

We'll send the PDF and the occasional essay - unsubscribe anytime.

Have a transformation in motion?

Tell us what's stuck. We'll tell you what we'd do.

No 90-minute discovery sales pitch. A real conversation about your problem, your constraints, and whether ISC is the right fit.