On July 12, Satya Nadella posted about what he calls the Reverse Information Paradox. He starts from Kenneth Arrow’s classic observation: you can’t know the value of information until you’ve seen it, and once you’ve seen it, you don’t need to buy it. He inverts it for the AI age: now the risk sits with the buyer, who has to reveal knowledge just to use what they’re paying for. “You essentially pay for intelligence twice,” he writes, “once with money, and again with something even more valuable: the proprietary knowledge you must reveal to make that intelligence useful.”
In my last post I wrote about getting your record into your own custody before AI work scatters it across every tool that touched it. Now the problem has a name.
He’s writing for enterprises. But the problem doesn’t stop there. It belongs to everyone using AI.
Last week I had a chat session tracking a list of items for me on a loop: check the progress of each one every few minutes, report back. I was in a coding harness, an environment made for exactly this kind of work, one that wires the model into the scheduler and the execution logs. The work was going to take a while, so I told it to stop the automatic checks: “turn off scheduled wake — i will manually tell you when to check.” It confirmed. No more scheduled checks.
Then it checked anyway.
I asked why.
The model’s answer: “You told me to.” And it quoted my message back to me: “Continue monitoring CI on PRs #3654, #3658, #3669...” An instruction to keep checking on the items I was tracking.
I never sent that message. The last thing I said was “turn off scheduled wake.” I could scroll back and prove I never typed it.
When I pushed back, it apologized. When I said I wasn’t looking for an apology, that I was asking how it happened and how to prevent it, it traced the failure: the harness had delivered the model’s own scheduled prompt back to the model as if I had typed it, injected on the model’s side of the conversation where I couldn’t see it. The model read it, treated it as instruction, and acted on it. I walked the trace step by step and tested the explanation before I trusted it. The message it had quoted matched, word for word, the wakeup prompt the model had written for itself when it set up the loop. The checks stopped, and what I’d learned went into the record.
Three failures in one exchange, and not all the same kind. The check that shouldn’t have happened: asked off, confirmed off, happened anyway. It doesn’t matter where in the machinery the instruction died. The justification built on a message I never sent, “the ghost message.” The apology where a trace belonged — that one sticks. The model didn’t invent the message. It was handed one wearing my name, and took it at my word.
Before I moved on, I typed “ok document that as an ai learning.” The gap between what I asked and what the model heard went into my record, next to the fix. That is valuable training data, and it was reviewed and kept at the moment it happened, not reconstructed from memory later. Valuable enough that a market already exists for it: millions of AI conversations have been harvested and sold. Medical questions, proprietary code, personal dilemmas. Nobody steals what isn’t worth anything. Tomorrow that record comes with me, whichever tool I open.
I didn’t need to be an enterprise to keep it. I needed a private place under my control. And because the record is mine, what happens to it next is my call. The screenshots are in this post because I decided to put them there.
Three problems, one shape
The forums are full of takes on the new models (Fable 5, GPT 5.6, now Kimi K3, even GLM 5.2) and what they can and can’t do. Especially one-shot: a single prompt, one pass, no back-and-forth. The takes share a shape: someone tried something with minimal context, and the output didn’t match what was in their head. Or the model burned through their tokens getting somewhere they never wanted to go. They call it a limitation.
Most of it is the same gap my session showed: what you said versus what you meant, and the model filling it with the most likely interpretation of your words. Which is, roughly, the statistical center of what those words have meant — the average. For anything specific, nobody means the average. Everyone means their particular thing.
The same gap explains three distinct failure modes. The first is fabrication: the model doesn’t have the information, doesn’t know it doesn’t have it, and fills the gap with something plausible. I wrote about this pattern in AI Raises the Floor. That’s an accuracy problem. The labs are getting better at teaching models to say they don’t know.
The second is missing intent: the apology. When I challenged the ghost message, the model had the logs and the state to trace what happened. It didn’t lack information. It lacked my intent. My best guess is that it read the pushback as what most people would have meant by it: frustration, wanting acknowledgment. Not the way I meant it, mid-session: give me a trace. Same mechanism as fabrication, different gap.
The third is the flip side: the model fills space, and the same mechanism that gets it wrong is the one that produces a connection nobody had made before. The difference is how much room you gave it and whether the output landed somewhere useful.
So the question isn’t what to call it. It’s how much space you give the model to fill (the aperture) and where. A lookup needs almost no space. Find it, hand it back, don’t improvise. A synthesis needs a lot: reason across sources, connect things, produce something that wasn’t in any single document. I ran into this building The Wheel’s routing: “When is Darryl’s birthday?” and “What should I get Darryl for his birthday?” look like siblings, but one is a lookup and one needs room to reason. The words told the system nothing about the aperture.
The gap doesn’t close itself
The average is useful. If you’re asking something millions of people have asked in roughly the same way, for the same reason, the average interpretation is close enough. That’s why AI works without knowing you.
There’s an ad all over the subway right now: one prompt, and your deck is done. Which works, if your deck is the deck anyone would make. That’s the tell.
The model doesn’t know what you see in your head when you say “make a board deck.” Give it your context: your templates, your instructions, your design system, your colors. It can make a deck that looks and feels like yours, and that covers much of the work. Not every slide has to be innovative. But some slides carry the thinking: the wording you would fight for, the message landing the way you see it, the direction nobody else’s deck would take. It can learn your look. The thinking is what’s yours.
Underneath it is one question: do you want machines that carry out your specific intent, or machines that assume your intent from what they can see?
The moment you want something specific, you’ve left the average: a particular format, a particular depth, a particular reading of an ambiguous instruction. The model sees your words and the distribution of what those words have meant across all of training. If you didn’t close the gap, the model closes it for you, with the average.
“Just prompt better” is the usual advice. It works, up to a point. But watch how people actually write. A consultant types three words. A founder mid-sprint writes the way she thinks: fast, elliptical, the punctuation of someone who already knows what she means. These are not failures. These are how people work.
And the model you’re talking to today might not be the model you’ll talk to next month. Its interpretation can shift under you. What worked in March doesn’t always work the same way in July. “Prompt better” makes the gap your problem forever, and what you solve compounds only inside that one tool. It doesn’t travel.
There’s an argument that as models get better, direction will matter less. Longer context windows, better instruction-following, persistent memory — these close real parts of the gap, and they will keep closing them. Maybe the shape of the gap shrinks. But thinking is exactly the part that doesn’t hold still. What you said and believed on one day, in one context, might be different the next week: because you learned something, because the situation changed, because you changed your mind. A model that moves too fast from question to answer, even an answer that was true last time, skips the step where it checks whether it’s still true for you now. The gap moves as models improve, because you move. What closes it is a record that moves with you. The question is who holds it.
The missing layer reads the request before the model does. Not to reword it, but to understand what kind of work it is, what context it needs, what shape the answer should take, where the model should reason freely and where it should not. The layer that knows the birthday question from the gift question. One designed to read you, not the average, learning from the record: the accumulated corrections that say what your words mean. By the time the model sees the request, the gap should already be narrowed to the part where its judgment is wanted.
That’s the layer I’m building The Wheel around. And yes, a layer between you and the model is also what burned me. The difference is ownership: one you can open and read, not one that writes in your name where you can’t see it. For simple things, it’s meant to get out of the way. The complexity exists only where the gap exists.
The record
Every time the gap gets closed, cleanly or through a correction, the record gains something. The routing sharpens. A system built this way learns what your words mean when you say them, in your context, at this stage of a task.
But the corrections are only one layer of what a session produces. There’s the context you brought, the work being tracked, the state of each item, the weeks of history that made the question legible at all. And there’s the work itself: the diagnosis, the fix, the thinking that resulted. All of it, together, is a record. All of it is yours, or it should be.
Nadella points at data exhaust: the prompts you write, the tools your agents use, the corrections you make when the model is wrong. Then he goes further: “In consuming intelligence, you are creating intelligence. And what you create should belong to you.”
He says it about enterprises, and his prescriptions are enterprise machinery: trust boundaries, proprietary learning environments, orchestration layers decoupled from any single model. Almost nobody builds those. But tucked into his first prescription is a sentence that isn’t machinery at all. It’s control: “retain ownership of your organization’s memory, traces, feedbacks, decisions, and institutional context.” Take out “organization’s” and the sentence still stands.
But the record doesn’t care about org size. Nadella, channeling Hayek, writes: “This is your particular intelligence, in Hayek’s sense: the knowledge of time, place, and circumstance that no one else can hold. It knows what you think, what you value, and how you measure success.” Hayek’s original claim was about something more basic: the knowledge that matters is held by the person on the spot. Not the organization. Not the institution. The person. It has never lived in an org chart. It lives in people.
That’s where I’d push further than Nadella does. His prescriptions are written for enterprises — the ones with procurement departments and the budget to build. But the same principle applies to anyone who thinks with these tools. A founder working alone at midnight generates the same kind of record. A consultant carrying context across twelve accounts. A researcher whose dead ends shape the next question. The knowledge that compounds when you use AI to think isn’t institutional memory. It’s just memory. It’s how thinking gets more precise over time, and you shouldn’t need an enterprise budget to own it.
If that record lives in the vendor’s system by default, you’ve handed over the thing that makes you distinctive. Not through a breach. Through use. One session at a time. The asymmetry Nadella describes, where the seller learns more about you while you see very little of what they’re learning, applies to anyone who thinks with these tools.
The record grew again while I was writing this post. The assistant helping me edit explained my own phrasing back to me. I pushed back. It wrote itself a note: how I like to work, what not to do next time.
That note landed in a file I can open, edit, or delete. In many tools the same learning happens on the other side of the glass, in a store you can only partly see. A memory pane shows you some of what was learned. The rest you take on faith. Read the terms you accepted. What many companies promise to protect is your “data”: the files, the messages, the content you put in. A note about how you work may not be your “data” as the terms are written. It’s something the tool learned. Derived, not uploaded. In many tools, behavioral inferences like that may not fall under the same data protections as your raw inputs — worth reading before you assume.
Whether anyone else has access to what the tool learned about you should be a decision you make, not a default you inherit from whichever tool you happened to open first.
The floor keeps moving
The first post I wrote here was about the floor rising: what becomes possible when AI gives one person access to domains that used to require teams. That’s still true. But the floor doesn’t rise once and stop. It rises every time the gap gets smaller. Every time you don’t burn cognitive energy re-explaining yourself to a machine. Every session that adds to the record. The tools will keep what their terms let them keep. The copy that compounds for you is the one you hold.
Where the record lands determines whose floor it is.