One lab calls it the largest attack in its history; another's founder calls it standard practice, under oath. Here is what distillation actually is — and why the models everyone is now fighting over were built from output that was ours to begin with, and may not be ours to use much longer.
In April 2026, Elon Musk took the stand in his own lawsuit against OpenAI and was asked whether xAI had used distillation on OpenAI's models to train Grok. He first called it a general practice across the industry. Pressed on whether that was a yes, he said: "Partly." xAI's position is that distillation is a standard research practice used across the field — for xAI to do. Its own terms of service ban everyone else from doing it back.
Two months later, Anthropic told the US Senate Banking Committee that operators tied to Alibaba's Qwen lab had run the largest known distillation attack in its history — 28.8 million exchanges through roughly 25,000 fraudulent accounts.
Same technique. Two names: standard research practice and industrial-scale attack. The gap between those names — and who gets to pick which one applies — is the whole story.
What distillation actually is
Distillation is how you make a smaller, cheaper model behave like a bigger, more expensive one. You take the strong model — the "teacher" — feed it a mountain of questions, and record its answers. Then you train your own smaller model — the "student" — on those question-and-answer pairs until it learns to respond the same way. The student never sees inside the teacher; it just studies millions of worked examples until it can imitate the results. The idea goes back to a 2015 paper by Hinton, Vinyals and Dean, and the name is the giveaway: you are distilling the teacher's behaviour down into something lighter that runs at a fraction of the cost.
There are two kinds. White-box distillation needs access to the teacher's internals — its weights and output probabilities. Black-box distillation has none of that. You see only what comes out of the API, so you collect the teacher's responses and fine-tune the student on them. Black-box is the kind that worries the frontier labs, because it needs nothing but a working key and patience.
It is also the kind the entire open-model world is built on. Stanford's Alpaca, Vicuna, and Microsoft's Orca were all built by distilling from GPT-class models. This is not a fringe exploit; it is one of the standard ways a small lab catches up to a large one.
And the techniques are openly published — including by the company now accused of misusing them. A 2024 paper, "Knowledge Distillation of Black-Box Large Language Models," co-authored by researchers at Alibaba Group, sets out a method for transferring capability from black-box proprietary teachers into smaller models, and names GPT-4 and Claude as exactly the kind of teacher it has in mind. So when Anthropic implies Alibaba knows how to do this: yes. It is in the peer-reviewed record, with their name on it, two years before the complaint.
Why it gets called an attack
Anthropic's June 10 letter followed a February disclosure naming three other Chinese labs — DeepSeek, Moonshot AI and MiniMax — and it went not to a court but to Congress, framed as a national-security matter.
It is worth being precise about what, in their telling, makes it an attack — because it is not the distillation. It is the method around the distillation: roughly 25,000 fraudulent accounts and commercial proxies used to evade the geographic rules that bar access from China. That is fraud and circumvention, and it is a real, nameable wrong. Strip it away, though, and what is left is a model learning from another model's outputs — the thing everyone does.
There is also a genuine structural problem, and the honest version of this argument should grant it: at the API level, a distillation query is indistinguishable from a legitimate one. There is no clean technical test that separates a developer using Claude from an operator harvesting it. That is why terms of service can't solve the problem, and why the labs escalated to government in the first place. But notice what that argument is about. It is about enforcement — whether you can stop the practice — not about whether the practice is wrong to begin with.
But whose output is it?
The labs guarding their outputs today built those outputs by distilling ours.
Not "ours" in any abstract sense. The capability inside these models was rendered out of the written commons — every book, every page, every archive of human expression — much of it taken without permission and, on the internal record, known to be taken without permission.
Meta is the clearest case. Court filings in Kadrey v. Meta describe the company torrenting roughly 81.7 terabytes of books from shadow libraries including LibGen and Z-Library, with internal chats showing staff calling the source pirated, the decision escalated to "MZ," and a plan to not disclose the use and to strip files of words like "stolen" and "pirated" to reduce exposure.
Anthropic's own foundation has the same shape. In Bartz v. Anthropic, the company was found to have downloaded more than seven million pirated books; Judge Alsup drew the line the field now lives by — training on books is transformative fair use, but downloading pirated copies is not — and Anthropic settled for $1.5 billion, the largest copyright recovery in US history. A separate suit by music publishers produced an out-of-court guardrails stipulation, after which Anthropic beat the publishers' injunction — and then, in January 2026, faced a fresh $3 billion suit over more than 20,000 works.
This is the part that the word "attack" is designed to make you forget. The output the labs now treat as a proprietary crown jewel is a distillation of human output they did not pay for at the point of taking. It was theirs to sell only because it was ours first.
The disagreement is the answer
The people who build these models do not agree that distillation is wrong.
xAI's founder, under oath, calls it general practice. OpenAI complains about DeepSeek but has not made a crusade of it. Anthropic takes it to the Senate as a threat to American leadership. Three postures, one act.
That spread is the finding. A settled ethical violation does not look like this — theft is not a thing one company campaigns against, a second shrugs off on the witness stand, and a third quietly practises while objecting when it is done to it. The disagreement is not noise around a clear wrong. It is evidence that there is no clear wrong underneath — only an act whose name changes with the position of the person describing it. Distillation is not a crime the industry discovered. It is a competitive threat the industry renamed, once it had something worth protecting and had forgotten where it got it.
There is an even plainer tell, written into the contracts. Every one of these labs — OpenAI, Anthropic, Mistral, xAI — bans distillation in its own terms of service, while the whole field distils the open web, the teacher models, and each other. A rule everyone writes down and nobody keeps is not a moral line; it is a weapon, kept for pointing at competitors. The principle was never "distillation is wrong." It was "distillation is wrong when you do it to me."
The enclosure becomes a lockbox
Here is why this is not an academic squabble about who copied whom.
The same national-security frame that turns distillation into an "attack" is now being turned on the models themselves — not to stop theft, but to control access. On 12 June, the US Commerce Department ordered Anthropic to disable Fable 5 and Mythos 5 for all foreign nationals, and Anthropic complied by switching them off worldwide. Two weeks later the government let it redeploy the more powerful model, Mythos 5, to a vetted set of US critical-infrastructure organisations — while Fable 5, the version everyone else had actually been using, stayed dark. The same week, OpenAI previewed its next-generation GPT-5.6 models — Sol, Terra and Luna — but only to a small group of trusted partners, at the request of the US government. Even OpenAI, complying, said plainly that this kind of gating should not become the default, because it keeps the best tools from the people who need them.
So the arc closes. The labs distilled the commons into private models. They called it theft when anyone distilled those models back out. And now a government can decide which of us is allowed to use the result — revoking access, narrowing it, and restoring it by letter. The point is not which model is switched on this week; it is that the switch exists, and a government's hand is on it. The enclosure of public output into a proprietary model was always a transfer of control. The export orders only made the control visible: the thing built from everyone's output is now a thing a single directive can withhold from everyone.
The output was ours first. We did not get a stake in the model, and now we are not guaranteed the use of it either.
The default was yes
There is a smaller version of this asymmetry, and I am sitting inside it as I write. The words in this essay are, right now, input to a model. Whether they stay input or become training data was not, until recently, a choice I got to make cleanly. Anthropic spent years promising that consumer chats were never used for training and were deleted within thirty days — then changed that, so conversations would be used to train its models unless the user opted out, with retention stretched to five years. Existing users got a pop-up: a prominent button to accept, and a much smaller toggle, defaulted to on, with a deadline to decide. The burden to refuse sat with me. Had I not found the box and turned it off, the sentences I am using to make this argument would have become training material for the next model.
That is the whole thesis in miniature. When the labs take human output, the default is yes — unless you fight it, on a screen built so most people won't. When someone takes the labs' output, the default is no, enforced by terms, letters to senators and export orders. Extraction from us is opt-out by design. Extraction from them is opt-out for no one; it is simply forbidden.
There is one practical consequence worth stating without drama. The models you can actually keep are the ones nobody can switch off for you — the open-weight ones you can run and train yourself, a growing share of them out of China. That is not a tidy ending; those carry their own dependencies and their own politics. But when the alternative is a frontier model that a letter to a senator can render an "attack" and an order to a company can render unavailable, "the one I can keep" stops being an ideological preference. It starts being the only kind of access that is actually yours.
A note for the recursively inclined.
There is a deeper version of the point above, and it runs on some heavy mathematics, so it waits down here for anyone who wants it.
When you type into one of these models, your words do not completely dissolve into a haze of numbers. They leave a clean, recoverable trace inside the system — one a simple procedure can read straight back out, word for word. Researchers recently proved that these models are, in a precise mathematical sense, lossless about their input: two different prompts essentially always leave two different internal traces, and the exact text can be reconstructed from the trace.
This matters because of how the law has reasoned about training. The fair-use argument that lets a model train on copyrighted books rests on transformation — the claim that the original is abstracted and repurposed so thoroughly that it survives only as patterns, not as the recoverable original. That is the reasoning a US court accepted in the same case behind the $1.5 billion settlement above, when it ruled the training itself was fair use. The transformation is real. This work suggests it may not be as total as the defence needs it to be.
Strictly, the researchers showed this for the prompt as it is processed, not for the training data already sitting inside the model — so it does not overturn the ruling. But it cracks the easy assumption that what enters a model is gone for good. There is, in their words, no free privacy once data enters a Transformer.