A New Language Study Does Not Show That Humans Are Just Next-Token Predictors

The viral X post delivers its argument in three beats: “It’s just next token prediction / it can never / Oops.” It is a clever analogy: a neat reversal of a familiar objection to large language models. It is not the paper’s conclusion. The post’s framing turns a study of short language patterns into claims that humans are next-token predictors, LLMs work like human brains, and Chomsky’s account of language has been proved wrong.

That is three conclusions piled on top of an experiment that tested none of them.

The actual result is narrower and better. Yngwie Nielsen and Morten Christiansen found evidence that English speakers are sensitive to short, ordered patterns of grammatical categories even when those word sequences do not form conventional syntactic constituents. In other words, language processing appears able to reuse a linear pattern such as verb–preposition–determiner across different phrases. That matters because theories of language have often treated hierarchical constituents as the primary units doing the work.

But “people process linear patterns” does not mean “people process only linear patterns.” It does not turn a reaction-time effect into a theory of the entire mind. And it does not make a human doing a phrasal-decision task equivalent to a model trained to predict tokens.

The Experiment, Without the Headline Inflation

The researchers began with the Corpus of Contemporary American English, a collection of more than one billion words from 1990 through 2019. They extracted more than 300,000 three-word sequences and tagged each word using Universal Dependencies part-of-speech categories.

Participants then saw short strings and had to decide quickly whether each could occur as part of an English sentence. A string such as work for a is possible; bring am top is not. On critical trials, one possible phrase was immediately followed by another. Sometimes prime and target shared an abstract category sequence without sharing words: added to a and defined by the, for example, both instantiate verb–preposition–determiner. Other times the patterns differed.

Responses were faster when the ordered part-of-speech pattern repeated. Study 2 added a useful control: the same categories in reverse order did not yield the same benefit. Studies 1–3 also matched the critical prime and target materials on semantic similarity, length, and unigram, bigram, and trigram frequency. That makes a simple “these words were more similar or more common” explanation less plausible.

The important word here is abstract. The tested structure was not the identity of the next word and not even a probability assigned to that word. It was an ordered sequence of grammatical categories shared by different words. Calling this “next-token prediction” erases the level at which the experiment was designed.

The paper also worked hard to establish that its supposedly non-constituent items really behaved that way in corpus sentences. The authors used the Berkeley Neural Parser to estimate how often a string formed a constituent in context. In Studies 3 and 4, the designated non-constituents had very low parser-based constituency scores, while the constituent comparison items scored much higher.

Study 4 made the contrast especially concrete. Its three-word “parts” crossed or stopped inside a constituent boundary, while four-word “wholes” containing those same kinds of sequences formed constituents. Priming still appeared across parts and wholes. That is evidence against an account in which only complete conventional constituents can supply the relevant processing pattern.

Why Priming Is Evidence, Not a Brain Scan of a Data Structure

Structural priming is a well-established method: exposure to one linguistic form changes how readily a person processes or produces a related form shortly afterward. If added to a makes defined by the easier, even though the words and meanings differ, the shared grammatical sequence is a reasonable explanation.

What priming cannot do by itself is reveal a literal object stored in the brain. Faster performance might reflect persistent activation, implicit learning, expectation, retrieval of a familiar sequence, or some combination of those processes. The broader priming literature treats the phenomenon as evidence about representation and processing, but not as a transparent readout of one uniquely determined mental format.

That distinction matters here. The experiments show sensitivity to reusable linear structure. They support the authors’ representational interpretation, especially alongside the controls and converging analyses. They do not demonstrate that the brain contains a table of part-of-speech trigrams, still less that this table is the whole language system.

The supplementary reanalysis strengthens the statistical-learning story without making it exclusive: more frequent part-of-speech sequences predicted faster decisions in most relevant conditions, even after controlling for the lexical frequency of the strings. Familiarity with a category pattern appears to matter. It can matter alongside other information.

The Corpus Results Travel Farther, but Prove Less Causally

The two additional analyses asked whether a related pattern appears outside the laboratory task. In the CELER eye-tracking corpus, 68 native US English readers tended to process a target sentence faster after a sentence with matching structure. In Switchboard telephone conversations, 358 speakers were more likely to use a part-of-speech sequence after it had appeared in the conversation.

Those results improve ecological credibility: the effect is not confined to pressing a button for isolated three-word strings. But both are observational analyses of existing corpora. They can show that structural repetition and easier reading or later reuse travel together; they do not randomly assign every feature of natural reading and conversation. “External validity” is not the same thing as a fresh causal experiment.

What the Paper Changes

The cleanest update is not from “hierarchy” to “sequence.” It is from a one-level story to a multi-level one.

Speakers can be sensitive to a short linear category pattern because it has occurred often and is useful for anticipating or assembling language. At the same time, they can represent which words group together, which phrase modifies which other phrase, and who did what to whom. A sequence can be reused during rapid processing even when it cuts across a constituent boundary; a hierarchy can still represent relations that linear order alone does not capture.

That coexistence is not a consolation prize. If a theory says only constituents are represented or that linear regularities have no independent role in processing, these results are a serious problem for it. The paper expands what a psychologically plausible language system must explain: frequency-sensitive chunks below or across constituent boundaries, plus the evidence that already motivates hierarchical organization.

The authors themselves frame the result as a challenge to accounts across generative and constructionist traditions. That breadth is worth noticing. The target is not simply one famous linguist. Any approach that grants privileged reality only to its preferred units has to explain why these non-constituent patterns prime behavior.

Did This Prove Chomsky Wrong?

No single claim called “Chomsky’s theory” is being tested here. The relevant distinction in generative linguistics is between competence—an idealized speaker’s knowledge of language—and performance—language as actually produced and understood under limits of memory, attention, and context.

The experiments are plainly evidence about performance. Under a strong separation, a reaction-time effect need not directly settle what belongs in the grammar. Under weaker versions, performance reflects grammatical knowledge imperfectly and can inform theories of that knowledge. The paper’s supplementary discussion addresses exactly this issue rather than pretending it disappears.

The fair conclusion is that the results challenge strong claims that hierarchical constituents are the exclusive, or always primary, units driving language processing. They do not show that hierarchical syntax is false. They do not show that every hierarchical grammar makes the same prediction. And they do not explain away dependencies and interpretations for which hierarchical relationships may still be indispensable.

So the viral declaration contains a real theoretical pressure point wrapped in an absolute claim the data do not support. “Non-hierarchical structures are represented too” is a substantive finding. “Language is not hierarchical” is a different proposition.

The LLM Analogy Stops Before the Evidence Starts

It is tempting to connect any evidence for sequential statistical learning to large language models. Humans and LLMs both exploit distributional regularities. Both can become sensitive to patterns that extend beyond particular words. That family resemblance is useful for generating questions.

It is not a mechanistic identification.

This paper did not test an LLM, compare model and human errors, manipulate a model objective, inspect neural architecture, or establish a shared learning algorithm. It did not test semantics, grounding, or general intelligence. Its participants were asked whether short word strings could occur in English; they were not trained by minimizing next-token prediction loss over a giant text corpus.

There is also a basic conceptual mistake in treating sequence and hierarchy as opposites. A system can emit one token at a time while internally representing nested structure. Statistical prediction is compatible with hierarchical internal representations; sequential output does not tell us what representation produced it. That is true as a matter of computational design, before making any claim about brains or current language models.

The paper therefore gives no license to say humans and LLMs use the same mechanism. It supports a more restrained comparison: human language processing, like many successful computational systems, draws on learned sequential regularities. How those regularities are learned, represented, connected to meaning, and combined with hierarchy remains an empirical question.

Read the Paper, Not the Screenshot

The viral X interpretation is useful as a record of what people wanted the result to mean, not as an authoritative account of the science. Its sweeping language compresses several open debates into a victory lap: representation becomes prediction, one processing effect becomes a complete theory, and a result about English phrases becomes a conclusion about human cognition and AI.

The study deserves better than that. Four preregistered experiments, an order-sensitive control, corpus-derived stimulus matching, parser-based constituency checks, and two natural-language analyses make a credible case that linear grammatical chunks are psychologically relevant. The imperfect matching in Study 4 and the observational nature of the corpus work constrain the inference; they do not erase the central result.

Good science often changes the menu rather than removing everything except one item. This paper gives linear, non-constituent patterns a stronger place on that menu. It does not close the kitchen.