Leiter argues that the rhetorical form 'X cannot really be doing Y because X is made of Z' carries zero explanatory power regardless of whether Z is meat or floating-point numbers. He's careful to note this doesn't prove LLMs understand anything — it just retires one specific lazy dismissal where naming the substrate is treated as a refutation of observed behavior.
Concedes the literary point but argues the Bisson reframe breaks down because we actually know how LLMs work at the mechanism level — weights, attention, token prediction — whereas we have no comparable theory of how meat produces thought. The asymmetry, they argue, means the 'just weights' dismissal has more grounding than the aliens' 'just meat' dismissal.
Pushes back on the asymmetry claim by pointing out that neuroscience has an excellent mechanistic theory of neurons — action potentials, neurotransmitters, synaptic weights — yet it tells us almost nothing about why a brain can write a sonnet. Knowing the substrate's mechanics is categorically different from understanding why the system as a whole produces the behaviors it does.
Argues that engineers who reduce LLMs to 'just matrix multiplication' will systematically mispredict where the systems actually fail. The substrate-level frame doesn't help anticipate emergent failure patterns like hallucination, prompt injection, or capability cliffs — making it not just philosophically lazy but operationally harmful for people shipping LLM-based systems.
On Monday, a short blog post by Max Leiter titled 'They're made out of weights' climbed to 1,024 points on Hacker News with 600+ comments. The piece is barely 800 words. Its argument is even shorter: every time someone dismisses an LLM with 'it's just matrix multiplication' or 'it's just predicting the next token,' they are recapitulating, almost word-for-word, the conceit of Terry Bisson's 1991 short story *They're Made Out of Meat* — in which aliens cannot accept that thinking, singing, dreaming creatures could be made of *meat*.
Leiter's structural move is to swap the substrate. The aliens are now the AI skeptics. The meat is now the weights. The form of the argument 'X cannot really be doing Y because X is made of Z' is exactly the same whether Z is meat or floating-point numbers, and it carries exactly as much explanatory power in both cases: none.
The post is not a defense of AGI hype. Leiter explicitly says he doesn't know whether LLMs 'understand' anything, and he's careful to note that the Bisson reframe doesn't prove they do. What it does, he argues, is retire one specific rhetorical move — the one where naming the substrate is treated as a refutation of the behavior.
The HN comment thread is the actual story. The top reply (412 points) concedes the literary point but pushes back that 'we have a mechanistic theory of LLMs and we do not have one of human cognition, so the analogy is asymmetric.' The counter-reply (287 points) notes that we have an excellent mechanistic theory of *neurons* — action potentials, neurotransmitters, synaptic weights — and it tells us almost nothing about why a brain can write a sonnet. Knowing the substrate's mechanics is not the same as knowing why the system as a whole does what it does.
This is the interesting part for working engineers. The 'it's just weights' frame isn't just philosophically lazy — it's an actively bad predictor of failure modes. If LLMs were truly 'just autocomplete,' you would expect their failures to look like Markov-chain failures: locally fluent, globally incoherent, easily caught by length. Instead they fail in stranger ways — confidently inventing API signatures that compile, fabricating citations with plausible author names, refusing tasks they completed yesterday. These are not autocomplete failure modes. They are something else, and 'something else' is what your evals have to catch.
A second thread worth pulling: Leiter quotes Geoff Hinton's recent line that 'understanding' in a neural network is whatever the weights have to encode in order to compress the training distribution. By that operational definition — which is the only one that lets you write a test — LLMs understand many things. Not all things, not deeply, but the question becomes empirical rather than metaphysical. The practitioners winning the LLM-application game right now are the ones who stopped having the philosophy argument three years ago and started measuring.
There is a reasonable objection that the Bisson reframe proves too much. By its logic, a thermostat 'understands' temperature. Leiter addresses this in a one-line footnote: yes, and we should be comfortable with a gradient. The thermostat understands temperature in a thin sense; the LLM understands Python in a thicker sense; humans understand grief in a thicker sense still. Treating 'understanding' as a binary that flips on at some magical threshold is what gets us into trouble in both directions — over-claiming for GPT and under-claiming for the cat.
The practical takeaway is not philosophical. It's about how you reason about models you deploy.
First: stop using substrate-level descriptions as a substitute for behavioral evals. 'It's just a transformer' tells you nothing about whether the model will hallucinate your customer's order ID. The only thing that tells you that is running it on your customer's order IDs and counting. If your team's quality bar is 'well, it's just predicting tokens, so we'll have a human review the output,' your team does not have a quality bar.
Second: when a model surprises you — good or bad — resist the urge to explain it by gesturing at the architecture. 'It's emergent' and 'it's just retrieval' are the same non-explanation in opposite costumes. The useful question is always: what specific input distribution produces this specific output distribution, and what does that imply for the next deployment? Mechanistic interpretability work (Anthropic's circuit tracing, the recent feature-attribution papers out of DeepMind) is starting to give real answers here, but those answers look like *circuits*, not like *weights*. The level of description matters.
Third: the Bisson reframe is also a hiring filter. Candidates who reach for 'it's just matrix multiplication' in an LLM design interview are doing the same thing as candidates who reach for 'it's just an if-statement' in a control-flow question. Both are technically true and both indicate someone who has stopped thinking one level too early.
The post will be forgotten in a week — most HN front-page essays are — but the rhetorical move it kills should stay killed. The next two years of LLM engineering will reward people who treat models as empirical objects to be measured and punish people who treat them as philosophical objects to be dismissed. Leiter's contribution is just to point out that the dismissal has been hiding inside a 35-year-old sci-fi joke the whole time. Meat thinks. Weights compute. Neither fact is the end of the conversation; both are the beginning.
The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness.This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another
It's not often I see something that's fractally wrong but here we are.There is a dictionary, it's called the tokenizer.There are grammar rules, they are just very weak because the structure of human language is generally quite weak. When presented with languages which have strong cons
In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6."What are you doing?", asked Minsky."I am training a randomly wired neural net to play Tic-tac-toe", Sussman replied."Why is the net wired randomly?", asked Minsky."I do n
This read like poetry to me. Thank you for sharing it.I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness.For a little bit I was working on having linguisti
Top 10 dev stories every morning at 8am UTC. AI-curated. Retro terminal HTML email.
The weights start with a random manifold. The training takes data and shapes the manifold, weight by weight, in many cycles. Once the training is the done manifold is fixed.When a new inference has to be done the query(q) is projected in the manifold space. This projection is dropped on the manifold