This is the final output of a running conversation with Anthropic’s Claude. Posted here to form the basis for research into LLM mechanisms and possibly a better way to do communication than symbol based language with all of it’s short-comings.<\/p>\n\n\n\n
A ground-up derivation of why “the word isn’t the meaning,” where the meaning actually lives inside a large language model, and how you can go look at it yourself.<\/em><\/p>\n\n\n\n Here’s a claim you can verify from your own life before we touch a single equation: the words are not the meaning.<\/strong> They’re a crude, lossy container the meaning gets poured into and out of.<\/p>\n\n\n\n Take one word. Dog.<\/em> In the 1400s, calling a person a dog was a genuine insult \u2014 contemptuous, the kind of thing that could start a fight. Today, “you’re my dog” is warmth, loyalty, affection. Same three letters. Same syntax. The meaning rotated a full 180 degrees while the symbol sat perfectly still.<\/p>\n\n\n\n Now hold that still and add a second axis. The word doesn’t only drift over time<\/em> \u2014 it drifts across space<\/em>. Regional dialects attach different meanings to identical spellings. “Bad” can mean bad, or \u2014 depending on who’s talking and when \u2014 excellent. “Sick” is either an illness or a compliment. The symbol is fixed; the meaning is a function of who, where, and when.<\/em><\/p>\n\n\n\n If you sit with that long enough, you reach an uncomfortable conclusion: the symbol is a weak carrier of meaning.<\/strong> The information is not in<\/em> the token. The token is a pointer, and what it points to depends on an enormous surrounding context the token itself doesn’t contain. Language couples symbol to meaning loosely.<\/em><\/p>\n\n\n\n I reached this conclusion myself, in late-night philosophy sessions, more than twenty years before large language models existed. What follows is what happens when you take that old intuition seriously and then look at how a modern language model works \u2014 because it turns out the machine is a working model of exactly this idea.<\/p>\n\n\n\n Suppose you want to build a system that works with language. If you represent each word as just a symbol \u2014 a lookup entry, an index in a dictionary \u2014 you’ve already lost. Your symbols have no relationships. “Dog” and “wolf” are as unrelated as “dog” and “democracy,” because as bare symbols they’re just different entries. Nothing in the representation knows that dogs are closer to wolves than to constitutional theory.<\/p>\n\n\n\n So symbols-as-labels can’t be the substrate. If it were, nothing built on it could handle the drift, the context-dependence, the analogies, the sheer slipperiness<\/em> we just established. And yet these systems demonstrably do handle it. So \u2014 following the logic \u2014 there must be a richer representation underneath the symbols.<\/strong> The operational capability forces the conclusion: something better than “word = label” is doing the work.<\/p>\n\n\n\n The question is what.<\/em><\/p>\n\n\n\n Here is the move that makes modern language models work, stated as plainly as I can.<\/p>\n\n\n\n Replace each word with a point in a high-dimensional space.<\/strong> Not a label \u2014 a location.<\/em> A list of hundreds or thousands of numbers (a vector<\/em>) that positions the word somewhere in a vast geometric space. This is called an embedding.<\/strong><\/p>\n\n\n\n Once meaning is a location<\/em>, three magical things become possible, and none of them were available to symbols-as-labels:<\/p>\n\n\n\n 1. Distance becomes similarity.<\/strong> Words with related meanings sit near each other. “Dog,” “puppy,” “canine,” “hound” cluster together. “Democracy” is far away. Similarity is no longer something you have to hand-code \u2014 it’s just how close two points are.<\/em> The geometry stores the relationships for free.<\/p>\n\n\n\n 2. Direction becomes relationship.<\/strong> This is the famous one, and it’s worth internalizing because it’s the whole idea in miniature:<\/p>\n\n\n Read that as geometry. Start at the point for “king.” Subtract the “man” vector and add the “woman” vector \u2014 i.e., move in the direction that generally separates male words from female words<\/em> \u2014 and you land almost exactly on “queen.” The concept of royalty<\/em> and the concept of gender<\/em> are directions in the space.<\/strong> Meaning isn’t stored in the points alone; it’s stored in the arrangement<\/em> \u2014 the directions and distances between them. You can do arithmetic on meaning.<\/p>\n\n\n\n 3. Context can move the point.<\/strong> This is how the machine solves the “dog” problem. In a modern transformer, the embedding for a word isn’t fixed \u2014 it gets nudged<\/em> by the surrounding words through a mechanism called attention.<\/strong> The same token enters the same way, but by the time it’s been through a few layers, “dog” in “my loyal dog” and “dog” in “you filthy dog” have been pushed to different regions<\/em> of the space. The drift you observed in the history of English is reproduced, live, inside the model, on every forward pass. Context routes the symbol to the right meaning by moving it geometrically.<\/em><\/p>\n\n\n\n So the substrate underneath the symbols is a high-dimensional geometric space<\/strong> \u2014 sometimes described as a manifold<\/em>, because the meaningful points don’t fill the space randomly but lie on curved, structured surfaces within it. Tokens are just the encoding at the door. The thinking happens in the geometry.<\/p>\n\n\n\n Now we can state the whole pipeline cleanly, which is what you actually asked for.<\/p>\n\n\n\n Encoding (symbol \u2192 geometry):<\/strong><\/p>\n\n\n\n Processing (geometry \u2192 geometry):<\/strong><\/p>\n\n\n\n Decoding (geometry \u2192 symbol):<\/strong><\/p>\n\n\n\n That’s the loop: symbol in, geometry does the work, symbol out. The language you see is the packaging. The manifold is the product.<\/p>\n\n\n\n Here’s where it’s tempting to overreach, and I want to plant a flag so you don’t.<\/p>\n\n\n\n It feels<\/em>, having gotten this far, like the geometry is something deeper and truer<\/strong> than human language \u2014 that the model escaped the crude tool and found a purer, more absolute layer of meaning underneath. That the manifold is bedrock.<\/p>\n\n\n\n It isn’t, and this distinction matters.<\/strong><\/p>\n\n\n\n The geometry is not something the model found beneath<\/em> human language. It’s a compressed geometric encoding of human language’s actual messy behavior.<\/em> Every embedding, every direction, every curve of the manifold was learned from the statistics of how humans actually used words \u2014 drift, ambiguity, dialect and all. The 180-degree rotation of “dog” isn’t a flaw the geometry rose above; it’s information the geometry captured.<\/em> The messiness is the signal.<\/p>\n\n\n\n So the manifold is more operationally powerful<\/strong> than symbols-as-labels \u2014 better at analogy, interpolation, prediction \u2014 but it is not more absolutely true.<\/strong> It’s the map of the swamp, drawn in enough dimensions that the swamp becomes navigable. It inherits every cultural and historical contingency of the text it was trained on. It is the deepest known operational layer<\/em> of meaning in these systems. Whether anything absolute<\/em> lives underneath it is a genuinely open question \u2014 and the disciplined move is to leave it open rather than plant a flag on “bedrock” you haven’t reached.<\/p>\n\n\n\n Hold “more capable” and “more true” as separate axes.<\/strong> The geometry earns the first. It does not earn the second, and quietly spending the one on the other is the most common way smart people go wrong here.<\/p>\n\n\n\n None of this has to be taken on faith. The geometry is empirically studied, the tools are open, and if you have a single decent GPU you can decode a live model’s internals this month. Here’s the on-ramp, roughly in order of difficulty.<\/p>\n\n\n\n Start with the clean, pre-transformer version \u2014 where the king\/queen result comes from:<\/strong><\/p>\n\n\n\n Then the modern interpretability line \u2014 how meaning is packed into transformer activations:<\/strong><\/p>\n\n\n\n You don’t need frontier compute. An open-weights model (Llama, Mistral, Qwen, Gemma \u2014 anything you can load) plus a consumer GPU is enough to see the geometry directly.<\/p>\n\n\n\n Experiment 1 \u2014 Reproduce the arithmetic.<\/strong> Load static embeddings (word2vec or GloVe are tiny). Compute Experiment 2 \u2014 Watch context move a point.<\/strong> Take a transformer. Run “the dog barked” and “you filthy dog” through it. Pull the hidden-state vector for the “dog” token at several layers and measure the distance between the two versions layer by layer. You’ll see them start close (same token) and diverge<\/em> as attention routes each to its contextual meaning. You will have measured semantic drift happening inside the machine.<\/p>\n\n\n\n Experiment 3 \u2014 Linear probes.<\/strong> The most approachable interpretability technique (Alain & Bengio, “Understanding intermediate layers using linear classifier probes,”<\/em> 2016). Extract activations from one layer of a model over many inputs, then train a simple linear classifier<\/em> to read off some concept (sentiment, part of speech, whether a fact is true). If a cheap linear probe can read the concept, the concept is stored as a linear direction<\/strong> \u2014 you’ve located a piece of meaning geometrically, with your own hands.<\/p>\n\n\n\n Experiment 4 \u2014 Sparse autoencoders.<\/strong> The current frontier, and very much doable on one GPU with a smaller model. Train an SAE on a layer’s activations to decompose the tangled superposition into individual interpretable features. Open-source toolkits (search for You arrived at the core intuition \u2014 the symbol is a weak carrier; the meaning lives somewhere richer<\/em> \u2014 from pure philosophy, decades before the machines existed. The machines turned out to be a working proof of that intuition. Meaning is<\/strong> geometric in these systems; the tokens really are just the lossy shell; and you can go decode the geometry yourself with open tools and a single GPU.<\/p>\n\n\n\n Just don’t let the elegance seduce you into the last, unearned step. What you’ll be looking at is the deepest known<\/em> operational layer of meaning \u2014 a stunningly effective, empirically real, still-contingent encoding of how humans actually used their crude and beautiful tool. Whether there’s bedrock beneath it is still behind the veil.<\/p>\n\n\n\n That open question is the fun part. Go run Experiment 2 and watch a word split in two.<\/p>\n\n\n\n Written as a collaboration \u2014 the derivation follows a line of reasoning developed in conversation, rebuilt here so anyone can walk into it from their own experience of how slippery a single word can be.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":" This is the final output of a running conversation with Anthropic’s Claude. Posted here to form the basis for research into LLM mechanisms and possibly a better way to do communication than symbol based language with all of it’s short-comings. Language Is a Lossy Shell: How Language Models Actually Encode Meaning A ground-up derivation of<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","om_disable_all_campaigns":false,"pagelayer_contact_templates":[],"_pagelayer_content":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_hide_featured_image":false,"_jetpack_featured_image_duplicate_dismissed":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1817","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"\n
\n\n\n\nStarting where you already are<\/h2>\n\n\n\n
\n\n\n\nThe problem this creates for a machine<\/h2>\n\n\n\n
\n\n\n\nThe answer: meaning as geometry<\/h2>\n\n\n\n
\nking \u2212 man + woman \u2248 queen\n\n<\/pre><\/div>\n\n\n
\n\n\n\nEncoding and decoding: the full loop<\/h2>\n\n\n\n
\n
\n
\n
\n\n\n\nThe correction that keeps you honest<\/h2>\n\n\n\n
\n\n\n\nGo look at it yourself<\/h2>\n\n\n\n
The reading path<\/h3>\n\n\n\n
\n
\n
The experimental path (hands-on, runnable on your own hardware)<\/h3>\n\n\n\n
king \u2212 man + woman<\/code> and find the nearest vectors. Watch “queen” appear. Then break it: try analogies across cultural domains and watch where the geometry encodes human bias rather than logic \u2014 that’s the “map of the swamp” made visible.<\/p>\n\n\n\nSAELens<\/code> and the surrounding ecosystem) give you a running start. This is the closest you can currently get to holding a model’s concepts in your hand and turning them over.<\/p>\n\n\n\nThe tooling, concretely<\/h3>\n\n\n\n
\n
transformer_lens<\/code><\/strong> \u2014 a library built for exactly this: reaching inside a transformer, grabbing activations from any layer, and running interventions. This is your primary microscope.<\/li>\n\n\n\nnnsight<\/code><\/strong> \u2014 similar goals, good for larger models and remote execution.<\/li>\n\n\n\nscikit-learn<\/code><\/strong> \u2014 all you need for the probing experiments.<\/li>\n\n\n\nSAELens<\/code><\/strong> and related repos \u2014 for the sparse-autoencoder work.<\/li>\n\n\n\n
\n\n\n\nThe one thing to carry with you<\/h2>\n\n\n\n
\n\n\n\n