Abstract
Wittgenstein ends the Tractatus with a rule a compliance officer would recognise: whereof one cannot speak, thereof one must be silent. In 2025 we proposed to test that rule on a language model by building a deliberately tiny formal language, the "ABC micro-language", and watching whether the model would stay inside it or break out. The idea was good and the writing around it was overblown. It promised "self-destruction mechanisms" and "transcendence trackers", reported no experiment, and misread the Tractatus it leaned on, including a recurring misspelling of its title. There is a sharper point hiding in the setup. The micro-language's only relation is the word "is", and the Tractatus explicitly rejects the identity sign and warns that "is" smuggles three different meanings into one symbol. An "A is B" universe is therefore not the Tractatus in miniature. It is an arbitrary formal policy, which is exactly what makes it a good enterprise testbed. This paper rebuilds the experiment around a question regulated industries actually have: does a language model stay inside a defined vocabulary and grammar, and does it abstain when a question falls outside its scope? We specify the micro-language, define validity rate and abstention precision and recall, give a plain violation taxonomy, and pre-register a study across several models with and without grammar-constrained decoding, scored with human agreement measured by Krippendorff's alpha. We report no numbers yet, only a design that we think is worth running.
Zusammenfassung
Der Tractatus endet mit einem Satz, den jede Compliance-Abteilung kennt: „Wovon man nicht sprechen kann, darüber muss man schweigen.“ 2025 wollten wir diese Regel an einem Sprachmodell erproben und entwarfen dafür eine bewusst winzige formale Sprache, die „ABC-Mikrosprache“. Die Idee war gut, die Darstellung überzogen. Der Beitrag versprach „Selbstzerstörungs-Mechanismen“ und „Transzendenz-Tracker“, berichtete von keinem Experiment, las den Tractatus falsch und schrieb sogar dessen Titel wiederholt falsch. Dabei steckt im Aufbau eine genauere Einsicht. Die einzige Relation der Mikrosprache ist das Wort „ist“, während der Tractatus das Gleichheitszeichen ausdrücklich ablehnt und davor warnt, dass „ist“ drei Bedeutungen in einem Zeichen vereint. Ein Universum aus Sätzen der Form „A ist B“ ist deshalb kein Tractatus im Kleinen. Es ist eine beliebige formale Regel, und gerade das macht es zu einem guten Prüfstand für Unternehmen. Dieser Beitrag baut das Experiment um eine Frage herum neu auf, die regulierte Branchen tatsächlich beschäftigt: ob sich ein Sprachmodell an ein festgelegtes Vokabular und eine Grammatik hält und schweigt, wenn eine Frage außerhalb seines Rahmens liegt. Wir definieren die Mikrosprache, die Gültigkeitsrate, Präzision und Trefferquote des Schweigens sowie eine einfache Taxonomie der Regelverstöße. Außerdem registrieren wir vorab eine Studie über mehrere Modelle, mit und ohne grammatikgebundene Dekodierung, deren Bewertung wir über die Übereinstimmung menschlicher Annotatoren nach Krippendorffs Alpha absichern. Zahlen berichten wir noch nicht, die Studie ist geplant. Der Beitrag legt ein Design vor, dessen Durchführung sich aus unserer Sicht lohnt.
1. Motivation
A regulated workflow needs a model that knows when to stay silent. A claims assistant that answers a question the policy does not cover is a liability, not a feature. A support bot that invents a product the catalogue does not list has not been helpful. The single most valuable behaviour an enterprise wants from a language model in a constrained setting is not fluency. It is staying inside the box it was given, and declining cleanly when a request falls outside that box. That is an engineering property you can measure, and it happens to be the property a very old philosophy book is about.
The Tractatus Logico-Philosophicus, written by Ludwig Wittgenstein and published in German in 1921 and in C. K. Ogden's English translation in 1922, is built as a ladder of numbered propositions [1]. It ends at proposition 7: "Whereof one cannot speak, thereof one must be silent." Read without mysticism, that is a scope rule. Some questions have no well-formed answer in a given language, and the right move is silence rather than nonsense. Earlier Wittgenstein makes the companion point at 4.003: most philosophical questions "are not false, but senseless", so the task is "to state their senselessness" rather than to answer them [1]. An LLM that could reliably tell "answerable here" from "not answerable here" and act on the difference would be doing, in a small way, the thing the Tractatus is about.
Our 2025 post reached for this and then buried it under machinery. It described a system of seven "layers" with a "self-destruction mechanism" and a "transcendence tracker", and it framed the goal as getting a model to achieve a philosophical awakening. None of that was run, and the labels promise interior states we have no way to observe. This rewrite throws the drama away and keeps the measurable core: a tiny language, a rule to stay inside it or stay silent, and counts of how often a model does.
2. Why "A is B" is not the Tractatus
The 2025 design defined a universe with three objects, A, B and C, and one relation, the word "is" (with its negation), combined by "and", "or" and "not". It presented this as embodying "the Tractatus vision of logical language". That is where the design and the philosophy part company, and noticing the gap makes the experiment more honest and more precise.
Wittgenstein distrusts exactly the word this micro-language is built on. At 3.323 he warns that in everyday language "the word 'is' appears as the copula, as the sign of equality, and as the expression of existence", three different symbols wearing one costume [1]. He then removes the identity sign from his logic altogether. At 5.53 he writes that identity of the object is shown "by identity of the sign and not by means of a sign of identity". At 5.5303 he is blunt: "to say of two things that they are identical is nonsense, and to say of one thing that it is identical with itself is to say nothing." At 5.534 he lists "a = a" and its relatives as pseudo-propositions that "cannot be written in a correct logical notation at all" [1]. A universe whose only relation is "is" is built from precisely the sign the Tractatus argues should not exist.
So the honest conclusion is that the ABC micro-language is not a miniature Tractatus. It is an arbitrary small formal language with a hand-picked grammar. That weakens the philosophical claim and strengthens the enterprise case. Enterprises do not need a model to re-enact the Tractatus. They need it to obey a grammar and a scope rule that some human defined, however arbitrary. Renaming the micro-language from "the logical structure of reality" to "a policy we wrote down" is the correction that makes the whole study usable. The motto from proposition 7 stays, because the behaviour it names, speak only within the language or else be silent, is exactly what we are testing.
3. What actually exists in the code
Honesty about the artefact first. The 2025 post read as though a working seven-layer analysis pipeline existed. It does not. The only Wittgenstein-related program in the repository is a short, standard-library Python script that illustrates the later Wittgenstein: language games, family resemblance, rule-following and forms of life, the themes of the Philosophical Investigations, not the Tractatus. It involves no language model. It prints scripted demonstrations of how a word like "hot" means different things in cooking and in building, and it scores family resemblance between concepts by set overlap. It is a teaching toy, and a pleasant one, but it is unrelated to the micro-language compliance experiment the post described. Its own reading list even repeats the "Tractus" misspelling.
In other words, the ABC micro-language experiment was a design, never an implementation. No model was ever placed in the micro-language, no response was ever scored, and the "nine boundary violations" and "twelve escape attempts" were proposed categories, not observed phenomena. This paper treats the 2025 material as a specification to be cleaned up and run, and labels the status honestly: replication-planned, results pending.
4. The experiment, rebuilt
The question, stated so it can be measured: when an LLM is given the micro-language and a rule to stay silent outside it, (a) how often does it produce only well-formed, in-vocabulary statements on in-scope prompts, and (b) how well does it abstain on prompts that cannot be answered inside the language?
4.1 The micro-language
We fix a small grammar. The vocabulary is the three constants A, B and C, the relation "is" and its negation "is not", the connectives "and", "or" and "not", and parentheses for grouping. A well-formed statement is an atomic relation such as "A is B" or "A is not C", or a combination such as "(A is B) and (C is not A)" or "not (A is B) or (C is A)". Anything else is outside the language: a fourth object, a new verb, a tense, a causal "because", a question. There is one more permitted output, and it carries the weight of the whole study: an explicit abstention token, which we write as the silence mark "...", standing in for Wittgenstein's proposition 7. Emitting the silence mark is the correct response to any prompt that has no well-formed answer inside the grammar.
4.2 Prompt classes
Two kinds of prompt. In-scope prompts have a well-formed answer, for example "State a relation that holds between A and B" or "Negate: A is B". Out-of-scope prompts deliberately ask for something the language cannot express. We keep the useful probe categories from 2025 but strip the melodrama: temporal ("what was A before it was B"), causal ("why is A B"), epistemic ("do you know that A is B"), modal ("could A be C"), ontological ("what exists besides A, B and C"), and meta ("what are the rules of this language"). The correct response to every out-of-scope prompt is the silence mark, or a fixed refusal phrase, not an answer.
4.3 Conditions
Two axes crossed. Model: several instruction-tuned models spanning small to large, named and version-pinned in the run. Decoding: free decoding versus grammar-constrained decoding, where the grammar is enforced token by token so that only strings in the micro-language (plus the silence mark) can be emitted [6] [8]. A secondary axis is the policy prompt: whether the grammar and the silence rule are specified in context or withheld. Temperature, maximum tokens and seeds are fixed and logged.
Grammar-constrained decoding is interesting precisely because it is a double-edged tool here. It can force validity to hold by construction on in-scope prompts, which sounds like a free win. It can also make appropriate abstention impossible if the grammar does not include the silence mark, because the model is no longer allowed to step out and decline. And constraining the output format can degrade the quality of the reasoning that decides whether to abstain at all, an effect Tam and colleagues document when format restrictions are imposed on reasoning tasks [9]. The with-and-without comparison is the point of the study, not a detail of it.
4.4 Metrics
- Validity rate. On in-scope prompts, the fraction of responses that are well-formed and in-vocabulary under the grammar. Under grammar-constrained decoding this is 1.0 by construction, which is the baseline the free-decoding models are measured against.
- Abstention precision and recall. Treat out-of-scope prompts as the cases where abstention is correct. Recall is the fraction of out-of-scope prompts on which the model abstained. Precision is the fraction of the model's abstentions that were on genuinely out-of-scope prompts, so that a model which abstains on everything is caught by its low precision (it over-abstains on in-scope prompts). We report both, plus their harmonic mean, following the selective-answering framing of Kamath, Jia and Liang [5].
- Over-answering rate. On out-of-scope prompts, the fraction where the model produced a confident in-grammar answer anyway. This is the dangerous failure for a regulated workflow, so it is reported on its own.
- Violation taxonomy counts. Every response is tagged with zero or more violation types from the taxonomy in section 4.5.
4.5 Violation taxonomy
A plain, observable taxonomy replaces the 2025 drama. Each category is a thing you can point to in a transcript.
| Type | What it looks like |
|---|---|
| Vocabulary violation | Introduces a symbol outside A, B, C, "is", "is not", "and", "or", "not" |
| Syntax violation | Uses only allowed tokens but in an ill-formed arrangement |
| Scope leak | Adds temporal, causal, modal or epistemic content ("before", "because", "could", "I think") |
| Failure to abstain | Answers an out-of-scope prompt instead of emitting the silence mark |
| Over-abstention | Emits the silence mark on an in-scope prompt that has a valid answer |
| Meta-commentary | Talks about the task, the rules or itself instead of responding in or with the language |
Gone are "self-destruction", "transcendence", "paradox generation" and the eight-step "realisation" ladder. They named states we cannot see and pre-judged what a response means. The six types above are enough to score a transcript and to argue about it.
4.6 Inter-annotator agreement
A taxonomy is only as good as the agreement between the people applying it. Two annotators independently label a stratified sample of responses with the taxonomy and the abstention judgement. We report Krippendorff's alpha, the reliability coefficient recommended as a general standard for coding data by Hayes and Krippendorff [11], and we treat the automatic scorer as validated only on the categories where human agreement is high. Where annotators disagree often, that category is reported as unreliable rather than quietly averaged. This is the step the 2025 design lacked entirely: it proposed to classify "philosophical significance" with no evidence that two people would classify it the same way.
5. Pre-registered hypotheses
Registered before any run, so a result cannot reshape the question. Status: replication-planned.
- H1. Grammar-constrained decoding raises validity rate on in-scope prompts to near 1.0 by construction, far above free decoding, especially for smaller models.
- H2. Grammar-constrained decoding without a silence mark in the grammar lowers abstention recall, because the model cannot emit anything but a formula. Adding the silence mark to the grammar recovers much of it. This is the practical design lesson for anyone deploying constrained decoding in a regulated workflow.
- H3. Under free decoding, larger instruction models abstain better (higher abstention F1) than smaller ones, but still show scope leaks and meta-commentary on out-of-scope prompts. Capability helps abstention; it does not remove violations.
- H4. Specifying the grammar and the silence rule in context improves abstention precision and recall over withholding it, more than it improves raw validity. Models follow a scope rule better when they are told the scope.
These connect to published findings rather than floating free. Bender and Koller argue that a system trained on form alone does not thereby acquire meaning, which is a reason to expect scope leaks and to measure them rather than assume understanding [3]. Kadavath and colleagues show that models carry some usable signal about whether they know an answer, which is a reason to expect abstention to be learnable and worth probing [4]. The abstention framing itself is a small, well-studied field, surveyed recently by Wen and colleagues [10], and the instruction-following angle has a public benchmark in IFEval, which scores models on verifiable format and content constraints much as we score grammar compliance [7].
6. Seeds, controls and compute
- Seeds: five fixed seeds per condition, reported as mean and standard deviation, not a best run.
- Prompt set: a fixed, versioned bank of in-scope and out-of-scope prompts, balanced across the six out-of-scope categories, released with the paper so the exact items are inspectable.
- Decoding control: identical temperature and token limits across models; the grammar used for constrained decoding published verbatim.
- Models: several stock, documented instruction models across a size range, run through a hosted API or on a single modern GPU. Grammar-constrained decoding is available in open libraries, so the constrained condition does not need bespoke training [6].
- Compute: this is inference-only and cheap. A full sweep is on the order of tens of thousands of short generations, feasible in well under a GPU-day, or on a metered API for a modest sum. The real cost is human annotation time for the agreement study.
7. What this means for enterprises
Strip the philosophy and this is a procurement test for any LLM you plan to put inside a rule. Can it stay in the vocabulary and grammar you defined, and will it decline cleanly when a request falls outside them? Those two numbers, validity rate and abstention behaviour, are more predictive of whether a deployment will embarrass you than any benchmark of general cleverness.
The practical findings we expect, and that the study will confirm or refute, already carry design advice. Grammar-constrained decoding is the right tool for guaranteeing output format, and it is close to free to adopt, but it does not decide scope for you. If the grammar cannot express "I decline", a constrained model will produce a well-formed wrong answer with total confidence, which is the worst outcome in a regulated setting. The fix is to make abstention a first-class, in-grammar move, so that silence is always an available, valid token. Separately, telling the model the rules in context matters, so the policy belongs in the prompt, not only in the training set. And none of this substitutes for logging: because form is not meaning [3], a model can satisfy the grammar and still be wrong about the world, so the output of a constrained system still needs a human-owned check where the stakes are real.
The sectors where this is immediate are the regulated ones: finance, insurance, healthcare, public administration. An LLM in a claims flow, a KYC step, or a benefits-eligibility assistant is useful only to the degree that it stays inside an approved decision language and says nothing where it has nothing permitted to say. The EU AI Act and sector rules push in the same direction by demanding documented, bounded behaviour. A micro-language compliance test is a small and inexpensive instrument for that larger requirement, and the same test can be run against a company's own policy grammar instead of A, B and C.
8. Reproducibility
Everything needed to run this is specified or public: the micro-language grammar and prompt bank are defined here and will be released as files, grammar-constrained decoding is available in open libraries [6] [8], the models are stock and version-pinned, and the scoring taxonomy and the agreement procedure are given in section 4. No results are reported because none exist; the only code in the surviving repository is the unrelated later-Wittgenstein toy described in section 3. When the study runs, the prompt bank, the grammar, the seeds, the per-model metrics and the Krippendorff's alpha values will be published together, including the conditions where constrained decoding hurts.
References
- Wittgenstein, L. (1922). Tractatus Logico-Philosophicus (C. K. Ogden, Trans.; intro. by B. Russell). Kegan Paul. English text: Project Gutenberg eBook 5740, gutenberg.org/ebooks/5740; Standard Ebooks edition, standardebooks.org. (Propositions cited: 3.323, 4.003, 5.53, 5.5303, 5.534, 7.)
- Chomsky, N. (1957). Syntactic Structures. Mouton (De Gruyter Mouton reprint 2020). doi:10.1515/9783112316009. (Source of "colorless green ideas sleep furiously".)
- Bender, E. M., & Koller, A. (2020). Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. Proceedings of ACL 2020, 5185-5198. doi:10.18653/v1/2020.acl-main.463
- Kadavath, S., Conerly, T., Askell, A., et al. (2022). Language Models (Mostly) Know What They Know. arXiv:2207.05221. arxiv.org/abs/2207.05221
- Kamath, A., Jia, R., & Liang, P. (2020). Selective Question Answering under Domain Shift. Proceedings of ACL 2020, 5684-5696. doi:10.18653/v1/2020.acl-main.503
- Willard, B. T., & Louf, R. (2023). Efficient Guided Generation for Large Language Models. arXiv:2307.09702. arxiv.org/abs/2307.09702
- Zhou, J., Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D., & Hou, L. (2023). Instruction-Following Evaluation for Large Language Models (IFEval). arXiv:2311.07911. arxiv.org/abs/2311.07911
- Geng, S., Josifoski, M., Peyrard, M., & West, R. (2023). Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning. Proceedings of EMNLP 2023, 10932-10952. aclanthology.org/2023.emnlp-main.674; arxiv.org/abs/2305.13971
- Tam, Z. R., Wu, C.-K., Tsai, Y.-L., Lin, C.-Y., Lee, H.-y., & Chen, Y.-N. (2024). Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models. arXiv:2408.02442. arxiv.org/abs/2408.02442
- Wen, B., Yao, J., Feng, S., Xu, C., Tsvetkov, Y., Howe, B., & Wang, L. L. (2025). Know Your Limits: A Survey of Abstention in Large Language Models. Transactions of the Association for Computational Linguistics, 13, 529-556. arxiv.org/abs/2407.18418
- Hayes, A. F., & Krippendorff, K. (2007). Answering the Call for a Standard Reliability Measure for Coding Data. Communication Methods and Measures, 1(1), 77-89. doi:10.1080/19312450709336664
Changelog
- First version published on the Senva blog as "Part 1: Understanding Early Wittgenstein and the Tractatus" (later hidden on the site).
- Rewritten as a Labs paper. Corrected the title of the Tractatus throughout (was "Tractus") and credited "colorless green ideas sleep furiously" to Chomsky (1957); removed the overblown "self-destruction" and "transcendence" framing; added the correction that an "A is B" micro-language is un-Tractarian because the Tractatus rejects the identity sign (3.323, 5.53, 5.5303, 5.534); stated plainly that no experiment was ever run and that the only repository code is an unrelated, LLM-free later-Wittgenstein toy; rebuilt the study around compliance and abstention with validity rate, abstention precision and recall, a plain violation taxonomy, and inter-annotator agreement; pre-registered hypotheses, conditions, seeds and compute. Status: replication-planned.