Epistemological Fault Lines Between Human and Artificial Intelligence
Reference: Quattrociocchi, W., Capraro, V., & Perc, M. (2025). Epistemological Fault Lines Between Human and Artificial Intelligence. arXiv:2512.19466v1 [cs.CY]. URL
Summary
The paper argues that the surface-level alignment between LLM outputs and human judgment masks a deep structural mismatch in how the two arrive at their answers. Tracing a historical shift from symbolic AI and information-retrieval systems — which treated language as a vehicle for explicit propositional content — to large-scale generative transformers, the authors observe that LLMs do not “reason under beliefs about the world” but rather perform stochastic pattern-completion over high-dimensional embeddings of linguistic transitions. The appearance of rationality is a by-product of having absorbed the human linguistic record, not of having the cognitive machinery that originally produced it.
To make the mismatch concrete, the authors decompose judgment into seven stages and show that at each stage the human and LLM pipelines follow fundamentally different trajectories: (1) sensory/social input vs. textual input; (2) perceptual-situational parsing vs. tokenization; (3) memory/intuitions/learned concepts vs. statistical pattern-recognition in embeddings; (4) emotions/motivations/goals vs. statistical inference via neural layers; (5) reasoning vs. textual context integration; (6) metacognitive calibration vs. forced confidence and hallucination; (7) value-sensitive judgment vs. probabilistic next-token prediction. Each pair corresponds to one of seven epistemological fault lines: Grounding, Parsing, Experience, Motivation, Causality, Metacognition, Value. These fault lines are structural, not bugs to be patched: they are consequences of the architecture itself, so fluency-on-the-surface keeps producing confidence-like outputs even where no justifying process exists.
The central positive construct is Epistemia: a structural condition in which linguistic plausibility substitutes for epistemic evaluation. Under Epistemia, a system produces answers that are syntactically well-formed, semantically fluent, and rhetorically convincing without instantiating the processes by which beliefs are normally formed, tested, or revised. The harm is not that the system is factually wrong — it often is not — but that the user is placed in a position of epistemic passivity where justification is displaced by pre-packaged persuasion. The paper closes by arguing for epistemic evaluation beyond surface alignment (benchmarks that probe calibration, counterfactual sensitivity, and refusal, not just accuracy), epistemic governance beyond behavioural alignment (regulating categories of epistemic failure rather than individual outputs), and epistemic literacy beyond critical thinking (teaching users procedural safeguards against surrogate evaluation, not merely scepticism).
Key Ideas
- Human and LLM judgment pipelines diverge at every stage, not only at the output; the apparent similarity is a linguistic-plausibility artefact.
- Seven epistemological fault lines — Grounding, Parsing, Experience, Motivation, Causality, Metacognition, Value — each corresponding to a structural divergence in the pipeline.
- Epistemia: linguistic plausibility substitutes for epistemic evaluation; a regime in which answers arrive without the labour of justification.
- Hallucination is not a bug but a structural consequence of a system that must always produce an output and lacks metacognition to suspend judgment.
- LLMs have no causal, counterfactual, or value-grounded evaluation; they integrate textual context to produce surface-level plausible continuations.
- Epistemia is not reducible to automation bias or over-trust — it is an architectural condition: users interact with systems that lack the internal machinery for forming, holding, or revising beliefs.
- Three responses: epistemic evaluation beyond surface alignment; epistemic governance beyond behavioural alignment; epistemic literacy beyond critical thinking.
Connections
Conceptual Contribution
- Claim: Linguistic fluency in LLMs is not evidence of judgment; the two pipelines diverge along seven structural fault lines, and their surface similarity produces a systemic epistemic pathology — Epistemia — in which linguistic plausibility replaces the labour of evaluation.
- Mechanism: Decompose judgment into seven stages (input → parsing → memory/concepts → motivation → reasoning → metacognition → value) and compare the human pipeline (sensory-social, grounded, embodied, metacognitively monitored, value-sensitive) with the LLM pipeline (textual, tokenized, statistically embedded, forced to output, probabilistic). Identify the divergence at each stage as a fault line; show each produces characteristic failures (grounding error, tokenization drift, spurious associations, missing motivational significance, counterfactual blindness, hallucination, absent accountability). Frame the aggregate as Epistemia — an architectural regime, not a user-side bias.
- Concepts introduced/used: Epistemia, Grounding, Hallucination, Chinese Room Argument, Symbol Grounding Problem, Intentional Stance, Intelligent Agents Theory and Practice
- Stance: philosophical / critical-theoretic perspective on LLMs
- Relates to: Extends Symbol Grounding Problem and the Searle/Dennett debates (Chinese Room Argument, Minds Brains and Science, True Believers - The Intentional Strategy and Why It Works) to the large-model era; sharpens critiques in Why AI Agents Communicate In Human Language; counterpoint to the scaling optimism of The Bitter Lesson; foundational framing for the “what is an LLM judgment, actually?” question that underlies LLM Agents evaluation.