Michał P. Karpowicz

Michał P. Karpowicz, Gilbert Strang

We investigate the Moore-Penrose pseudoinverse and generalized inverse of a matrix product $A=CR$ to establish a unifying framework for generalized and randomized matrix inverses. This analysis is rooted in first principles, focusing on the geometry of the four fundamental subspaces. We examine: (1) the reverse order law, $A^+ = R^+C^+$, which holds when $C$ has independent columns and $R$ has independent rows, (2) the universally correct formula, $A^+ = (C^+CR)^+(CRR^+)^+$, providing a geometric interpretation of the mappings between the involved subspaces, (3) a new generalized randomized formula, $A^+_p = (P^TA)^+P^TAQ(AQ)^+$, which gives $A^+_p = A^+$ if and only if the sketching matrices $P$ and $Q$ preserve the rank of $A$, i.e., $\mathrm{rank}(P^TA) = \mathrm{rank}(AQ) = \mathrm{rank}(A)$. The framework is extended to generalized $\{1,2\}$-inverses and specialized forms, revealing the underlying structure of established randomized linear algebra algorithms, including randomized SVD, the Nyström approximation, and CUR decomposition. We demonstrate applications in sparse sensor placement and effective resistance estimation. For the latter, we provide a rigorous quantitative analysis of an approximation scheme, establishing that it always underestimates the true resistance and deriving a worst-case spectral bound on the error of resistance differences.

Michał P. Karpowicz

This paper establishes a fundamental Impossibility Theorem: no LLM performing non-trivial knowledge aggregation can simultaneously achieve truthful knowledge representation, semantic information conservation, complete revelation of relevant knowledge, and knowledge-constrained optimality. This impossibility stems from the mathematical structure of information aggregation, not from engineering limitations. We prove this by modeling inference as an auction of ideas, where distributed components compete to influence responses using their encoded knowledge. The proof employs three independent approaches: mechanism design (Green-Laffont theorem), proper scoring rules (Savage), and transformer architecture analysis (log-sum-exp convexity). We introduce the semantic information measure and the emergence operator to analyze computationally bounded and unbounded reasoning. Bounded reasoning makes latent information accessible, enabling gradual insights and creativity, while unbounded reasoning makes all derivable knowledge immediately accessible while preserving the semantic content. We prove the conservation-reasoning dichotomy: meaningful reasoning necessarily violates information conservation. Our framework suggests that hallucination and imagination are mathematically identical, and both violate at least one of the four essential properties. The Jensen gap in transformer attention quantifies this violation as excess confidence beyond constituent evidence. This unified view explains why capable models must balance truthfulness against creativity. These results provide principled foundations for managing hallucination trade-offs in AI systems. Rather than eliminating hallucination, we should optimize these inevitable trade-offs for specific applications. We conclude with philosophical implications connecting the impossibility to fundamental limits of reason.