Tillmann Miltzow

Daniel Bertschinger, Christoph Hertrich, Paul Jungeblut et al.

We consider the problem of finding weights and biases for a two-layer fully connected neural network to fit a given set of data points as well as possible, also known as EmpiricalRiskMinimization. Our main result is that the associated decision problem is $\exists\mathbb{R}$-complete, that is, polynomial-time equivalent to determining whether a multivariate polynomial with integer coefficients has any real roots. Furthermore, we prove that algebraic numbers of arbitrarily large degree are required as weights to be able to train some instances to optimality, even if all data points are rational. Our result already applies to fully connected instances with two inputs, two outputs, and one hidden layer of ReLU neurons. Thereby, we strengthen a result by Abrahamsen, Kleist and Miltzow [NeurIPS 2021]. A consequence of this is that a combinatorial search algorithm like the one by Arora, Basu, Mianjy and Mukherjee [ICLR 2018] is impossible for networks with more than one output dimension, unless $\mathsf{NP}=\exists\mathbb{R}$.

Tillmann Miltzow

We introduce a lightweight and accessible approach to computation over the real numbers, with the aim of clarifying both the underlying concepts and their relevance in modern research. The material is intended for a broad audience, including instructors who wish to incorporate real computation into algorithms courses, their students, and PhD students encountering the subject for the first time. Rather than striving for completeness, we focus on a carefully selected set of results that can be presented and proved in a classroom setting. This allows us to highlight core techniques and recurring ideas while maintaining an approachable exposition. In some places, the presentation is intentionally informal, prioritizing intuition and practical understanding over full technical precision. We position our exposition relative to existing literature, including Matousek's lecture notes on ER-completeness and the recent compendium of ER-complete problems by Schaefer, Cardinal, and Miltzow. While these works provide deep and comprehensive perspectives, our goal is to offer an accessible entry point with proofs and examples suitable for teaching. Our approach follows modern formulations of real computation that emphasize binary input, real-valued witnesses, and restricted use of constants, aligning more closely with contemporary complexity theory, while acknowledging the foundational contributions of the Blum--Shub--Smale model.

Lucas Meijer, Tillmann Miltzow

In the art gallery problem, we are given a closed polygon $P$, with rational coordinates and an integer $k$. We are asked whether it is possible to find a set (of guards) $G$ of size $k$ such that any point $p\in P$ is seen by a point in $G$. We say two points $p$, $q$ see each other if the line segment $pq$ is contained inside $P$. It was shown by Abrahamsen, Adamaszek, and Miltzow that there is a polygon that can be guarded with three guards, but requires four guards if the guards are required to have rational coordinates. In other words, an optimal solution of size three might need to be irrational. We show that an optimal solution of size two might need to be irrational. Note that it is well-known that any polygon that can be guarded with one guard has an optimal guard placement with rational coordinates. Hence, our work closes the gap on when irrational guards are possible to occur.

Tillmann Miltzow, Reinier F. Schmiermann

A continuous constraint satisfaction problem (CCSP) is a constraint satisfaction problem (CSP) with an interval domain $U \subset \mathbb{R}$. We engage in a systematic study to classify CCSPs that are complete of the Existential Theory of the Reals, i.e., ER-complete. To define this class, we first consider the problem ETR, which also stands for Existential Theory of the Reals. In an instance of this problem we are given some sentence of the form $\exists x_1, \ldots, x_n \in \mathbb{R} : Φ(x_1, \ldots, x_n)$, where $Φ$ is a well-formed quantifier-free formula consisting of the symbols $\{0, 1, +, \cdot, \geq, >, \wedge, \vee, \neg\}$, the goal is to check whether this sentence is true. Now the class ER is the family of all problems that admit a polynomial-time many-one reduction to ETR. It is known that NP $\subseteq$ ER $\subseteq$ PSPACE. We restrict our attention on CCSPs with addition constraints ($x + y = z$) and some other mild technical conditions. Previously, it was shown that multiplication constraints ($x \cdot y = z$), squaring constraints ($x^2 = y$), or inversion constraints ($x\cdot y = 1$) are sufficient to establish ER-completeness. We extend this in the strongest possible sense for equality constraints as follows. We show that CCSPs (with addition constraints and some other mild technical conditions) that have any one well-behaved curved equality constraint ($f(x,y) = 0$) are ER-complete. We further extend our results to inequality constraints. We show that any well-behaved convexly curved and any well-behaved concavely curved inequality constraint ($f(x,y) \geq 0$ and $g(x,y) \geq 0$) imply ER-completeness on the class of such CCSPs.

Mikkel Abrahamsen, Linda Kleist, Tillmann Miltzow

Given a neural network, training data, and a threshold, it was known that it is NP-hard to find weights for the neural network such that the total error is below the threshold. We determine the algorithmic complexity of this fundamental problem precisely, by showing that it is $\exists\mathbb R$-complete. This means that the problem is equivalent, up to polynomial-time reductions, to deciding whether a system of polynomial equations and inequalities with integer coefficients and real unknowns has a solution. If, as widely expected, $\exists\mathbb R$ is strictly larger than NP, our work implies that the problem of training neural networks is not even in NP. Neural networks are usually trained using some variation of backpropagation. The result of this paper offers an explanation why techniques commonly used to solve big instances of NP-complete problems seem not to be of use for this task. Examples of such techniques are SAT solvers, IP solvers, local search, dynamic programming, to name a few general ones.