Informal Safety Guarantees for Simulated Optimizers Through Extrapolation from Partial Simulations
This addresses safety concerns for AI systems using self-supervised learning, but it is incremental as it builds on existing models like Cartesian frames.
The paper tackles the problem of proving alignment between simulacra in self-supervised language models by introducing a mathematical model based on Cartesian objects, and it shows that direct proof is impossible due to the Löbian obstacle, proposing a scheme called Partial Simulation Extrapolation to circumvent this by evaluating low-complexity simulations.
Self-supervised learning is the backbone of state of the art language modeling. It has been argued that training with predictive loss on a self-supervised dataset causes simulators: entities that internally represent possible configurations of real-world systems. Under this assumption, a mathematical model for simulators is built based in the Cartesian frames model of embedded agents, which is extended to multi-agent worlds through scaling a two-dimensional frame to arbitrary dimensions, where literature prior chooses to instead use operations on frames. This variant leveraging scaling dimensionality is named the Cartesian object, and is used to represent simulations (where individual simulacra are the agents and devices in that object). Around the Cartesian object, functions like token selection and simulation complexity are accounted for in formalizing the behavior of a simulator, and used to show (through the Löbian obstacle) that a proof of alignment between simulacra by inspection of design is impossible in the simulator context. Following this, a scheme is proposed and termed Partial Simulation Extrapolation aimed at circumventing the Löbian obstacle through the evaluation of low-complexity simulations.