Fractal AI: A fragile theory of intelligence
This addresses the problem of inefficient exploration and high computational costs in reinforcement learning for AI agents, with potential applications across multiple scientific domains, though it appears to be a novel paradigm rather than incremental.
The paper introduces Fractal AI, a theory for general artificial intelligence that uses cellular automaton-like structures to model information, enabling a new stochastic calculus. It presents an agent derived from this theory that solves Atari games orders of magnitude more efficiently than methods like Monte Carlo Tree Search, using less than 1000 samples per action compared to 3 million for MCTS, and can generate high-performing examples with minimal computation to transform reinforcement learning into a supervised problem.
Fractal AI is a theory for general artificial intelligence. It allows deriving new mathematical tools that constitute the foundations for a new kind of stochastic calculus, by modelling information using cellular automaton-like structures instead of smooth functions. In the repository included we are presenting a new Agent, derived from the first principles of the theory, which is capable of solving Atari games several orders of magnitude more efficiently than other similar techniques, like Monte Carlo Tree Search. The code provided shows how it is now possible to beat some of the current State of The Art benchmarks on Atari games, without previous learning and using less than 1000 samples to calculate each one of the actions when standard MCTS uses 3 Million samples. Among other things, Fractal AI makes it possible to generate a huge database of top performing examples with a very little amount of computation required, transforming Reinforcement Learning into a supervised problem. The algorithm presented is capable of solving the exploration vs exploitation dilemma on both the discrete and continuous cases, while maintaining control over any aspect of the behaviour of the Agent. From a general approach, new techniques presented here have direct applications to other areas such as Non-equilibrium thermodynamics, chemistry, quantum physics, economics, information theory, and non-linear control theory.