Learning One Abstract Bit at a Time Through Self-Invented Experiments Encoded as Neural Networks
This addresses the challenge of autonomous scientific discovery in AI, though it appears incremental as it builds on existing neural network and reinforcement learning methods.
The paper tackles the problem of enabling artificial scientists to not only answer questions but also invent new ones through self-generated experiments, including thought experiments encoded as neural networks, and shows that this leads to effective exploration in reinforcement-providing environments.
There are two important things in science: (A) Finding answers to given questions, and (B) Coming up with good questions. Our artificial scientists not only learn to answer given questions, but also continually invent new questions, by proposing hypotheses to be verified or falsified through potentially complex and time-consuming experiments, including thought experiments akin to those of mathematicians. While an artificial scientist expands its knowledge, it remains biased towards the simplest, least costly experiments that still have surprising outcomes, until they become boring. We present an empirical analysis of the automatic generation of interesting experiments. In the first setting, we investigate self-invented experiments in a reinforcement-providing environment and show that they lead to effective exploration. In the second setting, pure thought experiments are implemented as the weights of recurrent neural networks generated by a neural experiment generator. Initially interesting thought experiments may become boring over time.