Enhancing Computer Vision with Knowledge: a Rummikub Case Study
This addresses a specific weakness in computer vision for board game solving, but it is incremental as it applies known methods to a new domain.
The paper tackled the problem of neural networks failing to integrate and interpret image components as a whole by adding explicit knowledge and reasoning, applied to solving the board game Rummikub, resulting in background knowledge being as valuable as two-thirds of the dataset and reducing training time by half.
Artificial Neural Networks excel at identifying individual components in an image. However, out-of-the-box, they do not manage to correctly integrate and interpret these components as a whole. One way to alleviate this weakness is to expand the network with explicit knowledge and a separate reasoning component. In this paper, we evaluate an approach to this end, applied to the solving of the popular board game Rummikub. We demonstrate that, for this particular example, the added background knowledge is equally valuable as two-thirds of the data set, and allows to bring down the training time to half the original time.