Convolutional Differentiable Logic Gate Networks
This work addresses the need for fast and efficient inference in machine learning models, particularly for applications where computational resources are limited, by advancing logic gate networks in an incremental manner.
The paper tackles the problem of high inference costs in machine learning by scaling up differentiable logic gate networks through deep logic gate tree convolutions, logical OR pooling, and residual initializations, achieving 86.29% accuracy on CIFAR-10 with 61 million logic gates, which improves over state-of-the-art while being 29x smaller.
With the increasing inference cost of machine learning models, there is a growing interest in models with fast and efficient inference. Recently, an approach for learning logic gate networks directly via a differentiable relaxation was proposed. Logic gate networks are faster than conventional neural network approaches because their inference only requires logic gate operators such as NAND, OR, and XOR, which are the underlying building blocks of current hardware and can be efficiently executed. We build on this idea, extending it by deep logic gate tree convolutions, logical OR pooling, and residual initializations. This allows scaling logic gate networks up by over one order of magnitude and utilizing the paradigm of convolution. On CIFAR-10, we achieve an accuracy of 86.29% using only 61 million logic gates, which improves over the SOTA while being 29x smaller.