Learning Compact Boolean Networks
This work addresses the problem of high inference costs in machine learning for resource-constrained applications, representing a strong specific gain in efficiency.
The paper tackles the challenge of learning compact and accurate Boolean networks for resource-constrained settings by proposing strategies for learned connections, compact convolutions, and adaptive discretization, resulting in a method that achieves better accuracy with up to 37x fewer Boolean operations compared to prior state-of-the-art.
Floating-point neural networks dominate modern machine learning but incur substantial inference cost, motivating interest in Boolean networks for resource-constrained settings. However, learning compact and accurate Boolean networks is challenging due to their combinatorial nature. In this work, we address this challenge from three different angles: learned connections, compact convolutions and adaptive discretization. First, we propose a novel strategy to learn efficient connections with no additional parameters and negligible computational overhead. Second, we introduce a novel convolutional Boolean architecture that exploits the locality with reduced number of Boolean operations than existing methods. Third, we propose an adaptive discretization strategy to reduce the accuracy drop when converting a continuous-valued network into a Boolean one. Extensive results on standard vision benchmarks demonstrate that the Pareto front of accuracy vs. computation of our method significantly outperforms prior state-of-the-art, achieving better accuracy with up to 37x fewer Boolean operations.