TMComposites: Plug-and-Play Collaboration Between Specialized Tsetlin Machines
This work addresses performance limitations in Tsetlin Machines for more complex image classification tasks, offering an incremental improvement through collaborative specialization.
The paper tackles the problem of Tsetlin Machines struggling with complex image classification tasks like CIFAR-10 and CIFAR-100 by introducing TM Composites, a plug-and-play collaboration method between specialized TMs, resulting in accuracy increases of 12 percentage points on CIFAR-10 and 9 points on CIFAR-100, achieving new state-of-the-art results for TMs.
Tsetlin Machines (TMs) provide a fundamental shift from arithmetic-based to logic-based machine learning. Supporting convolution, they deal successfully with image classification datasets like MNIST, Fashion-MNIST, and CIFAR-2. However, the TM struggles with getting state-of-the-art performance on CIFAR-10 and CIFAR-100, representing more complex tasks. This paper introduces plug-and-play collaboration between specialized TMs, referred to as TM Composites. The collaboration relies on a TM's ability to specialize during learning and to assess its competence during inference. When teaming up, the most confident TMs make the decisions, relieving the uncertain ones. In this manner, a TM Composite becomes more competent than its members, benefiting from their specializations. The collaboration is plug-and-play in that members can be combined in any way, at any time, without fine-tuning. We implement three TM specializations in our empirical evaluation: Histogram of Gradients, Adaptive Gaussian Thresholding, and Color Thermometers. The resulting TM Composite increases accuracy on Fashion-MNIST by two percentage points, CIFAR-10 by twelve points, and CIFAR-100 by nine points, yielding new state-of-the-art results for TMs. Overall, we envision that TM Composites will enable an ultra-low energy and transparent alternative to state-of-the-art deep learning on more tasks and datasets.