LogicCBMs: Logic-Enhanced Concept-Based Learning
This work addresses the need for more expressive and interpretable models in machine learning, though it is incremental as it builds upon existing CBMs.
The authors tackled the limitation of linear concept combinations in Concept Bottleneck Models (CBMs) by enhancing them with propositional logic, resulting in improved accuracy and interpretability on benchmarks and synthetic datasets.
Concept Bottleneck Models (CBMs) provide a basis for semantic abstractions within a neural network architecture. Such models have primarily been seen through the lens of interpretability so far, wherein they offer transparency by inferring predictions as a linear combination of semantic concepts. However, a linear combination is inherently limiting. So we propose the enhancement of concept-based learning models through propositional logic. We introduce a logic module that is carefully designed to connect the learned concepts from CBMs through differentiable logic operations, such that our proposed LogicCBM can go beyond simple weighted combinations of concepts to leverage various logical operations to yield the final predictions, while maintaining end-to-end learnability. Composing concepts using a set of logic operators enables the model to capture inter-concept relations, while simultaneously improving the expressivity of the model in terms of logic operations. Our empirical studies on well-known benchmarks and synthetic datasets demonstrate that these models have better accuracy, perform effective interventions and are highly interpretable.