Comparing Differentiable Logics for Learning Systems: A Research Preview
This work addresses the problem of making ML models self-correcting and constraint-satisfying for autonomous systems, but it is incremental as it builds on existing literature.
The paper compares differentiable logics for integrating logical constraints into machine learning systems to ensure correctness and safety, finding that these methods introduce a hyperparameter that is difficult to tune and significantly affects performance.
Extensive research on formal verification of machine learning (ML) systems indicates that learning from data alone often fails to capture underlying background knowledge. A variety of verifiers have been developed to ensure that a machine-learnt model satisfies correctness and safety properties, however, these verifiers typically assume a trained network with fixed weights. ML-enabled autonomous systems are required to not only detect incorrect predictions, but should also possess the ability to self-correct, continuously improving and adapting. A promising approach for creating ML models that inherently satisfy constraints is to encode background knowledge as logical constraints that guide the learning process via so-called differentiable logics. In this research preview, we compare and evaluate various logics from the literature in weakly-supervised contexts, presenting our findings and highlighting open problems for future work. Our experimental results are broadly consistent with results reported previously in literature; however, learning with differentiable logics introduces a new hyperparameter that is difficult to tune and has significant influence on the effectiveness of the logics.