Reducing numerical precision preserves classification accuracy in Mondrian Forests
This addresses memory constraints for low-resource platforms like connected objects, but it is incremental as it applies an existing precision reduction technique to a specific method.
The study tackled the high memory consumption of Mondrian Forests for data stream classification by reducing floating-point precision from 64 to 8 bits, finding no significant difference in F1 score on human activity recognition datasets and sometimes improved performance due to regularization.
Mondrian Forests are a powerful data stream classification method, but their large memory footprint makes them ill-suited for low-resource platforms such as connected objects. We explored using reduced-precision floating-point representations to lower memory consumption and evaluated its effect on classification performance. We applied the Mondrian Forest implementation provided by OrpailleCC, a C++ collection of data stream algorithms, to two canonical datasets in human activity recognition: Recofit and Banos \emph{et al}. Results show that the precision of floating-point values used by tree nodes can be reduced from 64 bits to 8 bits with no significant difference in F1 score. In some cases, reduced precision was shown to improve classification performance, presumably due to its regularization effect. We conclude that numerical precision is a relevant hyperparameter in the Mondrian Forest, and that commonly-used double precision values may not be necessary for optimal performance. Future work will evaluate the generalizability of these findings to other data stream classifiers.