FLInt: Exploiting Floating Point Enabled Integer Arithmetic for Efficient Random Forest Inference
This work addresses performance bottlenecks for random forest inference on embedded and general computing systems, offering a practical speedup, though it is incremental as it optimizes an existing method without altering the core algorithm.
The paper tackles the inefficiency of floating-point operations in random forest inference on systems lacking hardware support by introducing FLInt, a method that replaces floating-point comparisons with integer and logic operations while preserving model accuracy. Experimental results show execution time reductions of up to approximately 30% on X86 and ARMv8 systems.
In many machine learning applications, e.g., tree-based ensembles, floating point numbers are extensively utilized due to their expressiveness. Nowadays performing data analysis on embedded devices from dynamic data masses becomes available, but such systems often lack hardware capabilities to process floating point numbers, introducing large overheads for their processing. Even if such hardware is present in general computing systems, using integer operations instead of floating point operations promises to reduce operation overheads and improve the performance. In this paper, we provide \mdname, a full precision floating point comparison for random forests, by only using integer and logic operations. To ensure the same functionality preserves, we formally prove the correctness of this comparison. Since random forests only require comparison of floating point numbers during inference, we implement \mdname~in low level realizations and therefore eliminate the need for floating point hardware entirely, by keeping the model accuracy unchanged. The usage of \mdname~basically boils down to a one-by-one replacement of conditions: For instance, a comparison statement in C: if(pX[3]<=(float)10.074347) becomes if((*(((int*)(pX))+3))<=((int)(0x41213087))). Experimental evaluation on X86 and ARMv8 desktop and server class systems shows that the execution time can be reduced by up to $\approx 30\%$ with our novel approach.