Tell Me Something New: A New Framework for Asynchronous Parallel Learning
This addresses the need for efficient and resilient parallel learning systems, particularly for large-scale machine learning tasks, though it appears incremental as it builds on existing parallel methods.
The paper tackles the problem of parallel computation in machine learning by introducing the 'Tell Me Something New' framework, which enables asynchronous learning without synchronization and achieves a 10x speedup over XGBoost and LightGBM on splice-site prediction.
We present a novel approach for parallel computation in the context of machine learning that we call "Tell Me Something New" (TMSN). This approach involves a set of independent workers that use broadcast to update each other when they observe "something new". TMSN does not require synchronization or a head node and is highly resilient against failing machines or laggards. We demonstrate the utility of TMSN by applying it to learning boosted trees. We show that our implementation is 10 times faster than XGBoost and LightGBM on the splice-site prediction problem.