CARLS: Cross-platform Asynchronous Representation Learning System
This is an incremental improvement for researchers and practitioners needing to scale deep learning with additional knowledge, such as in graph neural networks.
The authors tackled the problem of scaling deep learning frameworks by proposing CARLS, a cross-platform asynchronous system that enables model trainers, knowledge makers, and knowledge banks to work together, resulting in efficient scaling for learning paradigms like semi-supervised learning.
In this work, we propose CARLS, a novel framework for augmenting the capacity of existing deep learning frameworks by enabling multiple components -- model trainers, knowledge makers and knowledge banks -- to concertedly work together in an asynchronous fashion across hardware platforms. The proposed CARLS is particularly suitable for learning paradigms where model training benefits from additional knowledge inferred or discovered during training, such as node embeddings for graph neural networks or reliable pseudo labels from model predictions. We also describe three learning paradigms -- semi-supervised learning, curriculum learning and multimodal learning -- as examples that can be scaled up efficiently by CARLS. One version of CARLS has been open-sourced and available for download at: https://github.com/tensorflow/neural-structured-learning/tree/master/research/carls