Memory-Associated Differential Learning
This addresses the inefficiency of ignoring training data associations in machine learning, though it appears incremental as an extension of existing memory-based or differential methods.
The paper tackles the problem of conventional supervised learning wasting training data by proposing Memory-Associated Differential (MAD) Learning, a novel paradigm that memorizes training data and learns label differences and feature associations, achieving state-of-the-art performance on the ogbl-ddi dataset for link prediction.
Conventional Supervised Learning approaches focus on the mapping from input features to output labels. After training, the learnt models alone are adapted onto testing features to predict testing labels in isolation, with training data wasted and their associations ignored. To take full advantage of the vast number of training data and their associations, we propose a novel learning paradigm called Memory-Associated Differential (MAD) Learning. We first introduce an additional component called Memory to memorize all the training data. Then we learn the differences of labels as well as the associations of features in the combination of a differential equation and some sampling methods. Finally, in the evaluating phase, we predict unknown labels by inferencing from the memorized facts plus the learnt differences and associations in a geometrically meaningful manner. We gently build this theory in unary situations and apply it on Image Recognition, then extend it into Link Prediction as a binary situation, in which our method outperforms strong state-of-the-art baselines on ogbl-ddi dataset.