SML:Enhance the Network Smoothness with Skip Meta Logit for CTR Prediction
This work addresses CTR prediction for online advertising systems, offering an incremental improvement that can be easily integrated into existing models.
The paper tackled the problem of improving click-through rate (CTR) prediction by proposing Skip Meta Logit (SML), which enhances network smoothness using skip connections and meta tanh normalization, resulting in incremental performance boosts on SOTA models across two real-world datasets and delivering offline accuracy and online business gains at TikTok.
In light of the smoothness property brought by skip connections in ResNet, this paper proposed the Skip Logit to introduce the skip connection mechanism that fits arbitrary DNN dimensions and embraces similar properties to ResNet. Meta Tanh Normalization (MTN) is designed to learn variance information and stabilize the training process. With these delicate designs, our Skip Meta Logit (SML) brought incremental boosts to the performance of extensive SOTA ctr prediction models on two real-world datasets. In the meantime, we prove that the optimization landscape of arbitrarily deep skip logit networks has no spurious local optima. Finally, SML can be easily added to building blocks and has delivered offline accuracy and online business metrics gains on app ads learning to rank systems at TikTok.