Negative binomial regression and inference using a pre-trained transformer
This provides an efficient solution for researchers analyzing over-dispersed count data in large screens, though it is incremental as it compares existing methods rather than introducing a fundamentally new approach.
The paper tackled the computational challenge of negative binomial regression for over-dispersed count data in large-scale comparisons by using a pre-trained transformer trained on synthetic data. The transformer was 20 times faster than maximum likelihood optimization with better accuracy, but method of moment estimates unexpectedly performed equally well in accuracy while being 1,000 times faster with better-calibrated tests.
Negative binomial regression is essential for analyzing over-dispersed count data in in comparative studies, but parameter estimation becomes computationally challenging in large screens requiring millions of comparisons. We investigate using a pre-trained transformer to produce estimates of negative binomial regression parameters from observed count data, trained through synthetic data generation to learn to invert the process of generating counts from parameters. The transformer method achieved better parameter accuracy than maximum likelihood optimization while being 20 times faster. However, comparisons unexpectedly revealed that method of moment estimates performed as well as maximum likelihood optimization in accuracy, while being 1,000 times faster and producing better-calibrated and more powerful tests, making it the most efficient solution for this application.