Universal Inference Meets Random Projections: A Scalable Test for Log-concavity
This provides the first valid test for log-concavity in any dimension, addressing a gap in statistical modeling for fields like economics and reliability theory, though it is incremental as it builds on existing universal inference and MLE methods.
The paper tackles the problem of testing whether a data distribution is log-concave, a shape constraint with applications in economics and survival modeling, by proposing a provably valid finite-sample test using universal inference and random projections, achieving high power and computational efficiency.
Shape constraints yield flexible middle grounds between fully nonparametric and fully parametric approaches to modeling distributions of data. The specific assumption of log-concavity is motivated by applications across economics, survival modeling, and reliability theory. However, there do not currently exist valid tests for whether the underlying density of given data is log-concave. The recent universal inference methodology provides a valid test. The universal test relies on maximum likelihood estimation (MLE), and efficient methods already exist for finding the log-concave MLE. This yields the first test of log-concavity that is provably valid in finite samples in any dimension, for which we also establish asymptotic consistency results. Empirically, we find that a random projections approach that converts the d-dimensional testing problem into many one-dimensional problems can yield high power, leading to a simple procedure that is statistically and computationally efficient.