Beyond Occam's Razor in System Identification: Double-Descent when Modeling Dynamics
This work identifies a double-descent phenomenon in system identification, challenging traditional model complexity understanding for researchers and practitioners in control systems and machine learning.
This paper investigates whether the double-descent phenomenon, where model performance improves beyond the interpolation point with increased complexity, can be observed in dynamic system identification. The authors experimentally confirm this behavior using both synthetic and real-world datasets.
System identification aims to build models of dynamical systems from data. Traditionally, choosing the model requires the designer to balance between two goals of conflicting nature; the model must be rich enough to capture the system dynamics, but not so flexible that it learns spurious random effects from the dataset. It is typically observed that the model validation performance follows a U-shaped curve as the model complexity increases. Recent developments in machine learning and statistics, however, have observed situations where a "double-descent" curve subsumes this U-shaped model-performance curve. With a second decrease in performance occurring beyond the point where the model has reached the capacity of interpolating - i.e., (near) perfectly fitting - the training data. To the best of our knowledge, such phenomena have not been studied within the context of dynamic systems. The present paper aims to answer the question: "Can such a phenomenon also be observed when estimating parameters of dynamic systems?" We show that the answer is yes, verifying such behavior experimentally both for artificially generated and real-world datasets.