Subspace-Configurable Networks
This addresses robustness issues for edge device deployments, but appears incremental as it builds on existing subspace and adaptation methods.
The paper tackles the problem of deep learning models lacking robustness to dynamic changes in sensed data on edge devices by training a parameterized subspace of configurable networks, resulting in a low-dimensional subspace with simple structure that enables high efficiency under limited storage and computing resources.
While the deployment of deep learning models on edge devices is increasing, these models often lack robustness when faced with dynamic changes in sensed data. This can be attributed to sensor drift, or variations in the data compared to what was used during offline training due to factors such as specific sensor placement or naturally changing sensing conditions. Hence, achieving the desired robustness necessitates the utilization of either an invariant architecture or specialized training approaches, like data augmentation techniques. Alternatively, input transformations can be treated as a domain shift problem, and solved by post-deployment model adaptation. In this paper, we train a parameterized subspace of configurable networks, where an optimal network for a particular parameter setting is part of this subspace. The obtained subspace is low-dimensional and has a surprisingly simple structure even for complex, non-invertible transformations of the input, leading to an exceptionally high efficiency of subspace-configurable networks (SCNs) when limited storage and computing resources are at stake.