Designing with Non-Finite Output Dimension via Fourier Coefficients of Neural Waveforms
This addresses a constraint in neural network design for applications requiring variable complexity, though it is incremental as it only shows feasibility on a toy problem.
The paper tackles the limitation of fixed output dimensions in deep learning for design tasks by introducing a method to produce outputs of non-finite dimension using Fourier coefficients of neural waveforms, and demonstrates that neural networks can learn in this setting on a toy problem.
Ordinary Deep Learning models require having the dimension of their outputs determined by a human practitioner prior to training and operation. For design tasks, this places a hard limit on the maximum complexity of any designs produced by a neural network, which is disadvantageous if a greater allowance for complexity would result in better designs. In this paper, we introduce a methodology for taking outputs of non-finite dimension from neural networks, by learning a "neural waveform," and then taking as outputs the coefficients of its Fourier series representation. We then present experimental evidence that neural networks can learn in this setting on a toy problem.