Shu Li

Peihao Wang, Shenghao Yang, Shu Li et al.

Set representation has become ubiquitous in deep learning for modeling the inductive bias of neural networks that are insensitive to the input order. DeepSets is the most widely used neural network architecture for set representation. It involves embedding each set element into a latent space with dimension $L$, followed by a sum pooling to obtain a whole-set embedding, and finally mapping the whole-set embedding to the output. In this work, we investigate the impact of the dimension $L$ on the expressive power of DeepSets. Previous analyses either oversimplified high-dimensional features to be one-dimensional features or were limited to analytic activations, thereby diverging from practical use or resulting in $L$ that grows exponentially with the set size $N$ and feature dimension $D$. To investigate the minimal value of $L$ that achieves sufficient expressive power, we present two set-element embedding layers: (a) linear + power activation (LP) and (b) linear + exponential activations (LE). We demonstrate that $L$ being poly$(N, D)$ is sufficient for set representation using both embedding layers. We also provide a lower bound of $L$ for the LP embedding layer. Furthermore, we extend our results to permutation-equivariant set functions and the complex field.

Yingpin Chen, Zhenming Peng, Ali Gholami et al.

Time-frequency analysis has been applied successfully in many fields. However, the traditional methods, like short time Fourier transform and Cohen distribution, suffer from the low resolution or the interference of the cross terms. To solve these issues, we put forward a new sparse time-frequency analysis model by using the Lp-quasinorm constraint, which is capable of fitting the sparsity prior knowledge in the frequency domain. In the proposed model, we regard the short time truncated data as the observation of sparse representation and design a dictionary matrix, which builds up the relationship between the short time measurement and the sparse spectrum. Based on the relationship and the Lp-quasinorm feasible domain, the proposed model is established. The alternating direction method of multipliers (ADMM) is adopted to solve the proposed model. Experiments are then conducted on several theoretical signals and applied to the seismic signal spectrum decomposition, indicating that the proposed method is able to obtain a higher time-frequency distribution than state-of-the-art time-frequency methods. Thus, the proposed method is of great importance to reservoir exploration.