Jiang Zhu

Jiang Zhu, Xiangming Meng, Xupeng Lei et al.

We consider the problem of recovering an unknown signal ${\mathbf x}\in {\mathbb R}^n$ from general nonlinear measurements obtained through a generalized linear model (GLM), i.e., ${\mathbf y}= f\left({\mathbf A}{\mathbf x}+{\mathbf w}\right)$, where $f(\cdot)$ is a componentwise nonlinear function. Based on the unitary transform approximate message passing (UAMP) and expectation propagation, a unitary transform based generalized approximate message passing (GUAMP) algorithm is proposed for general measurement matrices $\bf{A}$, in particular highly correlated matrices. Experimental results on quantized compressed sensing demonstrate that the proposed GUAMP significantly outperforms state-of-the-art GAMP and GVAMP under correlated matrices $\bf{A}$.

Andrew Cohen, Weiping Dou, Jiang Zhu et al.

Linear Partial Differential Equations (PDEs) govern the spatial-temporal dynamics of physical systems that are essential to building modern technology. When working with linear PDEs, designing a physical system for a specific outcome is difficult and costly due to slow and expensive explicit simulation of PDEs and the highly nonlinear relationship between a system's configuration and its behavior. In this work, we prove a parametric form that certain physical quantities in the Fourier domain must obey in linear PDEs, named the CZP (Constant-Zeros-Poles) framework. Applying CZP to antenna design, an industrial application using linear PDEs (i.e., Maxwell's equations), we derive a sample-efficient parametric surrogate model that directly predicts its scattering coefficients without explicit numerical PDE simulation. Combined with a novel image-based antenna representation and an attention-based neural network architecture, CZP outperforms baselines by 10% to 25% in terms of test loss and also is able to find 2D antenna designs verifiable by commercial software with $33\%$ greater success than baselines, when coupled with sequential search techniques like reinforcement learning.

Shihan Dou, Enyu Zhou, Yan Liu et al.

Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks. Increasing instruction data substantially is a direct solution to align the model with a broader range of downstream tasks or notably improve its performance on a specific task. However, we find that large-scale increases in instruction data can damage the world knowledge previously stored in LLMs. To address this challenge, we propose LoRAMoE, a novelty framework that introduces several low-rank adapters (LoRA) and integrates them by using a router network, like a plugin version of Mixture of Experts (MoE). It freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge-edge forgetting. Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM.

Yang Zhong, Peter Renner, Weiping Dou et al.

To facilitate the antenna design with the aid of computer, one of the practices in consumer electronic industry is to model and optimize antenna performances with a simplified antenna geometric scheme. Traditional antenna modeling requires profound prior knowledge of electromagnetics in order to achieve a good design which satisfies the performance specifications from both antenna and product designs. The ease of handling multidimensional optimization problems and the less dependence on domain knowledge and experience are the key to achieve the popularity of simulation driven antenna design and optimization for the industry. In this paper, we introduce a flexible geometric scheme with the concept of mesh network that can form any arbitrary shape by connecting different nodes. For such problems with high dimensional parameters, we propose a machine learning based generative method to assist the searching of optimal solutions. It consists of discriminators and generators. The discriminators are used to predict the performance of geometric models, and the generators to create new candidates that will pass the discriminators. Moreover, an evolutionary criterion approach is proposed for further improving the efficiency of our method. Finally, not only optimal solutions can be found, but also the well trained generators can be used to automate future antenna design and optimization. For a dual resonance antenna design with wide bandwidth, our proposed method is in par with Trust Region Framework and much better than the other mature machine learning algorithms including the widely used Genetic Algorithm and Particle Swarm Optimization. When there is no wide bandwidth requirement, it is better than Trust Region Framework.