Eric King‐wah Chu

Zhen-Chen Guo, Eric King-Wah Chu, Wen-Wei Lin

The discretized Bethe-Salpeter eigenvalue problem arises in the Green's function evaluation in many body physics and quantum chemistry. Discretization leads to a matrix eigenvalue problem for $H \in \mathbb{C}^{2n\times 2n}$ with a Hamiltonian-like structure. After an appropriate transformation of $H$ to a standard symplectic form, the structure-preserving doubling algorithm, originally for algebraic Riccati equations, is extended for the discretized Bethe-Salpeter eigenvalue problem. Potential breakdowns of the algorithm, due to the ill condition or singularity of certain matrices, can be avoided with a double-Cayley transform or a three-recursion remedy. A detailed convergence analysis is conducted for the proposed algorithm, especially on the benign effects of the double-Cayley transform. Numerical results are presented to demonstrate the efficiency and structure-preserving nature of the algorithm.

Huai-An Diao, Hong Yan, Eric King-wah Chu

In this paper, we adopt a componentwise perturbation analysis for $\star$-Sylvester equations. Based on the small condition estimation (SCE), we devise the algorithms to estimate normwise, mixed and componentwise condition numbers for $\star$-Sylvester equations. We also define a componentwise backward error with a sharp and easily computable bound. Numerical examples illustrate that our algorithm under componentwise perturbations produces reliable estimates, and the new derived computable bound for the componentwise backward error is sharp and reliable for well conditioned and moderate ill-conditioned $\star$-Sylvester equations under large or small perturbations.

Rishabh Agarwal, Avi Singh, Lei M. Zhang et al. · mila

Large language models (LLMs) excel at few-shot in-context learning (ICL) -- learning from a few examples provided in context at inference, without any weight updates. Newly expanded context windows allow us to investigate ICL with hundreds or thousands of examples -- the many-shot regime. Going from few-shot to many-shot, we observe significant performance gains across a wide variety of generative and discriminative tasks. While promising, many-shot ICL can be bottlenecked by the available amount of human-generated examples. To mitigate this limitation, we explore two new settings: Reinforced and Unsupervised ICL. Reinforced ICL uses model-generated chain-of-thought rationales in place of human examples. Unsupervised ICL removes rationales from the prompt altogether, and prompts the model only with domain-specific questions. We find that both Reinforced and Unsupervised ICL can be quite effective in the many-shot regime, particularly on complex reasoning tasks. Finally, we demonstrate that, unlike few-shot learning, many-shot learning is effective at overriding pretraining biases, can learn high-dimensional functions with numerical inputs, and performs comparably to fine-tuning. We also find that inference cost increases linearly in the many-shot regime, and frontier LLMs benefit from many-shot ICL to varying degrees. Our analysis also reveals the limitations of next-token prediction loss as an indicator of downstream ICL performance.