Wei Yang

Yunqing Huang, Huayi Wei, Wei Yang et al.

We design and numerically validate a recovery based linear finite element method for solving the biharmonic equation. The main idea is to replace the gradient operator $\nabla$ on linear finite element space by $G(\nabla)$ in the weak formulation of the biharmonic equation, where $G$ is the recovery operator which recovers the piecewise constant function into the linear finite element space. By operator $G$, Laplace operator $Δ$ is replaced by $\nabla\cdot G(\nabla)$. Furthermore the boundary condition on normal derivative $\nabla u\cdot \pmb{n}$ is treated by the boundary penalty method. The explicit matrix expression of the proposed method is also introduced. Numerical examples on uniform and adaptive meshes are presented to illustrate the correctness and effectiveness of the proposed method.

Yufei Li, Zexin Li, Wei Yang et al.

Recent advancements in language models (LMs) have gained substantial attentions on their capability to generate human-like responses. Though exhibiting a promising future for various applications such as conversation AI, these LMs face deployment challenges on various devices due to their extreme computational cost and unpredictable inference latency. Such varied inference latency, identified as a consequence of uncertainty intrinsic to the nature of language, can lead to computational inefficiency and degrade the overall performance of LMs, especially under high-traffic workloads. Unfortunately, the bandwidth of these uncertainty sources is extensive, complicating the prediction of latency and the effects emanating from such uncertainties. To understand and mitigate the impact of uncertainty on real-time response-demanding systems, we take the first step to comprehend, quantify and optimize these uncertainty-induced latency performance variations in LMs. Specifically, we present RT-LM, an uncertainty-aware resource management ecosystem for real-time inference of LMs. RT-LM innovatively quantifies how specific input uncertainties, adversely affect latency, often leading to an increased output length. Exploiting these insights, we devise a lightweight yet effective method to dynamically correlate input text uncertainties with output length at runtime. Utilizing this quantification as a latency heuristic, we integrate the uncertainty information into a system-level scheduler which explores several uncertainty-induced optimization opportunities, including uncertainty-aware prioritization, dynamic consolidation, and strategic CPU offloading. Quantitative experiments across five state-of-the-art LMs on two hardware platforms demonstrates that RT-LM can significantly reduce the average response time and improve throughput while incurring a rather small runtime overhead.