Sanghyun Lee

Sanghyun Lee, Thomas Wick, Mary F. Wheeler

In this work, we present numerical studies of fixed-stress iterative coupling for solving flow and geomechanics with propagating fractures in a porous medium. Specifically, fracture propagations are described by employing a phase-field approach. The extension to fixed-stress splitting to propagating phase-field fractures and systematic investigation of its properties are important enhancements to existing studies. Moreover, we provide an accurate computation of the fracture opening using level-set approaches and a subsequent finite element interpolation of the width. The latter enters as fracture permeability into the pressure diffraction problem which is crucial for fluid filled fractures. Our developments are substantiated with several numerical tests that include comparisons of computational cost for iterative coupling and nonlinear and linear iterations as well as convergence studies in space and time.

Sanghyun Lee, Mary F. Wheeler

We present a novel approach to the simulation of miscible displacement by employing adaptive enriched Galerkin finite element methods (EG) coupled with entropy residual stabilization for transport. In particular, numerical simulations of viscous fingering instabilities in heterogeneous porous media and Hele-Shaw cells are illustrated. EG is formulated by enriching the conforming continuous Galerkin finite element method (CG) with piecewise constant functions. The method provides locally and globally conservative fluxes, which is crucial for coupled flow and transport problems. Moreover, EG has fewer degrees of freedom in comparison with discontinuous Galerkin (DG) and an efficient flow solver has been derived which allows for higher order schemes. Dynamic adaptive mesh refinement is applied in order to save computational cost for large-scale three dimensional applications. In addition, entropy residual based stabilization for high order EG transport systems prevents any spurious oscillations. Numerical tests are presented to show the capabilities of EG applied to flow and transport.

Andrea Bonito, Jean-Luc Guermond, Sanghyun Lee

Bouncing jets are fascinating phenomenons occurring under certain conditions when a jet impinges on a free surface. This effect is observed when the fluid is Newtonian and the jet falls in a bath undergoing a solid motion. It occurs also for non-Newtonian fluids when the jets falls in a vessel at rest containing the same fluid. We investigate numerically the impact of the experimental setting and the rheological properties of the fluid on the onset of the bouncing phenomenon. Our investigations show that the occurrence of a thin lubricating layer of air separating the jet and the rest of the liquid is a key factor for the bouncing of the jet to happen. The numerical technique that is used consists of a projection method for the Navier-Stokes system coupled with a level set formulation for the representation of the interface. The space approximation is done with adaptive finite elements. Adaptive refinement is shown to be very important to capture the thin layer of air that is responsible for the bouncing.

Sanghyun Lee, Abner J. Salgado

We present a stability analysis for two different rotational pressure correction schemes with open and traction boundary conditions. First, we provide a stability analysis for a rotational version of the grad-div stabilized scheme of [A. Bonito, J.-L. Guermond, and S. Lee. Modified pressure-correction projection methods: Open boundary and variable time stepping. In Numerical Mathematics and Advanced Applications - ENUMATH 2013, volume 103 of Lecture Notes in Computational Science and Engineering, pages 623-631. Springer, 2015]. This scheme turns out to be unconditionally stable, provided the stabilization parameter is suitably chosen. We also establish a conditional stability result for the boundary correction scheme presented in [E. Bansch. A finite element pressure correction scheme for the Navier-Stokes equations with traction boundary condition. Comput. Methods Appl. Mech. Engrg., 279:198-211, 2014]. These results are shown by employing the equivalence between stabilized gauge Uzawa methods and rotational pressure correction schemes with traction boundary conditions.

Sanghyun Lee, Chunsan Hong, Seungryong Kim et al.

Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models for language modeling, yet the effective design of transformer architectures for MDMs remains underexplored. In this paper, we show that selectively looping the early-middle transformer layers significantly improves both training efficiency and model performance in MDMs. We call this approach LoopMDM(Looped Masked Diffusion Model), which brings two key benefits: looping layers at training-time yields a depth-scaling effect without adding parameters, while varying the number of loops at inference-time enables flexible compute scaling. Despite the simplicity, the results are striking: across multiple pre-training corpora, LoopMDM matches the performance of same-size MDMs with up to 3.3 fewer training FLOPs, while its final performance outperforms them on various reasoning benchmarks, including up to 8.5 points on GSM8K. It even surpasses deeper non-looped MDMs trained with comparable per-step compute, indicating that selective looping is more effective than naive depth scaling. Furthermore, LoopMDM can scale inference-time compute by increasing the number of loops. Adaptively adjusting the number of loops throughout the sampling process further yields additional gains in compute efficiency while maintaining performance. Lastly, with attention analysis, we provide evidence that looping is effective in MDMs by promoting interactions among masked positions. Our code and weights will be publicly released.

Kapil Chawla, Youngjoon Hong, Jae Yong Lee et al.

We introduce Discontinuous Galerkin Finite Element Operator Network (DG--FEONet), a data-free operator learning framework that combines the strengths of the discontinuous Galerkin (DG) method with neural networks to solve parametric partial differential equations (PDEs) with discontinuous coefficients and non-smooth solutions. Unlike traditional operator learning models such as DeepONet and Fourier Neural Operator, which require large paired datasets and often struggle near sharp features, our approach minimizes the residual of a DG-based weak formulation using the Symmetric Interior Penalty Galerkin (SIPG) scheme. DG-FEONet predicts element-wise solution coefficients via a neural network, enabling data-free training without the need for precomputed input-output pairs. We provide theoretical justification through convergence analysis and validate the model's performance on a series of one- and two-dimensional PDE problems, demonstrating accurate recovery of discontinuities, strong generalization across parameter space, and reliable convergence rates. Our results highlight the potential of combining local discretization schemes with machine learning to achieve robust, singularity-aware operator approximation in challenging PDE settings.

Kapil Chawla, Sanghyun Lee, Yeonjong Shin

Material properties such as permeability fields in heterogeneous porous media are often represented as discontinuous, piecewise constant data tied to a given spatial discretization. Such representations are inherently mesh-dependent, requiring interpolation or projection whenever they are transferred to a different discretization. In this work, we develop \emph{Parallel and Adaptive Mesh-Free Approximation (PAM)}, a mesh-independent framework that approximates discontinuous data by a continuous, closed-form function. The resulting approximation can be evaluated consistently across different geometries and numerical discretizations, while preserving sharp interface features. The proposed PAM framework employs radial basis functions (RBFs) to construct continuous approximations of discontinuous data. To accurately capture discontinuities, we incorporate Shepard-normalization, which stabilizes the approximation near sharp interfaces. The coefficients of the RBF expansion are determined via sparse regression, enabling automatic selection of the most relevant basis functions and promoting robust representations. In addition, we develop a novel adaptive refinement approach which further enriches the approximation in regions of rapid spatial variation. We provide a theoretical analysis showing that the proposed normalized RBF framework achieves arbitrarily small $L^1$ error in approximating discontinuous step functions. To enhance computational efficiency, the domain is partitioned into subdomains, and the reconstruction problem is solved independently on each subdomain in parallel. Numerical experiments demonstrate the accuracy, adaptivity, and scalability of the proposed method, including applications to challenging heterogeneous permeability fields.

Chunsan Hong, Sanghyun Lee, Chieh-Hsin Lai et al.

Masked diffusion models (MDMs) have emerged as a promising alternative to autoregressive models (ARMs) for language modeling. However, MDMs are known to learn substantially more slowly than ARMs, which may become problematic when scaling MDMs to larger models. Therefore, we ask the following question: how can we accelerate standard MDM training while maintaining its final performance? To this end, we first provide a detailed analysis of why MDM training is slow. We find that the main factor is the locality bias of language: the predictive information for a token is concentrated in nearby positions. We further investigate how this bias slows learning and suggest a simple yet effective remedy: bell-shaped time sampling as a training strategy. Notably, MDMs trained with our training recipe reach the same validation negative log-likelihood (NLL) up to $\sim4\times$ faster than standard training on One Billion Word Benchmark (LM1B). We also show faster improvements in generative perplexity, zero-shot perplexity, and downstream task performance on various benchmarks.

Sanghyun Lee, Sunwoo Kim, Seungryong Kim et al.

Test-time scaling through reward-guided generation remains largely unexplored for discrete diffusion models despite its potential as a promising alternative. In this work, we introduce Iterative Reward-Guided Refinement (IterRef), a novel test-time scaling method tailored to discrete diffusion that leverages reward-guided noising-denoising transitions to progressively refine misaligned intermediate states. We formalize this process within a Multiple-Try Metropolis (MTM) framework, proving convergence to the reward-aligned distribution. Unlike prior methods that assume the current state is already aligned with the reward distribution and only guide the subsequent transition, our approach explicitly refines each state in situ, progressively steering it toward the optimal intermediate distribution. Across both text and image domains, we evaluate IterRef on diverse discrete diffusion models and observe consistent improvements in reward-guided generation quality. In particular, IterRef achieves striking gains under low compute budgets, far surpassing prior state-of-the-art baselines.

Seongju Jang, Francis Baek, SangHyun Lee

Due to the ever-changing nature of construction, many tasks on sites occur in an improvisational manner. Existing mobile construction robot studies remain limited in addressing improvisational tasks, where task-required locations, timing of task occurrence, and contextual information required for task execution are not known in advance. We propose an agent that understands improvisational tasks given in natural language, identifies the task-required location, and positions itself. The agent's functionality was decomposed into three Large Multimodal Model (LMM) modules operating in parallel, enabling the application of LMMs for task interpretation and breakdown, construction drawing-based navigation, and visual reasoning to identify non-predefined task-required locations. The agent was implemented with a quadruped robot and achieved a 92.2% success rate for identifying and positioning at task-required locations across three tests designed to assess improvisational task handling. This study enables mobile construction robots to perform non-predefined tasks autonomously.

Donghoon Ahn, Jiwon Kang, Sanghyun Lee et al.

Diffusion models excel in generating high-quality images. However, current diffusion models struggle to produce reliable images without guidance methods, such as classifier-free guidance (CFG). Are guidance methods truly necessary? Observing that noise obtained via diffusion inversion can reconstruct high-quality images without guidance, we focus on the initial noise of the denoising pipeline. By mapping Gaussian noise to `guidance-free noise', we uncover that small low-magnitude low-frequency components significantly enhance the denoising process, removing the need for guidance and thus improving both inference throughput and memory. Expanding on this, we propose \ours, a novel method that replaces guidance methods with a single refinement of the initial noise. This refined noise enables high-quality image generation without guidance, within the same diffusion pipeline. Our noise-refining model leverages efficient noise-space learning, achieving rapid convergence and strong performance with just 50K text-image pairs. We validate its effectiveness across diverse metrics and analyze how refined noise can eliminate the need for guidance. See our project page: https://cvlab-kaist.github.io/NoiseRefine/.

Sanghyun Lee, Seungryong Kim, Jongho Park et al.

Masked Diffusion Models (MDMs) as language models generate by iteratively unmasking tokens, yet their performance crucially depends on the inference time order of unmasking. Prevailing heuristics, such as confidence based sampling, are myopic: they optimize locally, fail to leverage extra test-time compute, and let early decoding mistakes cascade. We propose Lookahead Unmasking (LookUM), which addresses these concerns by reformulating sampling as path selection over all possible unmasking orders without the need for an external reward model. Our framework couples (i) a path generator that proposes paths by sampling from pools of unmasking sets with (ii) a verifier that computes the uncertainty of the proposed paths and performs importance sampling to subsequently select the final paths. Empirically, erroneous unmasking measurably inflates sequence level uncertainty, and our method exploits this to avoid error-prone trajectories. We validate our framework across six benchmarks, such as mathematics, planning, and coding, and demonstrate consistent performance improvements. LookUM requires only two to three paths to achieve peak performance, demonstrating remarkably efficient path selection. The consistent improvements on both LLaDA and post-trained LLaDA 1.5 are particularly striking: base LLaDA with LookUM rivals the performance of RL-tuned LLaDA 1.5, while LookUM further enhances LLaDA 1.5 itself showing that uncertainty based verification provides orthogonal benefits to reinforcement learning and underscoring the versatility of our framework. Code will be publicly released.

Hyun-Geun Shin, Yi-Yung Yang, Sanghyun Lee

This paper introduces a novel a posteriori error estimation framework for the enriched Galerkin (EG) finite element method applied to linear parabolic equations. While the EG method has been recognized for its local conservation property and computational efficiency compared to discontinuous Galerkin methods, its mathematical analysis in the context of a posteriori error estimation for parabolic problems remains unexplored. In this work, we prove reliability and efficiency using the residual-based approach. Furthermore, we integrate these error estimators into an adaptive mesh refinement strategy, demonstrating their effectiveness in achieving efficient and reliable error control through several numerical examples. The proposed approach provides a significant advancement in the mathematical foundation and practical applicability of the EG method for time-dependent problems.

Chunsan Hong, Sanghyun Lee, Jong Chul Ye

Masked diffusion models (MDMs) are a potential alternative to autoregressive models (ARMs) for language generation, but generation quality depends critically on the generation order. Prior work either hard-codes an ordering (e.g., blockwise left-to-right) or learns an ordering policy for a pretrained MDM, which incurs extra cost and can yield suboptimal solutions due to the two-stage optimization. Motivated by this, we propose order-expressive masked diffusion model (OeMDM) for a broad class of diffusion generative processes with various generation orders, enabling the interpretation of MDM, ARM, and block diffusion in a single framework. Furthermore, building on OeMDM, we introduce learnable-order masked diffusion model (LoMDM), which jointly learns the generation ordering and diffusion backbone through a single objective from scratch, enabling the diffusion model to generate text in context-dependent ordering. Empirically, we confirm that LoMDM outperforms various discrete diffusion models across multiple language modeling benchmarks.

Yi-Yung Yang, Sanghyun Lee, Dmitri Kuzmin

Enhanced geothermal systems (EGS) involve strongly coupled, advection-dominated flow and heat transfer in fractured porous media. Conventional models typically assume local thermal equilibrium with a single effective fluid temperature or, at best, an averaged pore-fluid temperature, so the thermal evolution of injected cold fluid is only inferred indirectly. In this work, we develop a local thermal non-equilibrium (LTNE) model that explicitly resolves the temperature of injected fluid as it moves through the reservoir and exchanges heat with the hot rock and resident fluid. The key ingredient is a concentration variable that tracks the injected fluid and induces a three-way LTNE coupling among rock, resident-fluid, and injected-fluid temperatures. This framework distinguishes, at the continuum scale, how newly injected fluid parcels are heated by conductive and convective exchange, and predicts production-well temperatures without relying on bulk averages. To discretize the resulting nonlinear, advection-dominated system, we employ an enriched Galerkin (EG) finite element method for Darcy flow, temperature, and concentration, providing local mass conservation with relatively few degrees of freedom. We further design a flux-corrected transport (FCT) strategy for the EG concentration and temperature equations to enforce a discrete maximum principle and suppress nonphysical oscillations while preserving local conservation. Time integration uses an IMPES-type splitting combined with a strong-stability-preserving Runge--Kutta scheme. Numerical experiments for fractured EGS problems show that the proposed LTNE--EG--FCT framework captures injected-fluid heating paths and thermal breakthrough behavior not resolved by standard single-temperature or averaged LTNE models.

Donghoon Ahn, Jiwon Kang, Sanghyun Lee et al.

Recent guidance methods in diffusion models steer reverse sampling by perturbing the model to construct an implicit weak model and guide generation away from it. Among these approaches, attention perturbation has demonstrated strong empirical performance in unconditional scenarios where classifier-free guidance is not applicable. However, existing attention perturbation methods lack principled approaches for determining where perturbations should be applied, particularly in Diffusion Transformer (DiT) architectures where quality-relevant computations are distributed across layers. In this paper, we investigate the granularity of attention perturbations, ranging from the layer level down to individual attention heads, and discover that specific heads govern distinct visual concepts such as structure, style, and texture quality. Building on this insight, we propose "HeadHunter", a systematic framework for iteratively selecting attention heads that align with user-centric objectives, enabling fine-grained control over generation quality and visual attributes. In addition, we introduce SoftPAG, which linearly interpolates each selected head's attention map toward an identity matrix, providing a continuous knob to tune perturbation strength and suppress artifacts. Our approach not only mitigates the oversmoothing issues of existing layer-level perturbation but also enables targeted manipulation of specific visual styles through compositional head selection. We validate our method on modern large-scale DiT-based text-to-image models including Stable Diffusion 3 and FLUX.1, demonstrating superior performance in both general quality enhancement and style-specific guidance. Our work provides the first head-level analysis of attention perturbation in diffusion models, uncovering interpretable specialization within attention layers and enabling practical design of effective perturbation strategies.

Sanghyun Lee, Chanyoung Park

The real-world traffic networks undergo expansion through the installation of new sensors, implying that the traffic patterns continually evolve over time. Incrementally training a model on the newly added sensors would make the model forget the past knowledge, i.e., catastrophic forgetting, while retraining the model on the entire network to capture these changes is highly inefficient. To address these challenges, we propose a novel Traffic Forecasting Mixture of Experts (TFMoE) for traffic forecasting under evolving networks. The main idea is to segment the traffic flow into multiple homogeneous groups, and assign an expert model responsible for a specific group. This allows each expert model to concentrate on learning and adapting to a specific set of patterns, while minimizing interference between the experts during training, thereby preventing the dilution or replacement of prior knowledge, which is a major cause of catastrophic forgetting. Through extensive experiments on a real-world long-term streaming network dataset, PEMSD3-Stream, we demonstrate the effectiveness and efficiency of TFMoE. Our results showcase superior performance and resilience in the face of catastrophic forgetting, underscoring the effectiveness of our approach in dealing with continual learning for traffic flow forecasting in long-term streaming networks.

Sanghyun Lee, Chanyoung Park

Accurate traffic flow forecasting is a crucial research topic in transportation management. However, it is a challenging problem due to rapidly changing traffic conditions, high nonlinearity of traffic flow, and complex spatial and temporal correlations of road networks. Most existing studies either try to capture the spatial dependencies between roads using the same semantic graph over different time steps, or assume all sensors on the roads are equally likely to be connected regardless of the distance between them. However, we observe that the spatial dependencies between roads indeed change over time, and two distant roads are not likely to be helpful to each other when predicting the traffic flow, both of which limit the performance of existing studies. In this paper, we propose Temporal Graph Learning Recurrent Neural Network (TGLRN) to address these problems. More precisely, to effectively model the nature of time series, we leverage Recurrent Neural Networks (RNNs) to dynamically construct a graph at each time step, thereby capturing the time-evolving spatial dependencies between roads (i.e., microscopic view). Simultaneously, we provide the Adaptive Structure Information to the model, ensuring that close and consecutive sensors are considered to be more important for predicting the traffic flow (i.e., macroscopic view). Furthermore, to endow TGLRN with robustness, we introduce an edge sampling strategy when constructing the graph at each time step, which eventually leads to further improvements on the model performance. Experimental results on four commonly used real-world benchmark datasets show the effectiveness of TGLRN.

Sanghyun Lee, Yeonjong Shin

We present a novel training method for deep operator networks (DeepONets), one of the most popular neural network models for operators. DeepONets are constructed by two sub-networks, namely the branch and trunk networks. Typically, the two sub-networks are trained simultaneously, which amounts to solving a complex optimization problem in a high dimensional space. In addition, the nonconvex and nonlinear nature makes training very challenging. To tackle such a challenge, we propose a two-step training method that trains the trunk network first and then sequentially trains the branch network. The core mechanism is motivated by the divide-and-conquer paradigm and is the decomposition of the entire complex training task into two subtasks with reduced complexity. Therein the Gram-Schmidt orthonormalization process is introduced which significantly improves stability and generalization ability. On the theoretical side, we establish a generalization error estimate in terms of the number of training data, the width of DeepONets, and the number of input and output sensors. Numerical examples are presented to demonstrate the effectiveness of the two-step training method, including Darcy flow in heterogeneous porous media.

Byungjoo Choi, Gaang Lee, Houtan Jebelli et al.

The construction industry has demonstrated a high frequency and severity of accidents. Construction accidents are the result of the interaction between unsafe work conditions and workers unsafe behaviors. Given this relation, perceived risk is determined by an individual response to a potential work hazard during the work. As such, risk perception is critical to understand workers unsafe behaviors. Established methods of assessing workers perceived risk have mainly relied on surveys and interviews. However, these post-hoc methods, which are limited to monitoring dynamic changes in risk perception and conducting surveys at a construction site, may prove cumbersome to workers. Additionally, these methods frequently suffer from self-reported bias. To overcome the limitations of previous subjective measures, this study aims to develop a framework for the objective and continuous prediction of construction workers perceived risk using physiological signals [e.g., electrodermal activity (EDA)] acquired from workers wristband-type biosensors. To achieve this objective, physiological signals were collected from eight construction workers while they performed regular tasks in the field. Various filtering methods were applied to exclude noises recorded in the signal and to extract various features of the signals as workers experienced different risk levels. Then, a supervised machine-learning model was trained to explore the applicability of the collected physiological signals for the prediction of risk perception. The results showed that features based on EDA data collected from wristbands are feasible and useful to the process of continuously monitoring workers perceived risk during ongoing work. This study contributes to an in-depth understanding of construction workers perceived risk by developing a noninvasive means of continuously monitoring workers perceived risk.

Sanghyun Lee, Mary F. Wheeler

In this paper, we propose an enriched Galerkin (EG) approximation for a two-phase pressure saturation system with capillary pressure in heterogeneous porous media. The EG methods are locally conservative, have fewer degrees of freedom compared to discontinuous Galerkin (DG), and have an efficient pressure solver. To avoid non-physical oscillations, an entropy viscosity stabilization method is employed for high order saturation approximations. Entropy residuals are applied for dynamic mesh adaptivity to reduce the computational cost for larger computational domains. The iterative and sequential IMplicit Pressure and Explicit Saturation (IMPES) algorithms are treated in time. Numerical examples with different relative permeabilities and capillary pressures are included to verify and to demonstrate the capabilities of EG.