Dongil Shin

Gyuwon Park, DongIl Shin, SolGil Oh et al.

The rapid evolution of Large Language Models (LLMs) has significantly impacted the field of natural language processing, but their growing complexity raises concerns about resource usage and transparency. Addressing these challenges, we participated in the NeurIPS LLM Efficiency Challenge, aiming to fine-tune a foundation model within stringent constraints. Our focus was the LLaMa2 70 billion model, optimized on a single A100 40GB GPU within a 24-hour limit. Our methodology hinged on a custom dataset, carefully assembled from diverse open-source resources and benchmark tests, aligned with the challenge's open-source ethos. Our approach leveraged Quantized-Low Rank Adaptation (QLoRA) Fine tuning, integrated with advanced attention mechanisms like Flash Attention 2. We experimented with various configurations of the LoRA technique, optimizing the balance between computational efficiency and model accuracy. Our fine-tuning strategy was underpinned by the creation and iterative testing of multiple dataset compositions, leading to the selection of a version that demonstrated robust performance across diverse tasks and benchmarks. The culmination of our efforts was an efficiently fine-tuned LLaMa2 70B model that operated within the constraints of a single GPU, showcasing not only a significant reduction in resource utilization but also high accuracy across a range of QA benchmarks. Our study serves as a testament to the feasibility of optimizing large-scale models in resource-constrained environments, emphasizing the potential of LLMs in real-world applications.

Andreas E. Robertson, Samuel B. Inman, Ashley T. Lenau et al.

Aleatoric uncertainties - irremovable variability in microstructure morphology, constituent behavior, and processing conditions - pose a major challenge to developing uncertainty-robust digital twins. We introduce the Variational Deep Material Network (VDMN), a physics-informed surrogate model that enables efficient and probabilistic forward and inverse predictions of material behavior. The VDMN captures microstructure-induced variability by embedding variational distributions within its hierarchical, mechanistic architecture. Using an analytic propagation scheme based on Taylor-series expansion and automatic differentiation, the VDMN efficiently propagates uncertainty through the network during training and prediction. We demonstrate its capabilities in two digital-twin-driven applications: (1) as an uncertainty-aware materials digital twin, it predicts and experimentally validates the nonlinear mechanical variability in additively manufactured polymer composites; and (2) as an inverse calibration engine, it disentangles and quantitatively identifies overlapping sources of uncertainty in constituent properties. Together, these results establish the VDMN as a foundation for uncertainty-robust materials digital twins.

Kawen Li, Hangjin Cho, Richard Norte et al.

From butterfly wings to remnants of nuclear detonation, aperiodic order repeatedly emerges in nature, often exhibiting reduced sensitivity to boundaries and symmetry constraints. Inspired by this principle, a paradigm shift is introduced in nanomechanical resonator design from periodic to aperiodic structures, focusing on a special class: quasicrystals (QCs). Although soft clamping enabled by phononic stopbands has become a central strategy for achieving high-$Q_m$ nanomechanical resonators, its practical realization has been largely confined to periodic phononic crystals, where band structure engineering is well established. The potential of aperiodic architectures, however, has remained largely unexplored, owing to their intrinsic complexity and the lack of systematic approaches to identifying and exploiting stopband behavior. Here we demonstrate that soft clamping can be realized in quasicrystal architectures and that high-$Q_m$ nanomechanical resonators can be systematically achieved through a data-driven design framework. As a representative demonstration, the 12-fold QC-based resonator exhibits a quality factor $Q_m \sim 10^7$ and an effective mass of sub-nanograms at MHz frequencies, corresponding to an exceptional force sensitivity of $26.4$~aN/$\sqrt{\text{Hz}}$ compared to previous 2D phononic crystals. These results establish QCs as a robust platform for next-generation nanomechanical resonators and open a new design regime beyond periodic order.

Dongil Shin, Andrea Cupertino, Matthijs H. J. de Jong et al.

From ultra-sensitive detectors of fundamental forces to quantum networks and sensors, mechanical resonators are enabling next-generation technologies to operate in room temperature environments. Currently, silicon nitride nanoresonators stand as a leading microchip platform in these advances by allowing for mechanical resonators whose motion is remarkably isolated from ambient thermal noise. However, to date, human intuition has remained the driving force behind design processes. Here, inspired by nature and guided by machine learning, a spiderweb nanomechanical resonator is developed that exhibits vibration modes which are isolated from ambient thermal environments via a novel "torsional soft-clamping" mechanism discovered by the data-driven optimization algorithm. This bio-inspired resonator is then fabricated; experimentally confirming a new paradigm in mechanics with quality factors above 1 billion in room temperature environments. In contrast to other state-of-the-art resonators, this milestone is achieved with a compact design which does not require sub-micron lithographic features or complex phononic bandgaps, making it significantly easier and cheaper to manufacture at large scales. Here we demonstrate the ability of machine learning to work in tandem with human intuition to augment creative possibilities and uncover new strategies in computing and nanotechnology.