Yuhang Gan

Yuhang Gan, Yiwei Yang, Yuyi Li et al.

Long-running LLM agents keep valuable state resident on GPUs: KV caches, request schedulers, communication state, and sometimes online adapters. Losing this state after a GPU or communicator failure can discard minutes to hours of work, yet existing recovery mechanisms either restart the whole serving stack or require application-specific checkpoint logic inside every attention and runtime component. This paper argues that fault tolerance for such workloads needs a GPU-resident execution context: checkpoint hooks must run at device synchronization points, observe binary kernels that frameworks and libraries actually execute, and recover without putting the host CPU on the critical path. We present Concordia, a runtime that uses a device-resident persistent kernel as the substrate for fault-tolerant LLM inference. Concordia interposes on GPU module loading and supports PTX- and SASS-level instrumentation, allowing checkpoint and pause hooks to be inserted below framework code and library boundaries. For each registered LLM state region, Concordia JIT-compiles a specialized delta-checkpoint handler -- for example, a KV-block scanner, adapter-page scanner, or recovery applier -- and hot-swaps it into the persistent kernel's operator table. The persistent kernel consumes a lock-free ring buffer of compute, checkpoint, append-log, and recovery tasks, so the same always-on executor triggers dirty-page detection, stages deltas, and appends committed records to a CPU-visible log in CXL memory or host DRAM.

Shiyi Ling, Yuhang Gan, Chen Qian

The escalating arms race between Internet censorship and evasion has driven censors to evolve from static rule-based filtering to sophisticated deep learning-based traffic analysis. While recent automated evasion tools have attempted to counter this by leveraging stochastic search and programmable heuristics, they continue to suffer from insufficient evasion robustness across diverse censorship modalities and poor usability due to complex, mechanism-specific configurations that require manual fitness tuning or domain-specific languages. In this paper, we propose a paradigm shift that reframes censorship evasion as a semantic image-to-image editing task, allowing users to execute it with a single prompt. We introduce FlowPaint, a novel generative framework that leverages the "world knowledge" of large diffusion models to automatically reshape censored traffic into benign patterns. FlowPaint utilizes an instruction-tuned diffusion architecture to perform semantic editing on network flows. Evaluations against both industrial-grade rule-based middleboxes and learning-based classifiers demonstrate that FlowPaint outperforms existing censorship evasion baselines, enabling users to counter diverse censorship paradigms solely by varying natural language instructions