Jung Woo Kim

Jaiyoung Park, Sejin Park, Jai Hyun Park et al.

As large language models (LLMs) become ubiquitous, privacy concerns pertaining to inference keep growing. Fully homomorphic encryption (FHE) has emerged as a primary cryptographic solution for non-interactive confidential LLM inference. However, existing solutions scale poorly with input token length, focusing on small models or input sizes. They also suffer from large outlier values, which strongly impact the evaluation of non-linear layers, leading to heavy polynomial approximation costs. We scale up FHE-based LLM inference in two directions. First, we accelerate FHE-based inference for 128 encrypted tokens. We adopt ML techniques (token prepending and orthogonal rotations) to mitigate outlier impacts on the FHE evaluation of non-linear layers. Separately, we devise a novel polynomial evaluation method for sparsely-packed ciphertexts to speed up our homomorphic SoftMax implementation. We combine these with recent fast homomorphic linear algebra techniques, achieving significantly improved efficiency. Second, we expand the prompt size up to thousands of tokens for contexts where only the final part of the input is sensitive and encrypted. Processing this requires handling standard plaintext-plaintext and ciphertext-ciphertext components, alongside a wide homomorphic computation for a novel plaintext-ciphertext component. To address this, we devise a dedicated homomorphic linear algebra algorithm, building a shallow homomorphic attention circuit that minimizes bootstrapping costs. Based on these ingredients, we present a CKKS-based end-to-end implementation of Llama-3-8B private inference. On 8 NVIDIA RTX PRO 6000 GPUs, 128 encrypted tokens take 20s for summarization and 18s/token for generation (vastly outperforming the SOTA 295s on costlier H100 GPUs). For a heterogeneous 4096-token input (last 128 encrypted), it takes 64s for summarization and 22s/token for generation.

Jincheol Ha, Guillaume Hanrot, Taeyeong Noh et al.

Among biometric verification systems, irises stand out because they offer high accuracy even in large-scale databases. For example, the World ID project aims to provide authentication to all humans via iris recognition, with millions already registered. Storing such biometric data raises privacy concerns, which can be addressed using privacy-enhancing techniques. Bloemen et al. describe a solution based on 2-out-of-3 Secret-Sharing Multiparty Computation (SS-MPC), for the World ID setup. In terms of security, unless an adversary corrupts 2~servers, the iris codes remain confidential and nothing leaks beyond the result of the computation. Their solution is able to match~$32$ users against a database of~$2^{22}$ iris codes in~$\approx 2$s , using~24 H100 GPUs, more than 40~communication rounds and $81$GB/party of data transferred (the timing assumes a network speed above~3Tb/s). In the present work, we explore the use of Threshold Fully Homomorphic Encryption (ThFHE) for the same task. The ThFHE solution brings a number of security advantages: no trusted setup, the encrypted database and queries can be public, the secret can be distributed among many parties, and active security can be added without significant performance degradation. Our proof-of-concept implementation of the computation phase handles $32$~eyes against a database of $7\cdot 2^{14}$ iris codes in~$\approx 1.8$s ($\approx 0.33s$ for 4 eyes against the same database), using 8 RTX-5090 GPUs. To this, one should add~2 to 3 rounds of communication (depending on deployment choice). We perform the matching using the CKKS (Th)FHE scheme. Our main technical ingredients are the use of recent progress on FHE-based linear algebra boosted using int8 GPU operations, and the introduction of a technique reducing the number of ciphertexts to be processed as early as possible.