Jong-Seon No

Hee-Youl Kwak, Dae-Young Yun, Yongjune Kim et al.

Low-density parity-check (LDPC) codes have been successfully commercialized in communication systems due to their strong error correction capabilities and simple decoding process. However, the error-floor phenomenon of LDPC codes, in which the error rate stops decreasing rapidly at a certain level, presents challenges for achieving extremely low error rates and deploying LDPC codes in scenarios demanding ultra-high reliability. In this work, we propose training methods for neural min-sum (NMS) decoders to eliminate the error-floor effect. First, by leveraging the boosting learning technique of ensemble networks, we divide the decoding network into two neural decoders and train the post decoder to be specialized for uncorrected words that the first decoder fails to correct. Secondly, to address the vanishing gradient issue in training, we introduce a block-wise training schedule that locally trains a block of weights while retraining the preceding block. Lastly, we show that assigning different weights to unsatisfied check nodes effectively lowers the error-floor with a minimal number of weights. By applying these training methods to standard LDPC codes, we achieve the best error-floor performance compared to other decoding methods. The proposed NMS decoder, optimized solely through novel training methods without additional modules, can be integrated into existing LDPC decoders without incurring extra hardware costs. The source code is available at https://github.com/ghy1228/LDPC_Error_Floor .

Seong-Joon Park, Hee-Youl Kwak, Sang-Hyo Kim et al.

In communication and storage systems, error correction codes (ECCs) are pivotal in ensuring data reliability. As deep learning's applicability has broadened across diverse domains, there is a growing research focus on neural network-based decoders that outperform traditional decoding algorithms. Among these neural decoders, Error Correction Code Transformer (ECCT) has achieved the state-of-the-art performance, outperforming other methods by large margins. To further enhance the performance of ECCT, we propose two novel methods. First, leveraging the systematic encoding technique of ECCs, we introduce a new masking matrix for ECCT, aiming to improve the performance and reduce the computational complexity. Second, we propose a novel transformer architecture of ECCT called a double-masked ECCT. This architecture employs two different mask matrices in a parallel manner to learn more diverse features of the relationship between codeword bits in the masked self-attention blocks. Extensive simulation results show that the proposed double-masked ECCT outperforms the conventional ECCT, achieving the state-of-the-art decoding performance with significant margins.

Junghyun Lee, Eunsang Lee, Young-Sik Kim et al.

Recent studies have explored the deployment of privacy-preserving deep neural networks utilizing homomorphic encryption (HE), especially for private inference (PI). Many works have attempted the approximation-aware training (AAT) approach in PI, changing the activation functions of a model to low-degree polynomials that are easier to compute on HE by allowing model retraining. However, due to constraints in the training environment, it is often necessary to consider post-training approximation (PTA), using the pre-trained parameters of the existing plaintext model without retraining. Existing PTA studies have uniformly approximated the activation function in all layers to a high degree to mitigate accuracy loss from approximation, leading to significant time consumption. This study proposes an optimized layerwise approximation (OLA), a systematic framework that optimizes both accuracy loss and time consumption by using different approximation polynomials for each layer in the PTA scenario. For efficient approximation, we reflect the layerwise impact on the classification accuracy by considering the actual input distribution of each activation function while constructing the optimization problem. Additionally, we provide a dynamic programming technique to solve the optimization problem and achieve the optimized layerwise degrees in polynomial time. As a result, the OLA method reduces inference times for the ResNet-20 model and the ResNet-32 model by 3.02 times and 2.82 times, respectively, compared to prior state-of-the-art implementations employing uniform degree polynomials. Furthermore, we successfully classified CIFAR-10 by replacing the GELU function in the ConvNeXt model with only 3-degree polynomials using the proposed method, without modifying the backbone model.

Hee-Youl Kwak, Dae-Young Yun, Yongjune Kim et al.

Ensuring extremely high reliability in channel coding is essential for 6G networks. The next-generation of ultra-reliable and low-latency communications (xURLLC) scenario within 6G networks requires frame error rate (FER) below $10^{-9}$. However, low-density parity-check (LDPC) codes, the standard in 5G new radio (NR), encounter a challenge known as the error floor phenomenon, which hinders to achieve such low rates. To tackle this problem, we introduce an innovative solution: boosted neural min-sum (NMS) decoder. This decoder operates identically to conventional NMS decoders, but is trained by novel training methods including: i) boosting learning with uncorrected vectors, ii) block-wise training schedule to address the vanishing gradient issue, iii) dynamic weight sharing to minimize the number of trainable weights, iv) transfer learning to reduce the required sample count, and v) data augmentation to expedite the sampling process. Leveraging these training strategies, the boosted NMS decoder achieves the state-of-the art performance in reducing the error floor as well as superior waterfall performance. Remarkably, we fulfill the 6G xURLLC requirement for 5G LDPC codes without a severe error floor. Additionally, the boosted NMS decoder, once its weights are trained, can perform decoding without additional modules, making it highly practical for immediate application. The source code is available at https://github.com/ghy1228/LDPC_Error_Floor.

Seong-Joon Park, Hee-Youl Kwak, Sang-Hyo Kim et al.

Error correcting codes (ECCs) are indispensable for reliable transmission in communication systems. The recent advancements in deep learning have catalyzed the exploration of ECC decoders based on neural networks. Among these, transformer-based neural decoders have achieved state-of-the-art decoding performance. In this paper, we propose a novel Cross-attention Message-Passing Transformer (CrossMPT), which shares key operational principles with conventional message-passing decoders. While conventional transformer-based decoders employ self-attention mechanism without distinguishing between the types of input vectors (i.e., magnitude and syndrome vectors), CrossMPT updates the two types of input vectors separately and iteratively using two masked cross-attention blocks. The mask matrices are determined by the code's parity-check matrix, which explicitly captures the irrelevant relationship between two input vectors. Our experimental results show that CrossMPT significantly outperforms existing neural network-based decoders for various code classes. Notably, CrossMPT achieves this decoding performance improvement, while significantly reducing the memory usage, complexity, inference time, and training time.

Seong-Joon Park, Hee-Youl Kwak, Sang-Hyo Kim et al.

Channel coding for 6G networks is expected to support a wide range of requirements arising from heterogeneous communication scenarios. These demands challenge traditional code-specific decoders, which lack the flexibility and scalability required for next-generation systems. To tackle this problem, we propose an AI-native foundation model for unified and code-agnostic decoding based on the transformer architecture. We first introduce a cross-attention message-passing transformer (CrossMPT). CrossMPT employs two masked cross-attention blocks that iteratively update two distinct input representations-magnitude and syndrome vectors-allowing the model to effectively learn the decoding problem. Notably, our CrossMPT has achieved state-of-the-art decoding performance among single neural decoders. Building on this, we develop foundation CrossMPT (FCrossMPT) by making the architecture invariant to code length, rate, and class, allowing a single trained model to decode a broad range of codes without retraining. To further enhance decoding performance, particularly for short blocklength codes, we propose CrossMPT ensemble decoder (CrossED), an ensemble decoder composed of multiple parallel CrossMPT blocks employing different parity-check matrices. This architecture can also serve as a foundation model, showing strong generalization across diverse code types. Overall, the proposed AI-native code-agnostic decoder offers flexibility, scalability, and high performance, presenting a promising direction to channel coding for 6G networks.

Dae-Young Yun, Hee-Youl Kwak, Yongjune Kim et al.

In this paper, we propose a neural window decoder (NWD) for spatially coupled low-density parity-check (SC-LDPC) codes. The proposed NWD retains the conventional window decoder (WD) process but incorporates trainable neural weights. To train the weights of NWD, we introduce two novel training strategies. First, we restrict the loss function to target variable nodes (VNs) of the window, which prunes the neural network and accordingly enhances training efficiency. Second, we employ the active learning technique with a normalized loss term to prevent the training process from biasing toward specific training regions. Next, we develop a systematic method to derive non-uniform schedules for the NWD based on the training results. We introduce trainable damping factors that reflect the relative importance of check node (CN) updates. By skipping updates with less importance, we can omit $\mathbf{41\%}$ of CN updates without performance degradation compared to the conventional WD. Lastly, we address the error propagation problem inherent in SC-LDPC codes by deploying a complementary weight set, which is activated when an error is detected in the previous window. This adaptive decoding strategy effectively mitigates error propagation without requiring modifications to the code and decoder structures.

Joon-Woo Lee, HyungChul Kang, Yongwoo Lee et al.

Fully homomorphic encryption (FHE) is one of the prospective tools for privacypreserving machine learning (PPML), and several PPML models have been proposed based on various FHE schemes and approaches. Although the FHE schemes are known as suitable tools to implement PPML models, previous PPML models on FHE encrypted data are limited to only simple and non-standard types of machine learning models. These non-standard machine learning models are not proven efficient and accurate with more practical and advanced datasets. Previous PPML schemes replace non-arithmetic activation functions with simple arithmetic functions instead of adopting approximation methods and do not use bootstrapping, which enables continuous homomorphic evaluations. Thus, they could not use standard activation functions and could not employ a large number of layers. The maximum classification accuracy of the existing PPML model with the FHE for the CIFAR-10 dataset was only 77% until now. In this work, we firstly implement the standard ResNet-20 model with the RNS-CKKS FHE with bootstrapping and verify the implemented model with the CIFAR-10 dataset and the plaintext model parameters. Instead of replacing the non-arithmetic functions with the simple arithmetic function, we use state-of-the-art approximation methods to evaluate these non-arithmetic functions, such as the ReLU, with sufficient precision [1]. Further, for the first time, we use the bootstrapping technique of the RNS-CKKS scheme in the proposed model, which enables us to evaluate a deep learning model on the encrypted data. We numerically verify that the proposed model with the CIFAR-10 dataset shows 98.67% identical results to the original ResNet-20 model with non-encrypted data. The classification accuracy of the proposed model is 90.67%, which is pretty close to that of the original ResNet-20 CNN model...

Junghyun Lee, Eunsang Lee, Joon-Woo Lee et al.

Homomorphic encryption is one of the representative solutions to privacy-preserving machine learning (PPML) classification enabling the server to classify private data of clients while guaranteeing privacy. This work focuses on PPML using word-wise fully homomorphic encryption (FHE). In order to implement deep learning on word-wise homomorphic encryption (HE), the ReLU and max-pooling functions should be approximated by some polynomials for homomorphic operations. Most of the previous studies focus on HE-friendly networks, where the ReLU and max-pooling functions are approximated using low-degree polynomials. However, for the classification of the CIFAR-10 dataset, using a low-degree polynomial requires designing a new deep learning model and training. In addition, this approximation by low-degree polynomials cannot support deeper neural networks due to large approximation errors. Thus, we propose a precise polynomial approximation technique for the ReLU and max-pooling functions. Precise approximation using a single polynomial requires an exponentially high-degree polynomial, which results in a significant number of non-scalar multiplications. Thus, we propose a method to approximate the ReLU and max-pooling functions accurately using a composition of minimax approximate polynomials of small degrees. If we replace the ReLU and max-pooling functions with the proposed approximate polynomials, well-studied deep learning models such as ResNet and VGGNet can still be used without further modification for PPML on FHE. Even pre-trained parameters can be used without retraining. We approximate the ReLU and max-pooling functions in the ResNet-152 using the composition of minimax approximate polynomials of degrees 15, 27, and 29. Then, we succeed in classifying the plaintext ImageNet dataset with 77.52% accuracy, which is very close to the original model accuracy of 78.31%.

Minki Song, Seunghwan Lee, Eunsang Lee et al.

Among many submissions to the NIST post-quantum cryptography (PQC) project, NewHope is a promising key encapsulation mechanism (KEM) based on the Ring-Learning with errors (Ring-LWE) problem. Since NewHope is an indistinguishability (IND)-chosen ciphertext attack secure KEM by applying the Fujisaki-Okamoto transform to an IND-chosen plaintext attack secure public key encryption, accurate calculation of decryption failure rate (DFR) is required to guarantee resilience against attacks that exploit decryption failures. However, the current upper bound of DFR on NewHope is rather loose because the compression noise, the effect of encoding/decoding of NewHope, and the approximation effect of centered binomial distribution are not fully considered. Furthermore, since NewHope is a Ring-LWE based cryptosystem, there is a problem of error dependency among error coefficients, which makes accurate DFR calculation difficult. In this paper, we derive much tighter upper bound on DFR than the current upper bound using constraint relaxation and union bound. Especially, the above-mentioned factors are all considered in derivation of new upper bound and the centered binomial distribution is not approximated to subgaussian distribution. In addition, since the error dependency is considered, the new upper bound is much closer to the real DFR than the previous upper bound. Furthermore, the new upper bound is parameterized by using Chernoff-Cramer bound in order to facilitate calculation of new upper bound for the parameters of NewHope. Since the new upper bound is much lower than the DFR requirement of PQC, this DFR margin is used to improve the security and bandwidth efficiency of NewHope. As a result, the security level of NewHope is improved by 7.2 % or bandwidth efficiency is improved by 5.9 %.

Minki Song, Seunghwan Lee, Eunsang Lee et al.

Among many submissions to the NIST post-quantum cryptography (PQC) project, NewHope is a promising key encapsulation mechanism (KEM) based on the Ring-Learning with errors (Ring-LWE) problem. Since the most important factors to be considered for PQC are security and cost including bandwidth and time/space complexity, in this paper, by doing exact noise analysis and using Bose Chaudhuri Hocquenghem (BCH) codes, it is shown that the security and bandwidth efficiency of NewHope can be substantially improved. In detail, the decryption failure rate (DFR) of NewHope is recalculated by performing exact noise analysis, and it is shown that the DFR of NewHope has been too conservatively calculated. Since the recalculated DFR is much lower than the required $2^{-128}$, this DFR margin is exploited to improve the security up to 8.5 \% or the bandwidth efficiency up to 5.9 \% without changing the procedure of NewHope. The additive threshold encoding (ATE) used in NewHope is a simple error correcting code (ECC) robust to side channel attack, but its error-correction capability is relatively weak compared with other ECCs. Therefore, if a proper error-correction scheme is applied to NewHope, either security or bandwidth efficiency or both can be improved. Among various ECCs, BCH code has been widely studied for its application to cryptosystems due to its advantages such as no error floor problem. In this paper, the ATE and total noise channel are regarded as a super channel from an information-theoretic viewpoint. Based on this super channel analysis, various concatenated coding schemes of ATE and BCH code for NewHope have been investigated. Through numerical analysis, it is revealed that the security and bandwidth efficiency of NewHope are substantially improved by using the proposed error-correction schemes.

Wijik Lee, Young-Sik Kim, Jong-Seon No

In this paper, we propose a new signature scheme based on a punctured Reed--Muller (RM) code with random insertion, which improves the Goppa code-based signature scheme developed by Courtois, Finiasz, and Sendrier (CFS). The CFS signature scheme has certain drawbacks in terms of scaling of the parameters and a lack of existential unforgeability under adaptive chosen message attacks (EUF-CMA) security proof. Further, the proposed modified RM code-based signature scheme can use complete decoding, which can be implemented using a recursive decoding method, and thus syndromes for errors larger than the error correctability can be decoded for signing, which improves the probability of successful signing and reduces the signing time. Using the puncturing and insertion methods, the proposed RM code-based signature scheme can avoid some known attacks for RM code-based cryptosystems. The parameters of the proposed signature scheme such as error weight parameter $w$ and the maximum signing trial $N$, can be adjusted in terms of signing time and security level, and it is also proved that the proposed signature scheme achieves EUF-CMA security.

Sung-Tai Choi, Seokbeom Hong, Jong-Seon No et al.

In this paper, for an odd prime $p$, the differential spectrum of the power function $x^{\frac{p^k+1}{2}}$ in $\mathbb{F}_{p^n}$ is calculated. For an odd prime $p$ such that $p\equiv 3\bmod 4$ and odd $n$ with $k|n$, the differential spectrum of the power function $x^{\frac{p^n+1}{p^k+1}+\frac{p^n-1}{2}}$ in $\mathbb{F}_{p^n}$ is also derived. From their differential spectrums, the differential uniformities of these two power functions are determined. We also find some new power functions having low differential uniformity.