Error Correcting Codes

Seong‐Joon Park, Kwak, Hee-Youl, Sang‐Hyo Kim +3 more

In communication and storage systems, error correction codes (ECCs) are pivotal in ensuring data reliability. As deep learning's applicability has broadened across diverse domains, there is a growing research focus on neural network-based decoders that outperform traditional decoding algorithms. Among these neural decoders, Error Correction Code Transformer (ECCT) has achieved the state-of-the-art performance, outperforming other methods by large margins. To further enhance the performance of ECCT, we propose two novel methods. First, leveraging the systematic encoding technique of ECCs, we introduce a new masking matrix for ECCT, aiming to improve the performance and reduce the computational complexity. Second, we propose a novel transformer architecture of ECCT called a double-masked ECCT. This architecture employs two different mask matrices in a parallel manner to learn more diverse features of the relationship between codeword bits in the masked self-attention blocks. Extensive simulation results show that the proposed double-masked ECCT outperforms the conventional ECCT, achieving the state-of-the-art decoding performance with significant margins.

Alain Poli

Zhengyi Jiang, Wenhao Liu, Zhongyi Huang +1 more

Analog error-correcting codes (Analog ECCs), introduced by Roth [1], address errors in vector-matrix multiplication arising from analog noise and sparse outliers in in-memory computing. A fundamental open problem concerns the lower bound on the single-error correction threshold $Γ_2(\mathcal C)$ for real $[n,k]$ linear codes with fixed redundancy $r=n-k\geq 2$. Li et al. [2] recently established that for redundancy $r=2$, every real $[n,n-2]$ linear code $\mathcal{C}$ satisfies $Γ_2(\mathcal C)\geq \csc^2(\fracπ{2n})$, resolving an open problem in [1], and showed that, for every fixed $r \geq 2$, there exists a class of $[n,k]$ linear code $\mathcal{C}$ over $\mathbb{R}$ such that $Γ_2(\mathcal{C}) \leq O(n^{1+\frac{1}{r-1}})$. This paper proves the matching converse in [2]. For every $[n,k]$ linear code $\mathcal{C}\subseteq \mathbb R^n$ with fixed redundancy $2\leq r

J. J. Stiffler

Matan Levy, Yoni Choukroun, Lior Wolf

Error correction codes (ECC) are crucial for ensuring reliable information transmission in communication systems. Choukroun & Wolf (2022b) recently introduced the Error Correction Code Transformer (ECCT), which has demonstrated promising performance across various transmission channels and families of codes. However, its high computational and memory demands limit its practical applications compared to traditional decoding algorithms. Achieving effective quantization of the ECCT presents significant challenges due to its inherently small architecture, since existing, very low-precision quantization techniques often lead to performance degradation in compact neural networks. In this paper, we introduce a novel acceleration method for transformer-based decoders. We first propose a ternary weight quantization method specifically designed for the ECCT, inducing a decoder with multiplication-free linear layers. We present an optimized self-attention mechanism to reduce computational complexity via codeaware multi-heads processing. Finally, we provide positional encoding via the Tanner graph eigendecomposition, enabling a richer representation of the graph connectivity. The approach not only matches or surpasses ECCT's performance but also significantly reduces energy consumption, memory footprint, and computational complexity. Our method brings transformer-based error correction closer to practical implementation in resource-constrained environments, achieving a 90% compression ratio and reducing arithmetic operation energy consumption by at least 224 times on modern hardware.