Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Log inSign up
Challenges
Datasets
Workspaces
Papers
Discussions
Leaderboard
Blog
Job Board
Q3AS

© 2026 Aqora Quantum S.A.S.

TermsPrivacyLegal Notice

IEEE International Symposium on Information Theory (ISIT)

Interpreting KO Codes

R. Shekhar, Natasha Devroye, Gy. Turán, Miloš Žefran

Published Jun 22, 2025• 0 citations
Publisher

Abstract

The KO (Kronecker Operation) code is a recent deep-learned error-correcting code using a neural network architecture to generalize a Reed-Muller code with Dumer decoding. Analyzing the encoder modules and using ablation techniques, we give interpretations of the KO encoder which significantly reduce the number of parameters. We also discuss interpretability aspects of the KO decoder. The interpretation opens up possibilities to give an explicit representation of KO codes, which could be useful for more efficient learning of KO codes and explaining the learning mechanism underlying the empirical observations made about its performance in previous work.

DOI
10.1109/isit63088.2025.11195264

Related papers

HiKO: A Hierarchical Framework for Beyond-Second-Order KO Codes

Shubham Srivastava, Adrish Banerjee

This paper introduces HiKO (Hierarchical Kronecker Operation), a novel framework for training high-rate neural error-correcting codes that enables KO codes to outperform Reed-Muller codes beyond second order. To our knowledge, this is the first attempt to extend KO codes beyond second order. While conventional KO codes show promising results for low-rate regimes ($r < 2$), they degrade at higher rates -- a critical limitation for practical deployment. Our framework incorporates three key innovations: (1) a hierarchical training methodology that decomposes complex high-rate codes into simpler constituent codes for efficient knowledge transfer, (2) enhanced neural architectures with dropout regularization and learnable skip connections tailored for the Plotkin structure, and (3) a progressive unfreezing strategy that systematically transitions from pre-trained components to fully optimized integrated codes. Our experiments show that HiKO codes consistently outperform traditional Reed-Muller codes across various configurations, achieving notable performance improvements for third-order ($r = 3$) and fourth-order ($r = 4$) codes. Analysis reveals that HiKO codes successfully approximate Shannon-optimal Gaussian codebooks while preserving efficient decoding properties. This represents the first successful extension of KO codes beyond second order, opening new possibilities for neural code deployment in high-throughput communication systems.

arXiv:2506.10121•Jun 11, 2025•0 citations

Interpreting Training Aspects of Deep-Learned Error-Correcting Codes

Natasha Devroye, Abhijeet Mulgund, Raj Shekhar +3 more

As new deep-learned error-correcting codes continue to be introduced, it is important to develop tools to interpret the designed codes and understand the training process. Prior work focusing on the deep-learned TurboAE has both interpreted the learned encoders post-hoc by mapping these onto nearby "interpretable" encoders, and experimentally evaluated the performance of these interpretable encoders with various decoders. Here we look at developing tools for interpreting the training process for deep-learned error-correcting codes, focusing on: 1) using the Goldreich-Levin algorithm to quickly interpret the learned encoder; 2) using Fourier coefficients as a tool for understanding the training dynamics and the loss landscape; 3) reformulating the training loss, the binary cross entropy, by relating it to encoder and decoder parameters, and the bit error rate (BER); 4) using these insights to formulate and study a new training procedure. All tools are demonstrated on TurboAE, but are applicable to other deep-learned forward error correcting codes (without feedback).

arXiv:2305.04347•IEEE International Symposium on Information Theory (ISIT)•Jun 25, 2023•3 citations

量子 Reed-Müller 碼

鄭閔鴻

In this thesis, we start from the theory of classical error-correcting codes. We also introduce classical Reed-Muller codes along with its properties. We discuss the encoding and decoding methods for classical Reed-Muller codes using the majority rule. Then we turn to the topic of quantum error-correcting codes.We talk about stabilizer codes and discuss how classical Reed-Muller codes can be transformed into quantum Reed-Muller codes using Steane's method. Finally, we give a construction of a quantum code with parameter [[8,3,3]], and discuss its encoding and decoding process.

•Jan 1, 2011•0 citations

On deep learning-based channel decoding

Tobias Gruber, Sebastian Cammerer, Jakob Hoydis +1 more

We revisit the idea of using deep neural networks for one-shot decoding of random and structured codes, such as polar codes. Although it is possible to achieve maximum a posteriori (MAP) bit error rate (BER) performance for both code families and for short codeword lengths, we observe that (i) structured codes are easier to learn and (ii) the neural network is able to generalize to codewords that it has never seen during training for structured, but not for random codes. These results provide some evidence that neural networks can learn a form of decoding algorithm, rather than only a simple classifier. We introduce the metric normalized validation error (NVE) in order to further investigate the potential and limitations of deep learning-based decoding with respect to performance and complexity.

Conference on Information Sciences and Systems•Mar 1, 2017•574 citations

On Deep Learning-Based Channel Decoding

Tobias Gruber, Sebastian Cammerer, Jakob Hoydis +1 more

We revisit the idea of using deep neural networks for one-shot decoding of random and structured codes, such as polar codes. Although it is possible to achieve maximum a posteriori (MAP) bit error rate (BER) performance for both code families and for short codeword lengths, we observe that (i) structured codes are easier to learn and (ii) the neural network is able to generalize to codewords that it has never seen during training for structured, but not for random codes. These results provide some evidence that neural networks can learn a form of decoding algorithm, rather than only a simple classifier. We introduce the metric normalized validation error (NVE) in order to further investigate the potential and limitations of deep learning-based decoding with respect to performance and complexity.

arXiv:1701.07738•Jan 26, 2017•42 citations

On the Interpretability of Neural Network Decoders

Lukas Bödeker, Luc J. B. Kusters, Markus Müller

Abstract Neural‐network (NN) based decoders are becoming increasingly popular in the field of quantum error correction (QEC), including for decoding of state‐of‐the‐art quantum computation experiments. In this work, established interpretability methods are used from the field of machine learning, to introduce a toolbox to achieve an understanding of the underlying decoding logic of NN decoders, which have been trained but otherwise typically operate as black‐box models. To illustrate the capabilities of the employed interpretability method, based on the Shapley value approximation, an examplary case study of a NN decoder is provided that is trained for flag‐qubit based fault‐tolerant (FT) QEC with the Steane code. The interpretation of particular decoding decisions of the NN is analysed, by doing so it is revealed how the NN learns to capture fundamental structures in the information gained from syndrome and flag qubit measurements, in order to come to a FT correction decision. Further, it is showed that the understanding of how the NN obtains a decoding decision can be used on the one hand to identify flawed processing of error syndrome information by the NN, resulting in decreased decoding performance, as well as for well‐informed improvements of the NN architecture. The diagnostic capabilities of the interpretability method presented here can help ensure successful application of machine learning for decoding of QEC protocols.

arXiv:2502.20269•Advanced Quantum Technologies•Jul 9, 2025•0 citations
The KO (Kronecker Operation) code is a recent deep-learned error-correcting code using a neural network architecture to generalize a Reed-Muller code with Dumer decoding. Analyzing the encoder modules and using ablation techniques, we give interpretations of the KO encoder which significantly reduce the number of parameters. We also discuss interpretability aspects of the KO decoder. The interpretation opens up possibilities to give an explicit representation of KO codes, which could be useful for more efficient learning of KO codes and explaining the learning mechanism underlying the empirical observations made about its performance in previous work.
This paper introduces HiKO (Hierarchical Kronecker Operation), a novel framework for training high-rate neural error-correcting codes that enables KO codes to outperform Reed-Muller codes beyond second order. To our knowledge, this is the first attempt to extend KO codes beyond second order. While conventional KO codes show promising results for low-rate regimes (r<2r < 2r<2), they degrade at higher rates -- a critical limitation for practical deployment. Our framework incorporates three key innovations: (1) a hierarchical training methodology that decomposes complex high-rate codes into simpler constituent codes for efficient knowledge transfer, (2) enhanced neural architectures with dropout regularization and learnable skip connections tailored for the Plotkin structure, and (3) a progressive unfreezing strategy that systematically transitions from pre-trained components to fully optimized integrated codes. Our experiments show that HiKO codes consistently outperform traditional Reed-Muller codes across various configurations, achieving notable performance improvements for third-order (r=3r = 3r=3) and fourth-order (r=4r = 4r=4) codes. Analysis reveals that HiKO codes successfully approximate Shannon-optimal Gaussian codebooks while preserving efficient decoding properties. This represents the first successful extension of KO codes beyond second order, opening new possibilities for neural code deployment in high-throughput communication systems.
As new deep-learned error-correcting codes continue to be introduced, it is important to develop tools to interpret the designed codes and understand the training process. Prior work focusing on the deep-learned TurboAE has both interpreted the learned encoders post-hoc by mapping these onto nearby "interpretable" encoders, and experimentally evaluated the performance of these interpretable encoders with various decoders. Here we look at developing tools for interpreting the training process for deep-learned error-correcting codes, focusing on: 1) using the Goldreich-Levin algorithm to quickly interpret the learned encoder; 2) using Fourier coefficients as a tool for understanding the training dynamics and the loss landscape; 3) reformulating the training loss, the binary cross entropy, by relating it to encoder and decoder parameters, and the bit error rate (BER); 4) using these insights to formulate and study a new training procedure. All tools are demonstrated on TurboAE, but are applicable to other deep-learned forward error correcting codes (without feedback).
In this thesis, we start from the theory of classical error-correcting codes. We also introduce classical Reed-Muller codes along with its properties. We discuss the encoding and decoding methods for classical Reed-Muller codes using the majority rule. Then we turn to the topic of quantum error-correcting codes.We talk about stabilizer codes and discuss how classical Reed-Muller codes can be transformed into quantum Reed-Muller codes using Steane's method. Finally, we give a construction of a quantum code with parameter [[8,3,3]], and discuss its encoding and decoding process.
We revisit the idea of using deep neural networks for one-shot decoding of random and structured codes, such as polar codes. Although it is possible to achieve maximum a posteriori (MAP) bit error rate (BER) performance for both code families and for short codeword lengths, we observe that (i) structured codes are easier to learn and (ii) the neural network is able to generalize to codewords that it has never seen during training for structured, but not for random codes. These results provide some evidence that neural networks can learn a form of decoding algorithm, rather than only a simple classifier. We introduce the metric normalized validation error (NVE) in order to further investigate the potential and limitations of deep learning-based decoding with respect to performance and complexity.
We revisit the idea of using deep neural networks for one-shot decoding of random and structured codes, such as polar codes. Although it is possible to achieve maximum a posteriori (MAP) bit error rate (BER) performance for both code families and for short codeword lengths, we observe that (i) structured codes are easier to learn and (ii) the neural network is able to generalize to codewords that it has never seen during training for structured, but not for random codes. These results provide some evidence that neural networks can learn a form of decoding algorithm, rather than only a simple classifier. We introduce the metric normalized validation error (NVE) in order to further investigate the potential and limitations of deep learning-based decoding with respect to performance and complexity.
Abstract Neural‐network (NN) based decoders are becoming increasingly popular in the field of quantum error correction (QEC), including for decoding of state‐of‐the‐art quantum computation experiments. In this work, established interpretability methods are used from the field of machine learning, to introduce a toolbox to achieve an understanding of the underlying decoding logic of NN decoders, which have been trained but otherwise typically operate as black‐box models. To illustrate the capabilities of the employed interpretability method, based on the Shapley value approximation, an examplary case study of a NN decoder is provided that is trained for flag‐qubit based fault‐tolerant (FT) QEC with the Steane code. The interpretation of particular decoding decisions of the NN is analysed, by doing so it is revealed how the NN learns to capture fundamental structures in the information gained from syndrome and flag qubit measurements, in order to come to a FT correction decision. Further, it is showed that the understanding of how the NN obtains a decoding decision can be used on the one hand to identify flawed processing of error syndrome information by the NN, resulting in decreased decoding performance, as well as for well‐informed improvements of the NN architecture. The diagnostic capabilities of the interpretability method presented here can help ensure successful application of machine learning for decoding of QEC protocols.