One of the fundamental goals of coding theory is to find optimal codes, in terms of achieving the largest probability of correct decoding. Most of the foundational works on optimal codes use three assumptions. First, optimality is defined relative to a code’s rate and minimum Hamming distance, and not its probability of correct decoding. The problem is that the former conditions are not always good indicators for the latter. Second, for practicality only optimal codes with some kind of algebraic structure are considered, the most common being linearity. While linear codes have practical advantages, it is known that often the best codes are not linear. Thirdly, codes are usually proven optimal only for memoryless channels. However, there are many real-life channels which exhibit memory. Attempts to extend these results often make use of interleaving. This technique renders some of these channels with memory equivalent to a memoryless one with respect to the decoder. However, this comes with two disadvantages namely: we fail to exploit the channel’s memory and we add latency to the communication system. In this thesis, we seek optimal codes which require milder conditions than those aforementioned. Specifically, we describe a class of optimal q-ary block codes over additive noise channels, which are a generalization of a class of optimal codes from Hamada. We prove these codes are optimal using our novel method of analyzing the probability of correct decoding of block codes based on so-called error indexed decoder regions. Our method allows us to prove that these codes must be either a linear code or a coset of one.
One of the fundamental goals of coding theory is to find optimal codes, in terms of achieving the largest probability of correct decoding. Most of the foundational works on optimal codes use three assumptions. First, optimality is defined relative to a code’s rate and minimum Hamming distance, and not its probability of correct decoding. The problem is that the former conditions are not always good indicators for the latter. Second, for practicality only optimal codes with some kind of algebraic structure are considered, the most common being linearity. While linear codes have practical advantages, it is known that often the best codes are not linear. Thirdly, codes are usually proven optimal only for memoryless channels. However, there are many real-life channels which exhibit memory. Attempts to extend these results often make use of interleaving. This technique renders some of these channels with memory equivalent to a memoryless one with respect to the decoder. However, this comes with two disadvantages namely: we fail to exploit the channel’s memory and we add latency to the communication system. In this thesis, we seek optimal codes which require milder conditions than those aforementioned. Specifically, we describe a class of optimal q-ary block codes over additive noise channels, which are a generalization of a class of optimal codes from Hamada. We prove these codes are optimal using our novel method of analyzing the probability of correct decoding of block codes based on so-called error indexed decoder regions. Our method allows us to prove that these codes must be either a linear code or a coset of one.
Most of the basic concepts of algebraic coding theory are derived for the memoryless binary symmetric channel. These concepts do not necessarily hold for time-varying channels or for channels with memory. However, errors in real-life channels seem to occur in bursts rather than independently, suggesting that these channels exhibit some statistical dependence or memory. Nonetheless, the same algebraic codes are still commonly used in current communication systems that employ interleaving to spread channel error bursts over the set of received codewords to make the channel appear memoryless to the block decoder. This method suffers from immediate shortcomings as it fails to exploit the channel’s memory while adding delay to the system. We study optimal maximum likelihood block decoding of binary codes sent over several binary additive channels with infinite and finite memory. We derive conditions on general binary codes and channels parameters under which maximum likelihood and minimum distance decoding are equivalent. The channels considered in this work are the infinite and finite memory Polya contagion channels [1],
This thesis concentrates on studying three problems in error-correcting codes: optimal encoding, trellis structure, and normalized weight of linear block codes. In general, the research in error-correcting codes can be roughly classified into coding practice and coding theory. The problems in coding practice focus on codes and decoders with more attention on the subject of the tradeoffs between performance and complexity, e.g., soft-decision decoding, bit error probability, or trellis of the codes. On the other hand, coding theory concentrates on the structure of codes based on sophisticated mathematical techniques, with little concern for the decoding complexity, e.g., coset weight distribution, algebraic constructions, or bounds on possible parameters of the codes. In this dissertation, we will present three fundamental topics in coding practice: optimal encoding of linear block codes for bit error probability, trellis structure of linear block codes for trellis-based decoding, and normalized weight of block coded modulation for bandwidth efficient communication systems. The central theme of the thesis is to transform the problems of the above three topics in coding practice into problems in coding theory and to develop the theory associated with the problems. Much of this thesis is devoted to studying what 'optimal' means in different contexts. In summary, we investigated the optimal information bit encoding of a given linear block code for minimizing the probability of an information bit error, with the goal of determining an optimal generator matrix by using the structure of the code. We showed that many natural codes have systematic optimal generator matrices in the quiet region. We analyzed the trellis structure of a self-dual code C to minimize the trellis complexity of the code by considering all equivalent codes of C, with the goal of reducing the soft-decision decoding complexity via permutations on the codeword coordinates. We determined the length/dimension profile of all eight [32, 16, 8] self-dual codes, and find their corresponding optimum permutations. We examined the performance gains achievable by adding interblock memory to block coded modulation systems, with the goal of finding the best codes for normalized distance rather than Hamming distance. We found a large number of best codes for normalized distance for the cases of practical interest.
The weight spectrum of sequences of binary linear codes that achieve arbitrarily small word error probability on a class of noisy channels at a nonzero rate is studied. We refer to such sequences as good codes. The class of good codes includes turbo, low-density parity-check, and repeat-accumulate codes. We show that a sequence of codes is good when transmitted over a memoryless binary-symmetric channel (BSC) or an additive white Gaussian noise (AWGN) channel if and only if the slope of its spectrum is finite everywhere and its minimum Hamming distance goes to infinity with no requirement on its rate growth. The extension of these results to code ensembles in probabilistic terms follows in a direct manner. We also show that the sufficient condition holds for any binary-input memoryless channel.
In any engineering discipline, one looks for methods of solving a given class of problems and then for methods of quantifying the performance and complexity of those solutions. Accordingly, in this chapter, we shall study performance in terms of the composition structure of block codes, and the probabilities of decoding error and of decoding failure. Later, after satisfactory methods are in hand, one turns to questions of optimality. Are these methods the best methods, and if not, in what ways and by how much are they inferior? To answer such questions, one needs to know how good are the known codes and how good are the best possible codes. Generally, we cannot answer either of these questions satisfactorily. Although for any given rate and blocklength, not too small, the best possible code is not known, a number of bounds are known – bounds beyond which no codes can exist, and bounds within which codes are sure to exist. We shall give some such bounds on the best possible codes later in this chapter. Knowledge of these bounds can deepen our understanding of the subject. Weight distributions of block codes If a linear block code has a minimum distance d min , then we know that at least one codeword of weight d min exists. Sometimes, we are not content with this single piece of information; we wish to know how many codewords have weight d min , and what are the weights of the other codewords.
With the development of communication technology, modern communication networks need to achieve ultra-high reliability to meet practical demands. Forward error correction coding has been widely used in real-world communication systems for providing reliable data transmission over noisy channels. In this thesis, we investigate the efficient performance evaluation methods for block codes over memoryless channels. Performance bounds are powerful tools to provide insights on the error performance of the coded systems. In many practical scenarios where performance bounds are not applicable (e.g., due to the unavailability of the relevant coding parameters under a given decoder), the Monte Carlo simulation is still popular despite its inefficiency, especially in the low error probability regime. As a variance-reduction technique, importance sampling (IS) can significantly reduce the sample size needed for reliable estimation. We firstly examine the problem of efficient performance evaluation of linear block codes over binary symmetric channels (BSCs). Contrary to the conventional wisdom, we prove that for BSCs, all bounds based on Gallager’s first bounding technique, including the famous union bound, are not asymptotically tight for all possible choices of the Gallager region. By proposing the so-called input demodulated-output weight enumerating function (IDWEF) of a code, asymptotically tight maximum-likelihood decoding upper and lower bounds for BSCs are derived. For the cases that coding parameters are absent or suboptimal decoders are applied, we propose an efficient IS estimator by deriving the optimal IS distribution of the Hamming weight of the error vector. In addition, the asymptotic percentage saving of the proposed IS estimator over the state-of-the-art IS estimator is characterized in terms of the sample size. Its accuracy in predicting the efficiency of the proposed IS estimator is verified by extensive computer simulation. We further examine the IS estimator design and efficiency analysis problems for the additive white generalized Gaussian noise (AWGGN) channel. The generalized Gaussian distribution is an effective model to describe real-world systems with impulsive noise. It can be specialized to the Laplace and Gaussian distributions depending on the choices of the decay rates of the heavy tail. By deriving the optimal IS distribution on the one-dimensional space mapped from the observation space, we present a general framework for designing IS estimators for linear block codes over a memoryless continuous channel. We apply the framework to the AWGGN channel and propose a noise-magnitude-domain IS (NM-IS) estimator. As an efficiency measure, the asymptotic IS gain of the proposed estimator is derived in a multiple integral as the signal-to-noise ratio (SNR) tends to infinity. Specifically, we reduce the gains of both Laplace and Gaussian noise cases to a one-dimensional integral. In addition, by limiting the use of the union bound to an L1-norm sphere with an optimized radius, we derive the sphere bound for the additive white Laplace noise channel. Simulation results show the efficiency of the proposed IS estimator in terms of the sample size and verify the accuracy of the prediction of the derived IS gain. Besides, we specialize the proposed framework to further improve the efficiency of the IS simulation based on the side information of the modulation. For the AWGGN channel with binary phase-shift keying (BPSK) modulation, we propose an enhanced NM-IS estimator, which outperforms the NM-IS estimator in efficiency. For the additive white Gaussian noise channel with M-PSK modulation, we propose an angular domain IS estimator and provide its efficiency analysis in terms of the asymptotic IS gain. Furthermore, in the performance evaluation of communication systems, it is often seen that researchers are more interested in the required SNR at a target error probability Pe than the other way around. The conventional solution to this Pe-targeted SNR estimation problem involves interpolation of simulated error probabilities at multiple handpicked SNRs for estimating the required SNR at the target error probability. We propose a Pe-targeted iterative IS estimator that solves the problem algorithmically. The proposed method can achieve the same efficiency (in terms of sample size) as estimating the error probability at the required SNR and allow the confidence interval of the SNR estimate to be obtained.
We consider the problem of decoding a linear block code used over a binary symmetric channel with crossover probability p when the goal of the decoding is to minimize the probability of an information bit error. In general, there will be several different decoding strategies, each corresponding to the optimal rule over some range p/spl isin/[p/sub i-1/, p/sub i/]. Each of these strategies can be implemented by standard array with the appropriate choice of coset leaders that are not in general of minimum weight. We give optimal rules for selecting coset leaders in the first region [0, p/sub 1/] and the region [p/sub max/, 1/2] for any linear mapping between information bits and codewords. The determination of the optimal rule has been an open problem for small p. The rule for p>p/sub max/ is important because for many codes, when p is fixed the optimal rule for any coset corresponds to either the optimal rule for small p, or the optimal rule for p near 1/2. The optimal decoding strategy varies with the generator matrix used in general, but it is demonstrated that the optimal generator matrix is separable when d/sup /spl perp//>2. The results are shown to extend in a natural way to the case where the goal is to minimize the probability that arbitrary collections of information bits are in error. We examine some specific codes, and compare the performance under optimal decoding to the performance of previously suggested strategies.>