One-shot achievable rate region for source coding when coded side information is available at the decoder (source coding with a helper) is proposed. The achievable region proposed is in terms of conditional smooth max Rényi entropy and smooth max Rényi divergence. Asymptotically (in the limit of large block lengths) this region is quantified in terms of spectral-sup conditional entropy rate and spectral-sup mutual information rate. In particular, it coincides with the rate region derived in the limit of unlimited arbitrarily distributed copies of the sources.
One-shot achievable rate region for source coding when coded side information is available at the decoder (source coding with a helper) is proposed. The achievable region proposed is in terms of conditional smooth max Rényi entropy and smooth max Rényi divergence. Asymptotically (in the limit of large block lengths) this region is quantified in terms of spectral-sup conditional entropy rate and spectral-sup mutual information rate. In particular, it coincides with the rate region derived in the limit of unlimited arbitrarily distributed copies of the sources.
A source coding system with side information at the decoder is a typical multiterminal source coding system where output sequences of two sources are independently encoded, but a decoder recovers only one output sequence from two codewords. Since Wyner, Ahlswede and Körner independently investigated this system, we call it as the WAK coding system. This paper investigates the ε-achievable rate region of the WAK coding system which allows the probability of error within a fixed tolerance ε(∈ (0, 1)), and clarifies the ε-achievable rate region for correlated general sources in terms of the smooth max-entropy and the smooth max Rényi divergence. To this end, we show a new one-shot converse theorem for the WAK coding system, and a one-shot covering lemma which is a refined version of Warsi's result. Then, combining these results, we clarify the ε-achievable rate region of the WAK coding system.
One-shot analogues for various information theory results known in the asymptotic case are proven using smooth min and max Rényi divergences. In particular, we prove that smooth min Rényi divergence can be used to prove one-shot analogue of the Stein's lemma. Using smooth min Rényi divergence we prove a special case of packing lemma in the one-shot setting. Furthermore, we prove a one-shot analogue of covering lemma using smooth max Rényi divergence. We also propose one-shot achievable rate for source coding under maximum distortion criterion. This achievable rate is quantified in terms of smooth max Rényi divergence.
This study considers the unconditional smooth Rényi entropy, the smooth conditional Rényi entropy proposed by Kuzuoka [\emph{IEEE Trans.\ Inf.\ Theory}, vol.~66, no.~3, pp.~1674--1690, 2020], and a new quantity which we term the conditional smooth Rényi entropy. In particular, we examine asymptotic expansions of these entropies when the underlying source with its side-information is stationary and memoryless. Using these smooth Rényi entropies, we establish one-shot coding theorems of several information-theoretic problems: Campbell's source coding, guessing problems, and task encoding problems, all allowing errors. In each problem, we consider two error formalisms: the average and maximum error criteria, where the averaging and maximization are taken with respect to the side-information of the source. Applying our asymptotic expansions to the derived one-shot coding theorems, we derive various asymptotic fundamental limits for these problems when their error probabilities are allowed to be non-vanishing. We show that, in non-degenerate settings, the first-order fundamental limits differ under the average and maximum error criteria. This is in contrast to a different but related setting considered by the present authors (for variable-length conditional source coding allowing errors) in which the first-order terms are identical but the second-order terms are different under these criteria.
This study considers the unconditional smooth Rényi entropy proposed by Renner and Wolf [ASIACRYPT, 2005], the smooth conditional Rényi entropy proposed by Kuzuoka [IEEE Trans. Inf. Th., 66(3), 1674–1690, 2020], and a novel quantity which we term theconditional smooth-⋆entropy.The latter two quantities can be specialized to the first in the absence of side-information. We explore the operational roles of these smooth Rényi entropies by establishing one-shot coding theorems for several information-theoretic problems, including Campbell’s source coding problem, the Arıkan–Massey guessing problem, and the Bunte–Lapidoth task encoding problem. We consider these problems in cases where the errors are non-vanishing and for each problem, we consider two error formalisms: the average and maximum error criteria, where the averaging and maximization are taken with respect to the side-information. Using the one-shot coding theorems, we conclude that Kuzuoka’s smooth conditional Rényi entropy and the conditional smooth-⋆ entropy are the solutions to the problems involving the average and maximum error criteria, respectively. Furthermore, we examine asymptotic expansions of these entropies when the underlying source with its side-information is stationary and memoryless. Applying our asymptotic expansions to the one-shot coding theorems, we derive various fundamental limits for these problems. We show that, under non-degenerate settings, the first-order fundamental limits differ under the average and maximum error criteria. This is in contrast to a different but related setting considered by the present authors [IEEE Trans. Inf. Th., 66(12), 7565–7587, 2020], for variable-length conditional source coding allowing errors, in which the first-order terms are identical but the second-order terms are different under these error criteria.
In the last few years, there has been a great interest in extending the information-theoretic scenario for the non-asymptotic or one-shot case, i.e., where the channel is used only once. We provide the one-shot rate region for the distributed source-coding (Slepian-Wolf) and the multiple-access channel. Our results are based on defining a novel one-shot typical set based on smooth entropies that yields the one-shot achievable rate regions while leveraging the results from the asymptotic analysis. Our results are asymptotically optimal, i.e., for the distributed source coding they yield the same rate region as the Slepian-Wolf in the limit of unlimited independent and identically distributed (i.i.d.) copies. Similarly for the multiple-access channel the asymptotic analysis of our approach yields the rate region which is equal to the rate region of the memoryless multiple-access channel in the limit of large number of channel uses.
A novel definition of the conditional smooth Rényi entropy, which is different from that of Renner and Wolf, is introduced. It is shown that our definition of the conditional smooth Rényi entropy is appropriate for providing lower and upper bounds on the optimal guessing moment in a guessing problem where the guesser is allowed to stop guessing and declare an error. Further a general formula for the optimal guessing exponent is presented. In particular, a single-letterized formula for a mixture of i.i.d. sources is obtained. It is also shown that our definition is appropriate to characterize the optimal exponential moment of the codeword length in the problem of source coding with common side-information available at the encoder and decoder under a constraint on the probability of a decoding error.