This letter deals with the Slepian-Wolf coding problem for general sources. The second-order achievable rate region is derived using quantity which is related to the smooth max-entropy and the conditional smooth max-entropy. Moreover, we show the relationship of the functions which characterize the second-order achievable rate region in our study and previous study.
This letter deals with the Slepian-Wolf coding problem for general sources. The second-order achievable rate region is derived using quantity which is related to the smooth max-entropy and the conditional smooth max-entropy. Moreover, we show the relationship of the functions which characterize the second-order achievable rate region in our study and previous study.
One-shot achievable rate region for source coding when coded side information is available at the decoder (source coding with a helper) is proposed. The achievable region proposed is in terms of conditional smooth max Rényi entropy and smooth max Rényi divergence. Asymptotically (in the limit of large block lengths) this region is quantified in terms of spectral-sup conditional entropy rate and spectral-sup mutual information rate. In particular, it coincides with the rate region derived in the limit of unlimited arbitrarily distributed copies of the sources.
In the last few years, there has been a great interest in extending the information-theoretic scenario for the non-asymptotic or one-shot case, i.e., where the channel is used only once. We provide the one-shot rate region for the distributed source-coding (Slepian-Wolf) and the multiple-access channel. Our results are based on defining a novel one-shot typical set based on smooth entropies that yields the one-shot achievable rate regions while leveraging the results from the asymptotic analysis. Our results are asymptotically optimal, i.e., for the distributed source coding they yield the same rate region as the Slepian-Wolf in the limit of unlimited independent and identically distributed (i.i.d.) copies. Similarly for the multiple-access channel the asymptotic analysis of our approach yields the rate region which is equal to the rate region of the memoryless multiple-access channel in the limit of large number of channel uses.
Two typical fixed-length random number generation problems in information theory are considered for general sources. One is the source resolvability problem and the other is the intrinsic randomness problem. In each of these problems, the optimum achievable rate with respect to the given approximation measure is one of our main concerns and has been characterized using two different information quantities: the information spectrum and the smooth Rényi entropy. Recently, optimum achievable rates with respect to f-divergences have been characterized using the information spectrum quantity. The f-divergence is a general non-negative measure between two probability distributions on the basis of a convex function f. The class of f-divergences includes several important measures such as the variational distance, the KL divergence, the Hellinger distance and so on. Hence, it is meaningful to consider the random number generation problems with respect to f-divergences. However, optimum achievable rates with respect to f-divergences using the smooth Rényi entropy have not been clarified yet in both problems. In this paper, we try to analyze the optimum achievable rates using the smooth Rényi entropy and to extend the class of f-divergence. To do so, we first derive general formulas of the first-order optimum achievable rates with respect to f-divergences in both problems under the same conditions as imposed by previous studies. Next, we relax the conditions on f-divergence and generalize the obtained general formulas. Then, we particularize our general formulas to several specified functions f. As a result, we reveal that it is easy to derive optimum achievable rates for several important measures from our general formulas. Furthermore, a kind of duality between the resolvability and the intrinsic randomness is revealed in terms of the smooth Rényi entropy. Second-order optimum achievable rates and optimistic achievable rates are also investigated.
This study considers the unconditional smooth Rényi entropy proposed by Renner and Wolf [ASIACRYPT, 2005], the smooth conditional Rényi entropy proposed by Kuzuoka [IEEE Trans. Inf. Th., 66(3), 1674–1690, 2020], and a novel quantity which we term theconditional smooth-⋆entropy.The latter two quantities can be specialized to the first in the absence of side-information. We explore the operational roles of these smooth Rényi entropies by establishing one-shot coding theorems for several information-theoretic problems, including Campbell’s source coding problem, the Arıkan–Massey guessing problem, and the Bunte–Lapidoth task encoding problem. We consider these problems in cases where the errors are non-vanishing and for each problem, we consider two error formalisms: the average and maximum error criteria, where the averaging and maximization are taken with respect to the side-information. Using the one-shot coding theorems, we conclude that Kuzuoka’s smooth conditional Rényi entropy and the conditional smooth-⋆ entropy are the solutions to the problems involving the average and maximum error criteria, respectively. Furthermore, we examine asymptotic expansions of these entropies when the underlying source with its side-information is stationary and memoryless. Applying our asymptotic expansions to the one-shot coding theorems, we derive various fundamental limits for these problems. We show that, under non-degenerate settings, the first-order fundamental limits differ under the average and maximum error criteria. This is in contrast to a different but related setting considered by the present authors [IEEE Trans. Inf. Th., 66(12), 7565–7587, 2020], for variable-length conditional source coding allowing errors, in which the first-order terms are identical but the second-order terms are different under these error criteria.
We consider a secure lossless source coding problem with a rate-limited helper. In particular, Alice observes an i.i.d. source Xnand wishes to transmit this source loss-lessly to Bob at a rate Rx. A helper, say Helen, observes a correlated source Ynand transmits at a rate Ryto Bob. A passive eavesdropper can observe the coded output of Alice. The equivocation ¿ is measured by the conditional entropy H(Xn\ Jx)/n, where Jxis the coded output of Alice. We first completely characterize the rate-equivocation region for this secure source coding model, where we show that Slepian-Wolf type coding is optimal. We next study two generalizations of this model and provide single-letter characterizations for the respective rate-equivocation regions. In particular, we first consider the case of a two-sided helper where Alice also has access to the coded output of Helen. We show that for this case, Slepian-Wolf type coding is suboptimal and one can further decrease the information leakage to the eavesdropper by utilizing the side-information at Alice. We finally generalize this result to the case when there are both secure and insecure rate-limited links from Helen and additional uncoded side informations Wnand Znavailable at Bob and Eve, respectively. For this model, we provide a complete characterization of the rate-equivocation region when Yn¿ Xn¿ (Wn, Zn) forms a Markov chain.
A source coding system with side information at the decoder is a typical multiterminal source coding system where output sequences of two sources are independently encoded, but a decoder recovers only one output sequence from two codewords. Since Wyner, Ahlswede and Körner independently investigated this system, we call it as the WAK coding system. This paper investigates the ε-achievable rate region of the WAK coding system which allows the probability of error within a fixed tolerance ε(∈ (0, 1)), and clarifies the ε-achievable rate region for correlated general sources in terms of the smooth max-entropy and the smooth max Rényi divergence. To this end, we show a new one-shot converse theorem for the WAK coding system, and a one-shot covering lemma which is a refined version of Warsi's result. Then, combining these results, we clarify the ε-achievable rate region of the WAK coding system.