Near-Optimality of the Minimum Average Redundancy Code for Almost All Monotone Sources

Abstract

Consider the source coding problem of finding the optimal code, in the sense of average redundancy, for the class of monotone sources with n symbols. The solution of this problem, known as the M code, is the Huffman code for the average distribution of the monotone sources. In this paper, we evaluate the average redundancy of the M code (on the class of monotone sources), and compare it with that of the Huffman code. It is demonstrated that for large n, although the M code is a fixed code (i.e., the codewords are independent of the symbol probabilities) for all monotone sources, its average redundancy is very close to that of the Huffman code. Moreover, it is shown that when n is large, the M code is a near-optimal code not only in the sense of average redundancy, but also the redundancy of almost all monotone sources. In particular, the redundancy of the M code converges in probability to its average value (≅0.029). As a result, the maximum redundancy of the M code, which can be as large as log n - log ln n, rarely occurs.