Information theoretic bounds for capacity and bayesian risk

Author

Ian Zieder, New Jersey Institute of TechnologyFollow

Author ORCID Identifier

0009-0000-0027-4472

Document Type

Dissertation

Date of Award

5-31-2024

Degree Name

Doctor of Philosophy in Electrical Engineering - (Ph.D.)

Department

Electrical and Computer Engineering

First Advisor

Alexander Haimovich

Second Advisor

Ali Abdi

Third Advisor

Joerg Kliewer

Fourth Advisor

Martina Cardone

Fifth Advisor

Alex R. Dytso

Abstract

In this dissertation, the problem of finding lower error bounds on the minimum mean-squared error (MMSE) and the maximum capacity achieving distribution for a specific channel is addressed. Presented are two parts, a new lower bound on the MMSE and upper and lower bounds on the capacity achieving distribution for a Binomial noise channel. The new lower bound on the MMSE is achieved via use of the Poincare inequality. It is compared to the performance of the well known Ziv-Zakai error bound. The second part considers a binomial noise channel and is concerned with the properties of the capacity-achieving distribution. In particular, for the binomial channel, it is not known if the capacity-achieving distribution is unique since the output space is finite (i.e., supported on integers 0, . . . , n) and the input space is infinite (i.e., supported on the interval [0, 1]), and there are multiple distributions that induce the same output distribution. This paper shows that the capacity-achieving distribution is unique by appealing to the total positivity property of the binomial kernel. In addition, we provide upper and lower bounds on the cardinality of the support of the capacity-achieving distribution. Specifically, an upper bound of order n/2 is shown, which improves on the previous upper bound of order n due to Witsenhausen. Moreover, a lower bound of order ?n is shown. Finally, additional information about the locations and probability values of the support points is established.

Recommended Citation

Zieder, Ian, "Information theoretic bounds for capacity and bayesian risk" (2024). Dissertations. 1763.
https://digitalcommons.njit.edu/dissertations/1763