Header menu link for other important links
X
Projection Theorems for the Renyi Divergence on $\alpha$-Convex Sets
, Igal Sason
Published in
2016
Volume: 62
   
Issue: 9
Pages: 4924 - 4935
Abstract
This paper studies forward and reverse projections for the Renyi divergence of order $\alpha$ ∈ (0, ∞) on $\alpha$-convex sets. The forward projection on such a set is motivated by some works of Tsallis et al. in statistical physics and the reverse projection is motivated by robust statistics. In a recent work, van Erven and Harremoes proved a Pythagorean inequality for Renyi divergences on $\alpha$-convex sets under the assumption that the forward projection exists. Continuing this study, a sufficient condition for the existence of a forward projection is proved for probability measures on a general alphabet. For $\alpha$ ∈ (1, ∞), the proof relies on a new Apollonius theorem for the Hellinger divergence and for $\alpha$ ∈ (0,1), the proof relies on the Banach-Alaoglu theorem from the functional analysis. Further projection results are then obtained in the finite alphabet setting. These include a projection theorem on a specific $\alpha$-convex set, which is termed an $\alpha$-linear family, generalizing a result by Csiszar to $\alpha$ ≠ 1. The solution to this problem yields a parametric family of probability measures, which turns out to be an extension of the exponential family and it is termed an $\alpha$-exponential family. An orthogonality relationship between the $\alpha$-exponential and $\alpha$-linear families is established and it is used to turn the reverse projection on an $\alpha$-exponential family into a forward projection on an $\alpha$-linear family. This paper also proves a convergence result of an iterative procedure used to calculate the forward projection on an intersection of a finite number of $\alpha$-linear families.
About the journal
JournalIEEE Transactions on Information Theory
ISSN00189448