One of the fundamental problems of statistics and data science is identifying and measuring dependence. This problem dates back to the works of Bravais, Galton, and Pearson in the 18th century on dependence measure design, and to the work of Rényi in the late 1950s on axiomatizing the desired properties of such measures. For discrete random variables, categorical dependence measures—primarily those based on Shannon's mutual information and entropy, and maximal correlation—are valid choices when only the information content is important.
However, when some possible underlying physical interpretation is of the essence, other measures need to be sought after. Consequently, much effort has been put into both the design and property axiomatization of such dependence measures when the dependence strength is dictated by the inference quality with respect to some metric.
In this talk, I will first propose a new set of natural axioms that reflect desired innate geometric properties. I will show that, in fact, none of the existing dependence measures satisfies this set of axioms and has a known feasible evaluation algorithm. Finally, I will propose a new computationally efficient dependence measure that satisfies all the proposed axioms and compare its performance to that of classical dependence measures such as maximal correlation and correlation ratio, as well as recently proposed measures such as xicor (Chatterjee JASA ‘21), distance correlation (Székely et al. Ann. Stat. ‘07), and maximal information coefficient (Reshef et al. Science ‘11).
Joint work with Elad Domanovitz and Yoad Nitzan.