Maximum Entropy and the Glasses You Are Looking Through
This work provides a theoretical clarification for researchers in statistics and machine learning, though it appears incremental as it builds on existing critiques of MaxEnt.
The authors addressed the representation dependence problem of the Maximum Entropy Principle by interpreting it in game-theoretic terms and making a formal distinction between different ways of applying Maximum Entropy distributions, which allows avoiding this issue in many cases.
We give an interpretation of the Maximum Entropy (MaxEnt) Principle in game-theoretic terms. Based on this interpretation, we make a formal distinction between different ways of {em applying/} Maximum Entropy distributions. MaxEnt has frequently been criticized on the grounds that it leads to highly representation dependent results. Our distinction allows us to avoid this problem in many cases.