Comment on "Why does deep and cheap learning work so well?" [arXiv:1608.08225]
This comment clarifies a theoretical misunderstanding in machine learning, but it is incremental as it defends an existing result without introducing new methods or data.
The authors address claims by Lin and Tegmark that a mapping between deep belief networks and the variational renormalization group is invalid, showing these claims are incorrect due to a misunderstanding of the variational RG procedure and that the counterexample is compatible with the original mapping.
In a recent paper, "Why does deep and cheap learning work so well?", Lin and Tegmark claim to show that the mapping between deep belief networks and the variational renormalization group derived in [arXiv:1410.3831] is invalid, and present a "counterexample" that claims to show that this mapping does not hold. In this comment, we show that these claims are incorrect and stem from a misunderstanding of the variational RG procedure proposed by Kadanoff. We also explain why the "counterexample" of Lin and Tegmark is compatible with the mapping proposed in [arXiv:1410.3831].