Yiyuan Luo

Dionysis Arvanitakis, Vaggos Chatziafratis, Yiyuan Luo et al.

Quartet Reconstruction, the task of recovering a phylogenetic tree from smaller trees on four species called \textit{quartets}, is a well-studied problem in theoretical computer science with far-reaching connections to statistics, graph theory and biology. Given a random sample containing $m$ noisy quartets, labeled by an unknown ground-truth tree $T$ on $n$ taxa, we want to output a tree $\widehat T$ that is \textit{close} to $T$ in terms of quartet distance and can predict unseen quartets. Unfortunately, the empirical risk minimizer corresponds to the $\mathsf{NP}$-hard problem of finding a tree that maximizes agreements with the sampled quartets, and earlier works in approximation algorithms gave $(1-\eps)$-approximation schemes (PTAS) for dense instances with $m=Θ(n^4)$ quartets, or for $m=Θ(n^2\log n)$ quartets \textit{randomly} sampled from $T$. Prior to our work, it was unknown how many samples are information-theoretically required to learn the tree, and whether there is an efficient reconstruction algorithm. We present optimal results for reconstructing an unknown phylogenetic tree $T$ from a random sample of $m=Θ(n)$ quartets, corrupted under the Random Classification Noise (RCN) model. This matches the $Ω(n)$ lower bound required for any meaningful tree reconstruction. Our contribution is twofold: first, we give a tree reconstruction algorithm that, not only achieves a $(1-\eps)$-approximation, but most importantly \textit{recovers} a tree close to $T$ in quartet distance; second, we show a new $Θ(n)$ bound on the Natarajan dimension of phylogenies (an analog of VC dimension in multiclass classification). Our analysis relies on a new \textit{Quartet-based Embedding and Detection} procedure that identifies and removes well-clustered subtrees from the (unknown) ground-truth $T$ via semidefinite programming.

Dionysis Arvanitakis, Vaggos Chatziafratis, Yiyuan Luo et al.

We study phase transitions and algorithms for refuting CSPs arising in hierarchical clustering (as well as ranking, and ordinary CSPs). Here, $n$ variables are assigned to leaves of a tree, so as to satisfy $m$ constraints, specifying evolutionary relationships. Two canonical $NP$-hard optimization problems are Triplet and Quartet Reconstruction, where the input consists of triplets $xy|z$ or quartets $xy|zw$, and the goal is to find a tree $T^*$ maximizing agreement with constraints. Our main results are (as density $λ=m/n$ increases): 1. We show the existence and precisely locate the sharp threshold $λ^*\approx1.2277$ for Triplets (via closed-form solution). To the best of our knowledge, this is the first sharp threshold for the broad family of Phylogenetic CSPs. Moreover, we give a lower and upper bound for Quartets. 2. We provide strong refutation algorithms that certify that $val(T^*)\le5/9 + ε$, where $val(T^*)$ is the fraction of constraints satisfied by the (unknown) optimal tree. For triplets, our algorithm succeeds w.h.p if $m =Ω(n)$, and for quartets if $m = Ω(n^{3/2})$. 3. We obtain strongest possible refutations at slightly larger densities (for triplets $m=O(n^{3/2}\log ^3n)$, for quartets $m=O(n^2)$): we certify that $T^*$ is no better than a random assignment, i.e., $val(T^*)\le 1/3+ε$. In fact, we obtain strongest possible refutations for finite-alphabet CSPs with or without negations. Our refutations above are instantiations of our general theorem that applies more broadly to Phylogenetic and Ordering CSPs (and all CSPs failing to support $t$-wise independence), and generalizes the current algorithmic frontier on refuting random CSPs~\citep{allen2015refute}. A crucial difference here, unlike Boolean CSPs, is that there are no negated variables, so prior works relying on negations -- a source of randomness -- do not apply.