Inference of Phylogenetic Networks from Sequence Data using Composite Likelihood

Abstract While phylogenies have been essential in understanding how species evolve, they do not adequately describe some evolutionary processes. For instance, hybridization, a common phenomenon where interbreeding between two species leads to formation of a new species, must be depicted by a phylogenetic network, a structure that modifies a phylogeny by allowing two branches to merge into one, resulting in reticulation. However, existing methods for estimating networks are computationally expensive as the dataset size and/or topological complexity increase. The lack of methods for scalable inference hampers phylogenetic networks from being widely used in practice, despite accumulating evidence that hybridization occurs frequently in nature. Here, we propose a novel method, PhyNEST (Phylogenetic Network Estimation using SiTe patterns), that estimates phylogenetic networks directly from sequence data. PhyNEST achieves computational efficiency by using composite likelihood as well as accuracy by using the full genomic data to incorporate all sources of variability, rather than first summarizing the data by estimating a set of gene trees, as is required by most of the existing methods. To efficiently search network space, we implement both hill-climbing and simulated annealing algorithms. Simulation studies show that PhyNEST can accurately estimate parameters given the true network topology and that it has comparable accuracy to two popular methods that use composite likelihood and a set of gene trees as input, implemented in SNaQ and PhyloNet. For datasets with a large number of loci, PhyNEST is more efficient than SNaQ and PhyloNet when considering the time required for gene tree estimation. We applied PhyNEST to reconstruct the evolutionary relationships among Heliconius butterflies and Papionini primates, characterized by hybrid speciation and widespread introgression, respectively. PhyNEST is implemented in an open-source Julia package and publicly available at https://github.com/sungsik-kong/PhyNEST.jl .