On reconstructing species trees from gene trees in term of duplications and losses

Abstract

This paper studies various issues of reconstructing a species tree from gene trees under the duplication and mutation costs. First, the properties of the duplication and mutation costs are studied. We prove a less obvious fact that the duplication cost satisfies the triangle inequality. We also present a linear time algorithm for enumerating all the losses with respect to all the duplications associated with the least common ancestor mapping from a gene tree to a species tree, which answers a problem raised recently in [4]. Second, the complexity of finding an optimal species tree from gene trees is investigated. We prove that the problem is NP-complete for both the duplication and mutation costs. Both proofs are non-trivial. The concept of reconciled trees was introduced by Goodman et al. and formalized by Page for visualizing the relation between gene and species trees. Here, constructing a best reconciled tree for gene trees is proved to be NP-complete. We also propose a general reconstructing problem and show it to be NP-complete even for the best-known nni distance. Finally, we define another efficiently computable metric based on the duplication cost and prove that the problem of finding an optimal species tree from gene trees can be approximated with factor 2 in polynomial time under the new metric. Using this approximation result, we propose a heuristic method for finding an optimal species tree from gene trees under the duplication or mutation cost.