Covariance of pairwise differences on a multi-species coalescent tree and implications for FST
Research output: Contribution to journal › Journal article › Research › peer-review
Standard
Covariance of pairwise differences on a multi-species coalescent tree and implications for FST. / Guerra, Geno; Nielsen, Rasmus.
In: Philosophical Transactions of the Royal Society B: Biological Sciences, Vol. 377, No. 1852, 20200415, 2022.Research output: Contribution to journal › Journal article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - Covariance of pairwise differences on a multi-species coalescent tree and implications for FST
AU - Guerra, Geno
AU - Nielsen, Rasmus
N1 - Publisher Copyright: © 2022 Royal Society Publishing. All rights reserved.
PY - 2022
Y1 - 2022
N2 - The multi-species coalescent (MSC) provides a theoretical foundation for modern phylogenetics and comparative population genetics. Its theoretical properties have been heavily studied but there are still aspects of the MSC that are largely unknown, including the covariances in pairwise coalescence times, which are fundamental for understanding the properties oFSTatistics that combine data from multiple species, such as the fixation index (FST). The major contribution of this study is the derivation and implementation of exact expressions for the covariances of pairwise coalescence times under phylogenetic models with piecewise constant changes in population size, assuming no gene flow after species divergence. We use these expressions to derive the variance in average pairwise differences within and between populations. We then derive approximations for the expectation and bias of a sequence-based estimator of FST, a commonly used genetic measurement of population differentiation, when it is applied to a non-recombining region of the genome. We show that the estimator of FST is generally biased downward. A freely available software package is provided, STCov, to calculate the mean, variances and covariances in coalescence times presented here under user-defined piecewise-constant species trees. This article is part of the theme issue Celebrating 50 years since Lewontin's apportionment of human diversity'.
AB - The multi-species coalescent (MSC) provides a theoretical foundation for modern phylogenetics and comparative population genetics. Its theoretical properties have been heavily studied but there are still aspects of the MSC that are largely unknown, including the covariances in pairwise coalescence times, which are fundamental for understanding the properties oFSTatistics that combine data from multiple species, such as the fixation index (FST). The major contribution of this study is the derivation and implementation of exact expressions for the covariances of pairwise coalescence times under phylogenetic models with piecewise constant changes in population size, assuming no gene flow after species divergence. We use these expressions to derive the variance in average pairwise differences within and between populations. We then derive approximations for the expectation and bias of a sequence-based estimator of FST, a commonly used genetic measurement of population differentiation, when it is applied to a non-recombining region of the genome. We show that the estimator of FST is generally biased downward. A freely available software package is provided, STCov, to calculate the mean, variances and covariances in coalescence times presented here under user-defined piecewise-constant species trees. This article is part of the theme issue Celebrating 50 years since Lewontin's apportionment of human diversity'.
KW - covariance
KW - F
KW - Multi-species coalescent
KW - pairwise differences
KW - population differentiation
U2 - 10.1098/rstb.2020.0415
DO - 10.1098/rstb.2020.0415
M3 - Journal article
C2 - 35430886
AN - SCOPUS:85128380309
VL - 377
JO - Philosophical Transactions of the Royal Society B: Biological Sciences
JF - Philosophical Transactions of the Royal Society B: Biological Sciences
SN - 0962-8436
IS - 1852
M1 - 20200415
ER -
ID: 307333945