Computationally Efficient Estimation of False Discovery Rate Using Sequential Permutation p-Values

Abstract:

This talk simultaneously addresses two important aspects of multiple testing. The first is computational expense, particularly when each of the multiple tests is a Monte Carlo procedure, e.g. permutation test. The second is controlling the number of false discoveries made among the multiple tests. To reduce computation time for each of the multiple Monte Carlo tests, this article employs the sequential analysis of Besag and Clifford (1991) who propose stopping sampling from the empirical distribution if there is little evidence against the null hypothesis rather than continuing to sample until a prespecified size is reached. Secondly, the false discovery rate (FDR) is a way to quantify the reliability of significant findings found from testing multiple hypotheses and most procedures use an estimate of the number of true null hypotheses. Nettleton et al. (2006) propose a histogram based estimator of the number of true null hypotheses that estimates the number of p-values that follow a continuous uniform distribution. We extend this to handle the discrete `pseudo uniform' p-values produced by the procedure Besag and Cliffford (1991) to obtain approximate control of the FDR.

All are welcome.