Title:E-values as unnormalized weights in multiple testing

Abstract:Standard weighted multiple testing methods require the weights to deterministically add up to the number of hypotheses being tested (equivalently, the average weight must be unity). We show that this normalization is not required when the weights are not constants, but are themselves e-values obtained from independent data. This could result in a massive increase in power, especially if the non-null hypotheses have e-values much larger than one. More broadly, we study how to combine an e-value and a p-value, and design multiple testing procedures where both e-values and p-values are available for every hypothesis (or one of them is available for an implied hypothesis). For false discovery rate (FDR) control, analogous to the Benjamini-Hochberg procedure with p-values (p-BH) and the recent e-BH procedure for e-values, we propose two new procedures: ep-BH and pe-BH, which have valid FDR guarantee under different dependence assumptions. These procedures are designed based on several admissible combining functions for p/e-values, which also yields methods for family-wise error rate control. We demonstrate the practical power benefits with a case study with RNA-Seq and microarray data. We also collect several miscellaneous results, such as a tiny but uniform improvement of e-BH, a soft-rank permutation e-value, and the use of e-values as masks in interactive multiple testing.