Equivalence of GSEApy and Broad GSEA with phenotype permutations

[bobzimmermann-D4C]

Hello,

Thanks for the awesome tool!

I noticed that in the README.md you show that GSEApy prerank nearly perfectly reproduces the output of the Broad implementation of GSEA. I was also able to reproduce this result. (See the left panel below.)

I was wondering if this is also the intention when running standard GSEA with permutation_type="phenotype". So far, I am not able to produce p-values closely similar to Broad GSEA in this mode. Is there a setting I am missing? Here is my call to gsea():

gseapy.gsea(
        'data/GSE22863_lung_adenocarcinoma_Expression.txt',
        'data/REACTOME.gmt',
        'data/GSE22863_lung_adenocarcinoma_Class.cls',
        permutation_num=5000,
        min_size=1,
        max_size=20000,
    )

and the corresponding Broad command:

java -cp gsea-3.0.jar \
      -res data/GSE22863_lung_adenocarcinoma_Expression.txt \
      -cls data/GSE22863_lung_adenocarcinoma_Class.cls \
      -gmx data/REACTOME.gmt \
      -out . \
      -params gsea/gsea_params_phenotype.txt

I've attached the gsea_params_phenotype.txt file. The result is shown in the right panel of the plot below.

gsea_params_phenotype.txt

[zqfang]

I don't compare the gene ranking algorithm between the two tools because, in the end, both Prerank and GSEA use the same enrichment algorithm for the final calculation.

For gsea, there are some default parameters that differ between the two tools.

for example, the default metric is

• GSEA(Broad) -> metric: Signal2Noise
• GSEApy -> method: log2_ratio_of_classes

other minor filtering may affect the finalresults. for instance, the two tools used different inputs after filtering the expression table or gene set file.

[bobzimmermann-D4C]

Thank you for your answer! Actually, although the documentation states that log2_ratio_of_classes is the default, it is in fact signal_to_noise. This can be seen here. I tested setting the method explicitly to signal_to_noise, but this had no effect on the outcome (see two attached images). Are there any plans to amend this issue?

Again, thanks!

[zqfang]

Yeah you'er right. The docs were not updated. That's my bad.

Somehow, I had difficulty finding the same permutation procedure for simulating the phenotype permutation in the Java version. I can't extract the final phenotype permutated gene expression table.

However, it's just random sampling for the group labels (the current implementation of GSEApy and GSEA-R).