429 Client Error: TOO MANY REQUESTS Error

[CCranney]

Hi,

I have run into the following error using CPTAC. In many ways, I suspect this is my own fault - I am running SLURM jobs on HPC that regularly call cptac functionality, and suspect I ran too many such jobs simultaneously. I have run this code before, and it has generally worked in the past.

cptac error: Requesting data failed with the following error: 429 Client Error: TOO MANY REQUESTS for url: https://zenodo.org/api/records/7897498 (/home/cranneyc/dataModuleMethods/RnaToProteinDataModule/scripts/SHAP/0_SHAP_run.py, line 2)

I make this issue mostly because I thought the cptac package effectively downloads the required datasets after the first use and stores them locally. This makes the first use slow, but every subsequent use fast. In my head, there would then be no reason to access zenodo.com after that first download. Is there a GET call to zenodo.com every time, regardless of whether or not the data is already downloaded? If so, is this a required feature?

[CCranney]

Issue #39 also highlighted this error, but it wasn't investigated because no code was produced. I generally call cptac with the following code, if that helps. Again, however, I suspect the code itself works fine, it's just a problem if you run it numerous times within a small time frame.

def get_cptac_mod(cancerType):
    if cancerType == 'brca':
        return cptac.Brca()
    elif cancerType == 'ccrcc':
        return cptac.Ccrcc()
    elif cancerType == 'coad':
        return cptac.Coad()
    elif cancerType == 'gbm':
        return cptac.Gbm()
    elif cancerType == 'hnscc':
        return cptac.Hnscc()
    elif cancerType == 'lscc':
        return cptac.Lscc()
    elif cancerType == 'luad':
        return cptac.Luad()
    elif cancerType == 'ov':
        return cptac.Ov()
    elif cancerType == 'pdac':
        return cptac.Pdac()
    elif cancerType == 'ucec':
        return cptac.Ucec()
    else:
        raise Exception('cancer type not found in CPTAC datasets')

then, later:

        #...
        self.proteome = cptacMod.get_proteomics(source=source).fillna(0)
        self.transcriptome = cptacMod.get_transcriptomics(source=source).fillna(0)
        #...