[BUG] JsonDecodeError

[Maarten-vd-Sande]

Describe the bug
My colleague @Rebecza is trying to download a single-cell ATAC-seq dataset and uses pysradb to get some metadata (seq2science), and managed to find a JsonDecodeError 🐛 . It's a list of approx 750 ENA samples, the strange this is the JsonDecodeError appears with the full list, but when split up in smaller lists it seems to work...

To Reproduce
I put it on colab, not sure if the link is working
https://colab.research.google.com/drive/1bC2WiA63JJnWYZew0pk6iovk537vQzaU?usp=sharing

[saketkc]

Thanks for creating a reproducible example. My guess is a long list of ids is causing SRA to timeout. I would suggest processing it in batches just the way you have done while I figure out if it can indeed be fixed.

[kpj]

I ran into the same problem and also solved it with the same approach (iterating over chunks of the accession list).

For querying, this seems to be implemented for SraSearch already:

pysradb/pysradb/search.py

Lines 757 to 760 in c23d4a7

	for i in range(0, len(uids), SRA_SEARCH_GROUP_SIZE):
	current_uids = ",".join(
	uids[i : min(i + SRA_SEARCH_GROUP_SIZE, len(uids))]
	)

Would it make sense to do the same for SRAweb as well? It seems like all terms are simply joined so far:

pysradb/pysradb/sraweb.py

Lines 252 to 253 in c23d4a7

	if isinstance(term, list):
	term = " OR ".join(term)