Is there a file Size limit for uploading with REST API?

[wkpalan]

I am trying to upload a large file 1-2GB into Zendo sandbox. I keep getting a EPIPE error when I submit my request.

data = {"filename":'test.tar'}
files ={'file' : ("test.tar",open('test.tar','rb'),{'Expires': '0'})}

r = requests.post('https://sandbox.zenodo.org/api/deposit/depositions/274851/files',
              params={'access_token': ACCESS_TOKEN},files=files,data=data)

The error I get is as follows

ConnectionError: ('Connection aborted.', error("(32, 'EPIPE')",))

Is this due to a size limit imposed on the uploads?

[frankois]

Indeed there is a size limit old API, which supports up to 100MB per file. The new API supports up to 50GB per file and you can see how to use it at the following links:

• using curl: https://gist.github.com/slint/2263b2212743a68b2851b2cb7865675e
• using Python: https://gist.github.com/slint/92e4d38eb49dd177f46b02e1fe9761e1

This API is not completely stable, so it might change a bit in the future. As soon as we will develop a stable version, we will update the documentation here: http://developers.zenodo.org/#deposition-files

Let me know should you need further help.

[wkpalan]

Thank you for the reply @frankois. I will check the new API out.

[abubelinha]

I am a bit confused when I read about the new and old APIs: I can't find them.

I have just discovered Zenodo and this is the only documentation I found. But it doesn't say it refers to new or old API:
https://developers.zenodo.org

So I have not clear if both old and new versions keep working at the same time, having different url access ... or there is just one API access, so extant python applications might stop working due to changes and new functionalities.
Please @frankois could you clarify this?

Is there a published changelog somewhere, so we can know when the each new API feature was released?
I mean: if I find a python script somewhere, how can I tell (by its date) if it was written for the old or new API?

---

More specifically, I was trying to use this Python interface to Zenodo API:
https://github.com/SiLeBAT/zenodo-python

But, as I reported there, my uploads are failing because files in the repository get corrupted.
I suspect it could be related with this change in the API mentioned by @frankois .
Could please some expert review this line in their code:

data = {'file': open(file_path, 'rb')}

And compare against the link above provided by @frankois , where @slint says:
"We pass the file object (fp) directly to request as 'data' for stream upload".
So, I guess the line above referred to old API, and perhaps it should be corrected to something like this?

data = open(file_path, 'rb')

[johanneskoester]

Dear @frankois. Any update on stabilizing the API for bigger file sizes? Since your post is quite old, has this effort been abandoned?

[ppanero]

Hi @johanneskoester, some changes have been made since then to improve the files submission (e.g. offloading, instead of passing the files through the python app, config, etc.). Does the issue still persists?

[johanneskoester]

I actually haven't checked, I was just referring to the blue note here at the top:
https://developers.zenodo.org/#delete

It says that there is still a 100MB limit.