Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
feat: enable restricted access support in zenodo remote provider (#1623)
* feat: enable restricted access support in zenodo remote provider

* refer to source

* move declaration
  • Loading branch information
johanneskoester committed May 3, 2022
1 parent 73c1c9a commit 692caf9
Show file tree
Hide file tree
Showing 2 changed files with 73 additions and 8 deletions.
46 changes: 40 additions & 6 deletions docs/snakefiles/remote_files.rst
Expand Up @@ -824,10 +824,9 @@ Avoid creating uploads with too many files, and instead group and zip them to ma
# let Snakemake assert the presence of the required environment variable
envvars:
"MYZENODO_PAT"
"ZENODO_ACCESS_TOKEN"
access_token = os.environ["MYZENODO_PAT"]
zenodo = RemoteProvider(deposition="your deposition id", access_token=access_token)
zenodo = RemoteProvider(deposition="your deposition id", access_token=os.environ["ZENODO_ACCESS_TOKEN"])
rule upload:
input:
Expand All @@ -841,11 +840,46 @@ Avoid creating uploads with too many files, and instead group and zip them to ma
It is possible to use `Zenodo sandbox environment <https://sandbox.zenodo.org>`_ for testing by setting ``sandbox=True`` argument.
Using sandbox environment requires setting up sandbox account with its personal access token.
Auto
====
Restricted access
-----------------
If you need to access a deposition with restricted access, you have to additionally provide a ``restricted_access_token``.
This can be obtained from the restricted access URL that Zenodo usually sends you via email once restricted access to a deposition (requested via the web interface) has been granted by the owner.
Let ``
https://zenodo.org/record/000000000?token=dlksajdlkjaslnflkndlfnjnn`` be the URL provided by Zenodo.
Then, the ``restricted_access_token`` is ``dlksajdlkjaslnflkndlfnjnn``, and it can be used as follows:
.. code-block:: python
from snakemake.remote.zenodo import RemoteProvider
import os
# let Snakemake assert the presence of the required environment variable
envvars:
"ZENODO_ACCESS_TOKEN",
"ZENODO_RESTRICTED_ACCESS_TOKEN"
zenodo = RemoteProvider(
deposition="your deposition id",
access_token=os.environ["ZENODO_ACCESS_TOKEN"],
restricted_access_token=os.environ["ZENODO_RESTRICTED_ACCESS_TOKEN"]
)
rule upload:
input:
"output/results.csv"
output:
zenodo.remote("results.csv")
shell:
"cp {input} {output}"
Auto remote provider
====================
A wrapper which automatically selects an appropriate remote provider based on the url's scheme.
It removes some of the boilerplate code required to download remote files from various providers:
It removes some of the boilerplate code required to download remote files from various providers.
The auto remote provider only works for those which do not require the passing of keyword arguments to the
``RemoteProvider`` object.
.. code-block:: python
Expand Down
35 changes: 33 additions & 2 deletions snakemake/remote/zenodo.py
Expand Up @@ -48,7 +48,7 @@ def __init__(
keep_local=keep_local,
stay_on_remote=stay_on_remote,
provider=provider,
**kwargs
**kwargs,
)
if provider:
self._zen = provider.remote_interface()
Expand Down Expand Up @@ -121,6 +121,12 @@ def __init__(self, *args, **kwargs):
"environment at https://sandbox.zenodo.org."
)

self.restricted_access_token = None
self._restricted_access_cookies = None

if "restricted_access_token" in kwargs:
self.restricted_access_token = kwargs["restricted_access_token"]

if "sandbox" in kwargs:
self._sandbox = kwargs.pop("sandbox")
else:
Expand Down Expand Up @@ -148,9 +154,13 @@ def _api_request(
session.headers["Authorization"] = "Bearer {}".format(self._access_token)
session.headers.update(headers)

cookies = self.restricted_access_cookies

# Run query.
try:
r = session.request(method=method, url=url, data=data, files=files)
r = session.request(
method=method, url=url, data=data, files=files, cookies=cookies
)
if json:
msg = r.json()
return msg
Expand Down Expand Up @@ -189,3 +199,24 @@ def get_files(self):
)
for f in files
}

@property
def restricted_access_cookies(self):
"""Retrieve cookies necessary for restricted access.
Inspired by https://gist.github.com/slint/d47fe5628916d14b8d0b987ac45aeb66
"""
if self.restricted_access_token and self._restricted_access_cookies is None:
url = (
self._baseurl
+ f"/record/{self.deposition}?token={self.restricted_access_token}"
)
resp = self._api_request(url)
if "session" in resp["cookies"]:
self._restricted_access_cookies = resp["cookies"]
else:
raise WorkflowError(
"Failure to retrieve session cookie with given restricted access token. "
f"Is the token valid? Please check by opening {url} manually in your browser."
)
return self._restricted_access_cookies

0 comments on commit 692caf9

Please sign in to comment.