Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider using Qiskit Serverless for cron job notebook testing #1323

Open
Tracked by #1326
Eric-Arellano opened this issue May 8, 2024 · 1 comment
Open
Tracked by #1326

Comments

@Eric-Arellano
Copy link
Collaborator

In our cron job, we run every notebook that submits jobs:

run: tox -- --write --only-submit-jobs

But the jobs might take longer than GH Actions' 6 hour limit when there is a long queue. Notably, we cannot batch the different jobs because they are distinct notebooks so using with Batch is not viable. We now use concurrency with asyncio, but we still can exceed the time limit.

So, instead, @frankharkins had the great idea of using Qiskit Serverless. On day 1, we'd submit all the notebooks to Qiskit Serverless. On day 2, a new cron job would run that retrieves the results from Qiskit Serverless.

Question: what workflow do we want for local runs?

@frankharkins
Copy link
Member

Putting this on hold for a bit as the action is actually doing OK at the moment and we might be able to get higher priority if it starts being a problem again. I'm worried serverless would add yet another layer of things to go wrong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

2 participants