[epic] revamp how we test notebooks

[Eric-Arellano]

Concurrency / queues

Beta Give feedback

1. Switch to dedicated testing hub #1150
   +0
2. Use asyncio to parallelize notebook testing #1201
   +0
3. Consider using Qiskit Serverless for cron job notebook testing #1323
   infra 🏗️ on hold ⏸️ +2
4. Asyncio warning on Windows #1303
   infra 🏗️ +1
   
5. VQE tutorial takes too long to run #1229
   bug 🐛 content 📄 tutorial 📒 +3
   

   

Usability

Beta Give feedback

1. Change how we specify Python versions for testing #1207
   +0
2. Upload artifacts from cron job #1159
   +0
3. Add default posarg to tox.ini #1049
   +0
4. Replace Squeaky CI advice with custom advice #897
   +0
5. Improve file filtering in CI #734
   +0
6. Upload notebooks in CI #527
   +0
7. Require all notebooks to be classified #1319
   infra 🏗️ +1
   
8. More explicitly direct people to run notebooks with tox #1239
   infra 🏗️ +1
9. Work out how to handle notebook testing on forks #967
   infra 🏗️ +1
10. Automate updating requirements for notebook testing #920
    infra 🏗️ +1

Better testing

Beta Give feedback

1. Test notebooks on pull request #483
   +0
2. Add new cell tag features to README #1096
   +0
3. Mock hardware when testing notebooks #1184
   +0
4. Run notebooks that submit jobs in cron job #1047
   +0
5. Report notebook warnings #642
   +0
6. Ignore local testing warnings when testing with fake backend #1333
   +0
7. Ignore MPL cache warnings #1362
   +0
8. Check for errors when executing initialization code #1373
   +0

Closed source tutorials support

Beta Give feedback

1. Make notebook tester into package #1208
   +0
2. Add tutorials to notebook tests #1155
   +0

Code quality

Beta Give feedback

1. Refactor nb-tester config and args #1351
   +0
2. Refactor nb-tester execution process #1346
   +0
3. Pull notebook classification out into separate test #1397
   +0