Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test individual archivers #45

Open
2 tasks
zschira opened this issue Jan 30, 2023 · 1 comment
Open
2 tasks

Test individual archivers #45

zschira opened this issue Jan 30, 2023 · 1 comment

Comments

@zschira
Copy link
Member

zschira commented Jan 30, 2023

Background

Right now the individual archivers (inheriting from AbstractDatasetArchiver) are largely untested. There's been discussion in #43 about the appropriate way to test these archivers. In the long run, we will want to have new archives tested through nightly builds to fully verify the entire workflow. This, however will likely be future work once we are actually generating new archives regularly, and have decided how we want to integrate those archives into PUDL. In the immediate future, there are likely helpful tests that we can create at the unit/integration level to perform some basic sanity checking.

Scope

This issue will only track the development of unit/integration tests, and will leave full integration into the nightly builds process for future work. Things we can check for at this level include:

  • Are all of the links we attempt to download files from valid?
  • Is the structure of the archive what is expected by PUDL (partitions are formatted correctly, and filenames look like what we'd expect
@zaneselvans
Copy link
Member

@zschira Are there unit or integration tests that you imagine falling under this issue that wouldn't be covered by what's described in #70?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Icebox
Development

No branches or pull requests

3 participants