Skip to content

Update dependencies and pyproject.toml #432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from
Open

Conversation

katie-lamb
Copy link
Collaborator

@katie-lamb katie-lamb commented Jul 4, 2025

In the process of running a data update I thought it would be helpful to update some dependencies and simplify where we name dependencies. Initially it was spread between requirements.txt and setup.py. But setup.py was recently removed. I thought it would be simpler if everything was in pyproject.toml. This PR makes the following changes:

  • Removes PUDL as a dependency since we're now just accessing PUDL tables directly from AWS
  • Updates some key dependencies to their latest version - namely python and pandas
  • Removes requirements.txt in favor of putting all dependencies under pyproject.toml. It seems like the main advantage of requirements.txt was having one place where Docker could install all development dependencies, but I instead had Docker just download the dbcp package + all the dependencies in pyproject.toml. I'm not that familiar with Docker so I'm not sure if this is problematic or not the standard convention.

Remaining To Do:

  • Add more to the DBCP documentation google doc about comparison notebooks and running them from within the docker container
  • Check on things like dependabot to see if it's keeping dependencies updated
  • Separate out development dependencies from package dependencies
  • Debug ETL with updated dependencies -- namely python 3.12 and pandas 2.0
  • Compare data warehouse and data mart to see if anything has changed from newer versions of Python and pandas

@katie-lamb katie-lamb changed the base branch from main to dev July 4, 2025 00:08
@katie-lamb katie-lamb self-assigned this Jul 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant