We recommend managing python and requirement versions with pyenv
and venv
- Install Python 3
- Install pip
- Install Jupyter-lab or Jupyter Classic
- Run Jupyter-Lab with
jupyter-lab
and Jupyter Classic withjupyter notebook
from the command line.
.../ose-equity-tool-etl/ $ pip install -r requirements.txt
- Create and/or configure the
config/
file for the desired category. - Edit the
DB_EDDT_TARGET
notebooks/etl.ipynb
file to point to the updated data source folder - Run the
etl.ipynb
notebook through Jupyter (Classic or Lab). - (Optional) Serve and test the generated files locally
- Serve the files by running
docker compose up
within this repository - Test the files by running the
equity-tool
repo/application with the following settings- In env variables set
NEXT_PUBLIC_DO_SPACE_URL=http://localhost:4566
- In
src/pages/data/[geography]/[geoid]/[category]/[subgroup].tsx
setspaceFolder = "local"
- In env variables set
- Serve the files by running
- Commit the changes
- Place an annotated tag on the commit with the format
YYYY-MM-DD--v#
- The date must match the edm folder used to generate the output files
- The version should increment by integers, reflecting number of outputs generated from the same source folder
- example tag
git tag -a 2023-04-21--v1 -m 'update vintage years'
- push the tag
git push origin 2023-04-21--v1
- Copy the generated files in
/output
to the static file server (DO).
- The DO Spaces folder name must match the git tag
- Remember that to test the new tables in EDDE, you have to add the respective category to the
categories
constant in the EDDE data page.src/pages/data/[geography]/[geoid]/[category]/[subgroup].tsx