Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve data / code organization in the archive #457

Open
alee opened this issue May 29, 2019 · 0 comments
Open

improve data / code organization in the archive #457

alee opened this issue May 29, 2019 · 0 comments
Labels
need input needs input from comses editors / directorate / members

Comments

@alee
Copy link
Member

alee commented May 29, 2019

We should improve how we handle input data dependencies and code.

A continual source of confusion for model authors requesting peer review is how to refer to the data files the submit (in the data slot).

Right now code is placed in

./code/model.py

and data is placed in

./data/quux.asc

In order for model.py to access quux.asc it needs to use the relative path ../data/quux.asc. Frequently source code will use ./data/quux.asc or something along those lines though.

We could try adding a symlink in the ./code/ directory to the data directory if the ./code/data/ directory doesn't already exist.

E.g.,

.
├── code
│   ├── data -> ../data
│   └── model.py
└── data
    └── quux.asc

This would let model authors continue to refer to their data via the relative path ./data/quux.asc. Thoughts @comses/editors?

Then the technical part about putting a symlink in the zip archive we build is a bit hairy but seems possible: https://stackoverflow.com/questions/35782941/archiving-symlinks-with-python-zipfile

@alee alee added the need input needs input from comses editors / directorate / members label May 29, 2019
@alee alee changed the title consider building a data symlink in the download archive to ../data in the /code directory improve data / code archive organization Sep 14, 2020
@alee alee changed the title improve data / code archive organization improve data / code organization in the archive Sep 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need input needs input from comses editors / directorate / members
Projects
None yet
Development

No branches or pull requests

1 participant