folder structure ideas

[iantaylor-NOAA]

I posted an issue recently #68 suggesting that it would be easier to compare across {sa4ss} assessment repositories (and borrow code from each other) if there were a common default folder structure that folks followed.
@kellijohnson-NOAA wisely suggested that GitHub Discussions might be the best way to share ideas on that topic.

Here's what I'm thinking about for petrale based on chatting with @kellijohnson-NOAA, looking at other {sa4ss} repos and working on lingcod last time. Please comment below if you think there are better ways to structure any of this so I can think about copying what you are doing.

[structure edited 28 Dec 2022 to reflect some of the discussion below and 4 Jan 2023 to use "documents"]

petrale/
.gitignore [includes stuff like /data-raw/, /models/, *.pdf, *ppt]
README.md
DESCRIPTION [optional file if you want repo to work as an R package]
NAMESPACE [optional file if you want repo to work as an R package]
+-- doc/
│   +-- figures/ [all the figures that are not standard r4ss figures associated with a specific model]
│   +-- tables/ [all the tables that are not exec summary tables]
│   +-- 00a.Rmd
│   +-- ...
+-- documents/ [required name for files posted to github pages, such as those created by pkgdown, may only exist in a branch]
+-- data/
|   +-- blah.rda [processed data that is not confidential]
+-- data-raw/ [in .gitignore, shared via Google Drive]
|   +-- blah.csv
+-- models/ [in .gitignore, shared via Google Drive]
|   +-- 2019.001.001_base/
|   +-- ...
|   +-- 2023.099.001_base/
|       +-- plots/
|       +-- tables/ [exec summary tables in .csv format]
|       +-- tex_tables/ [exec summary tables in .tex format created by sa4ss::read_model()]
|       +-- starter.ss
|       +-- Report.sso
|       +-- ...
+-- presentations/
|   +-- STAR_presentation.Rmd
|   +-- STAR_responses.Rmd 
|   +-- STAR_presentation.pdf [in .gitignore, shared via Google Drive]
|   +-- STAR_presentation.ppt [in .gitignore, shared via Google Drive]
+-- R/
    +-- some_script.R
    +-- fancy_function.R

[brianlangseth-NOAA]

@iantaylor-NOAA I like this structure. Simple. My concern is with the presentations folder. You provide markdown scripts, which if that is the content, I agree. However, if .ppt files are to go there, there may be concerns with file size. General best practice is to not include pdf's, so I would think excluding .ppt, and presentations compiled as .pdfs may be preferable. Long story short, have presentation scripts, or exclude the presentation folder within our github repos.

[kellijohnson-NOAA]

We could easily create a function to add powerpoint files from all directories to the gitignore file.

[kellijohnson-NOAA]

I think we should add "docs" so users know that creating a package down website is also an option and to not confuse it with doc that stores the assessment document.

[kellijohnson-NOAA]

We could use usethis::use_directory() to populate the directory structure just like usethis::create() for creating R packages. Where it wouldn't overwrite any existing directory but it would set everything else up.

[kellijohnson-NOAA]

What are people's thoughts on also adding the typical files necessary to make a directory an R package while we are at it? I found this helpful for lingcod because it would ensure that users had all of the necessary dependencies via the DESCRIPTION file and that all functions were easily loadable.

[chantelwetzel-noaa]

Hmmm. I am less than enthused about these suggestions, perhaps I just need to warm up to it. Currently, in my repo for next year's assessments I created a "docs" folder to align closer with the sa4ss suggested set-up (used to call this folder write_up so not a big deal) and I called this folder 'docs' because it will contain multiple assessment documents. I would be fairly unhappy if there was both a doc and docs folder automatically created since the similarity of these names will surely result in me repeatedly opening the wrong folder. However, I am coming from the perspective that I have little to no inclination to create a R package. I will admit that I think my current approach could be improved for clarity ease of future reproducibility but am not yet convinced that a R package is the right way to do that.

[iantaylor-NOAA]

@chantelwetzel-noaa, I see your point and was continually annoyed by the "doc" vs "docs" with lingcod. The problem is that github gives us no choice but to use "docs" for github pages:

"the source folder can either be the root of the repository (/) on the source branch or a /docs folder on the source branch. " (from https://docs.github.com/en/pages/getting-started-with-github-pages/configuring-a-publishing-source-for-your-github-pages-site#about-publishing-sources.

To avoid that conflict, I suppose we could move our folder from "doc" to "write_up" (like https://github.com/chantelwetzel-noaa/dover_sole_2021/tree/master/write_up)

@kellijohnson-NOAA, maybe the R package files could be optional. I like the idea of usethis::use_directory(), but it's also not that hard to manually add these things.

[chantelwetzel-noaa]

Alternatively, we could just write out the whole folder making it clearer "document" or I could just continue to be the old crank yelling "get off my lawn".

[kellijohnson-NOAA]

Pages can also be deployed in a branch and do not need to take up the docs folder. Though, I am unsure if it would still be confusing.

[brianlangseth-NOAA]

@kellijohnson-NOAA I agree with Ian and think having a package structure as the default be optional. Although I was on lingcod, and think the package structure was helpful, that was and still is something I could not do on my own. The result would be a lot of questions to you, and frustration on my part over organization.

[iantaylor-NOAA]

This is a follow-up to note that I edited the folder structure diagram in my original post above (link) to reflect the discussion in this thread and also changed "doc" to "documents" since that's what @chantelwetzel-noaa did in https://github.com/pfmc-assessments/copper_rockfish_2023/tree/main/documents (which makes more sense than the "document" suggestion above when you have multiple areas with separate documents). Also, check out this beautiful website if you haven't already: https://pfmc-assessments.github.io/copper_rockfish_2023/.

I'm going to implement that folder structure for petrale but obviously everyone is free to choose what they want to do.