Minutes
Here, you will find minutes from ss3sim developer meetings organized by date. Dates in the table of contents are linked to their corresponding headers and minutes below. The following is a template for each correspondence you wish to document:
Subject - Type of correspondence - Comma-separated participant list
- Add date to table of contents using yyyy-mm-dd
- Write content in either paragraph or bullet-point format
- Commit with the following leading text minutes(yyyy-mm-dd): Subject and brief summary
Conditional age at length - email, call - kellijohnson-NOAA, k-doering-NOAA
- sample sizes in ss3sim are assumed to be numbers of fish. In a real assessment, the input sample size for marginal age and length comps would be number of tows, because we assume fish within a tow are not independent, but the tows are. In ss3sim, we assume all fish samples are independent (a simplification of reality b/c no spatial structure)
- In sample_calcomp, we should change to also sampling lengths (independent of the sample_lcomp function). The user would input sample sizes for lengths and for ages, where you need to sample a large number of lengths than ages. We probably then need to combine these with any other lengths sampled, if done - if on the same line, then need to do a weighted average by the sample size. Or, if SS can read multiple lines for the same year and fleet, can just add these as a new line
Conditional age at length - in person - kellijohnson-NOAA, k-doering-NOAA
- code in CAL branch should be merged into development once k-doering-NOAA checks a few potential issues (number of fish v. number of tows in sample sizes in sample_calcomp, what happens when sample size of fish is less than 1?, check that errors are given for dirichlet MN CAL sampling, if rebinning with CAL requested)
- Start exploring the question: how does sample size inputs/data weighting affect growth estimation when CAL is used?
- Use same OM with CAL, marginal ages, marginal lengths generated. Set seed so each scenario uses the same datasets.
- Change input sample sizes in EM to match or mismatch with OM
- Change data weighting method in EM
- Look at error in the growth parameter estimation.
- Explore other interesting topics that come up, and cap number of scenarios at ~ 15ish. Want to capture the extremes and best case scenarios.
- Adding multiple species for ss3sim will be considered in Feb. By end of Feb, plan for a new release of ss3sim and r4ss.
Conditional age at length, tests and examples, issue #254 - in person - kellijohnson-NOAA, k-doering-NOAA
- Kelli is continuing work on issue #254 and we discovered how to make the forecast.ss file for cod EM years all relative to the end year (use -999 for start year and 0 for end yr) so that the forecast files does not need to be manipulated when the number of years in the EM changes.
- We discussed how to split up testthat tests and examples (which are all really tests), but didn't come to any conclusions.
- Kelli mentioned that examples can be wrapped in
\dontshow{}within examples, which might be useful if we want to include code in the examples for developers only (e.g., adding in expectations?). -
devtools::test()or evendevtools::check()does not give informative feedback sometimes. However, the argumentfilterintest()can be helpfule. For example, for testthat tests in a file named test-ss3simfun.R, you can rundevtools::test(filter = "ss3simfun")to run just the tests in that script. This may help get more informative feedback when a lot of tests are broken so can be a helpful strategy. - Conditional age at length still needs some work, but in general, the philosophy (if using) should be to add in as many expected values as the user can possibly want, get expected values, then only sample from the values that the user specifies in the case files. We also need to change the Lbin_lo and Lbin_hi so that they are the same for both bins. Once we get the code working, there are plenty of potential questions that can be asked, given that conditional age at length does not have clear best practices (although some very useful simulation studies have been done (e.g., Lee et al. 2019)
- also discussed work on
r4ss::SS_readctl()done in K.D.'s fork. This work should be added into r4ss soon, as it is a significant improvement over what is currently available. Likely, it would enable ss3sim to useSS_readctl()instead of relying onreadLines()andgrep(), which would be an improvement.
Conditional age at length - Google Chat - kellijohnson-NOAA, k-doering-NOAA
We want to add back capabilities to simulate CAAL data and use CAAL in the EM into ss3sim, but how to do this needs to be considered carefully. There are (at least) 2 problems to consider:
- Sampling from expected values from Stock Synthesis is not the technically correct way to do this. The correct way would be to sample from the population, but sampling from the population instead of expected values may be difficult.
- Even though they are often treated as such, tows are not completely independent. It would be good to have the capabilities to not treat tows as independent when sampling from the operating model (OM) to add more realism.
Before generating code to add back in CAAL capabilities, we should consider code in the sampling branch, A. Punt's FORTRAN implementation of simulating CAAL data, and the past ss3sim code (using ss 3.24), as well as the issues identified.
Testing - In person - iantaylor-NOAA, k-doering-NOAA, kellijohnson-NOAA
Met to discuss an email thread initiated by k-doering-NOAA about Travis-CI and her experience with SSMSE, which led to multiple topics about testing code.
- Tests are a good thing even if it appears that something is working (e.g., k-doering-NOAA wrote new tests for old functions in ss3sim and found bugs that no one knew existed)
- Don't worry about slowing the package down by writing 'checks' inside of functions
- Tests are formal tests to see if checks work, where checks are inside of functions
- No package is perfect but it is good to have goals
- In theory all packages should have tests
- ss3sim should have tests that run SS
- Examples also count as tests! It is one potential way of doing integrative testing. They should all work, even if enclosed in
\dontrun{}or\donttest{}
Package uncoupling
- Should r4ss be split into multiple packages; there is a lot to maintain in r4ss and people have a hard time remembering what all of the functions are; more functions should be made without @output
- What should constitute a package and when should things be split up?
Testing of things within VLAB will require Jenkins and Travis-CI is only free on publicly available repositories. kellijohnson-noaa talked to Todd Hay, suggested by k-doering-NOAA, and he said that both Will and Sarah know Jenkins and would be happy to help set up Stock Synthesis in this capacity.
Project boards - Email - k-doering-NOAA, kellijohnson-NOAA
Need to separate issues (user driven) vs. development notes by using project boards instead of just issues for both sets of topics. Separating the two will, hopefully, decrease the number of users dropping off of the repository because of too many emails. Lastly, there needs to be a single place where anyone can look back and refresh their memory on what was said between developers during meetings and maybe also to summarize email chains.
- issues should be for code developments that are in the works, user questions, and found bugs
- project boards should be for organization of who is doing what and what will be done in the future, focused on keeping development organized with clear deadlines and assigned tasks
-
wiki Minutes will document minutes from developer meetings and email chains
- kellijohnson-NOAA tried using a project board, but it was a failure
- kellijohnson-NOAA decided on using the developer portion of the wiki to store notes because they are easier to read here than in a project board
- a time line will help keep kellijohnson-NOAA on track and give other developers a clear guide on when things should be completed