Contribution Review Criteria

When contributing to VisionEval, the following set of questions will be asked of the contribution. These review criteria help ensure contributions to VisionEval satisfy the Goals and Objectives of the system. Make sure to review the Developer Orientation before making a submittal. Questions are categorized by review type. The automated test system also provides some helpful information where identified.

To submit a contribution for review, issue a pull request with a comment introducing your contribution. The comment should include a brief overview, responses to the questions, and pointers to related information. The entire submittal should ideally be self contained so the documentation of the methods should be in the R package as well.

Review Question	Test System Help	Review Team - Software	Review Team - Documentation	Review Team - Methods
1. Does it contain all the elements that are required by the VisionEval [[system specifications	https://github.com/gregorbj/VisionEval/blob/master/api/model_system_design.md]]?		x	x
2. Why is it better, and/or different than [[existing modules	https://github.com/gregorbj/VisionEval/tree/master/sources/modules]]? Does it do good science and provide documentation justifying this claim? Is it consistent with good practice in strategic modeling? How might it overlap with existing modules?			x
3. Is the module documentation complete? Does it include documentation of model estimation, algorithms, and instructions for using?			x	x
4. If the module allows the estimation of regional parameters, does it provide default data, does it have clear documentation of what the estimation data needs to be and how it is to be formatted, and does it include proper data specifications to ensure that the user’s input data are correct?		x	x	x
5. Is it based on geographic definitions that are consistent with the model system definitions?			x	x
6. Does the module compute quickly enough and provide documentation justifying this claim?		x	x
7. Does it includes all source files and data? If a contributed module does not include all source data, it should include a minimal example data file for testing and so it is clear what data structure is needed to run the module. It should also include clear instructions on how to fetch the data and/or a clear explanation of why non-included data is confidential and contact information for data owners.	x	x	x
8. Does the module only call R code and packages that work on all operating systems? If the code includes any non-R code (e.g. FORTRAN, C++) will that code compile on all operating systems?	x	x
9. Is it licensed with the VisionEval license that allows the code to be freely distributed and modified and includes attribution so that the ‘provenance’ of the code can be tracked?	x
10. Does it only interact with the computing environment by returning a properly structured list to the framework (i.e. it does not modify the global environment, does not read or write files, and only calls framework functions that are allowed)?	x	x
11. Does it include regression tests to enable checking that consistent results will be returned when updates are made to the framework and/or R programming environment?	x	x
12. Does it include [[sufficient test coverage	Automated-Testing]] and test data? Does it pass the ‘testModule’ test which validates that it will run correctly in the model system?	x	x