Developer documentation

README.md

@@ -35,6 +35,9 @@ _Not sure which Endpoint to use? We have a [page](search-workflows.md) for that_
 - [Software Requirements](requirements.md) A list of all software requirements for Pelias
 
 ### Pelias project development
+- [Developer guide](/development/guide.md). Guide to submitting changes to Pelias code.
+- [Project infrastructure](/development/project_infra.md). Guide to Pelias build tools, continuous integration, Docker image, etc.
+- [Contributor Status](/development/contributor.md). Repeat contributors to Pelias are granted additional permissions, explained here!
 - [Release notes](release-notes.md). See notable changes in Pelias over time
 - [Development roadmap](development/roadmap.md). Plans for future improvements to Pelias. Read this to see what's coming and how you can help
 

development/contributor.md

@@ -0,0 +1,16 @@
+# Pelias Contributor
+
+We want to make it as easy as possible for people to contribute to the Pelias
+project.
+
+For repeat contributors, we generally extend an invitation to join the [Pelias
+contributors GitHub team](https://github.com/orgs/pelias/teams/contributors).
+
+Joining this team allows you to create new branches and pull requests
+_directly_ in repositories that are a part of the Pelias org. Hopefully this
+saves some time and hassle!
+
+Because contributors can create branches in Pelias repositories, our [project
+infrastruture](./project_infra.md) will also ensure that Docker images are
+automatically built and generated for those changes. This can greatly speed
+development in combination with our [Docker projects](https://github.com/pelias/docker).

development/guide.md

@@ -0,0 +1,118 @@
+# Pelias Developer Guide
+
+So, you want to help out with code changes to Pelias? Great!
+
+This guide is meant to help you make effective, efficient changes that can be
+accepted into the Pelias project.
+
+## General Guidelines
+
+### Use our test suites to help you
+
+All Pelias modules come with a test suite to help ensure your changes do not break existing
+functionality. We encourage you to extend the test suites for new features or bug fixes you submit.
+
+Generally, these test suites fall into two categories: unit tests and functional (sometimes called
+end-to-end or integration) tests.
+
+Unit tests should run in just a few seconds, while functional tests may take a few minutes.
+
+You can always run just a projects unit tests with:
+
+```
+npm test
+```
+
+You can run the functional tests with one of the following, depending on the repository
+
+```
+npm run functional
+npm run integration
+npm run end-to-end
+```
+
+The functional tests may take a few minutes to run, which is why we've separated them out.
+
+### Take advantage of our pre-commit hooks
+
+Most of our repositories use the [pre-commit](https://www.npmjs.com/package/pre-commit) Node.js
+module to run various checks before allowing you to commit your code. Please follow these pre-commit hooks!
+
+Currently our pre-commit hooks generally test:
+
+- That the unit tests pass
+- That there are no missing or extraneous NPM dependencies
+- That the code passes our linter rules.
+
+Note that if you really need to, you can bypass them with `git commit --no-verify`. This can be
+useful if you want to test something out real quick, but plan to clean it up later before submitting
+the change.
+
+### Releases are driven by commit messages
+
+We manage Pelias releases through commit message contents using [Semantic Release](https://github.com/semantic-release/semantic-release).
+
+If you'd like, you can help us make that task easier by following our [release
+tagging guide](./tagging_releases.md). But don't worry too much about it, the core
+team can always make any needed tags before merging your pull request.
+
+## Best practices for Pull Requests
+
+### Reach out to us first before doing a lot of work
+
+Nothing makes us more sad than saying no to a pull request that someone worked hard on.
+
+If you're ever in doubt about whether or not making a change is worthwhile, ask us first. Open an
+issue, email the core team, etc.
+
+We'll discuss what makes sense and why, give you context about our ideas for
+the project going forward that might influence your work, and discuss any plans
+you have before you proceed.
+
+We're also happy to review "draft" or unfinished PRs before they are done, so we can give input on
+whether or not a particular approach is looking promising.
+
+### Keep your pull requests small
+
+We always appreciate small pull requests that focus on a specific change. This makes it easier to
+test and review.
+
+Whenever possible, we support your efforts to split one large pull request into several smaller
+ones.
+
+### Rebase against master
+
+If other changes have been made while your Pull Request is in progress, please occasionally update
+your PR to rebase against those new changes.
+
+This helps ensure that we can review your changes effectively, that your changes will merge as
+intended when the time comes, and, **most importantly**, that the project commit history is clean so
+that its easy to figure out how and why changes came in when working in the future.
+
+You can generally do this with `git rebase origin master`.
+
+### Allow changes to your pull requests
+
+GitHub allows maintainers to make changes to a pull request created by another user. Please [enable
+this
+functionality](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork) as we often want to make little changes to your pull request rather than go back and forth over them.
+
+## Commonly used tools
+
+As a large project with many smaller components, we try to keep the Pelias
+development process as consistent as possible.
+
+This section lists various tools and utilities that are common in Pelias. Where
+possible, use these tools.
+
+If one of these tools isn't suitable for a particular purpose, ideally we would
+settle on a new option for the entire project.
+
+| Functionality | Pelias Standard |
+| --- | --- |
+| Javascript Linting | [JSHint](https://github.com/jshint/jshint/) |
+| Test framework | [Tape](https://github.com/substack/tape) |
+| Node.js stream library | [through2](https://github.com/rvagg/through2) |
+| General utility library | [lodash](https://github.com/lodash/lodash) |
+| SQLite library | [better-sqlite3](https://github.com/JoshuaWise/better-sqlite3) |
+

development/project_infra.md

@@ -0,0 +1,24 @@
+# Pelias Project Infrastructure
+
+The Pelias project utilizes lots of outside services to aid in development.
+
+## Continuous Integration
+
+All branches and pull requests opened on Pelias projects run our test suites through TravisCI.
+
+This includes both unit tests, which run quickly, and the longer-running functional tests.
+
+## Docker images
+
+Any branch created in a Pelias GitHub repository will have several [Docker images created](https://github.com/pelias/ci-tools#build-docker-images).
+
+The images will be tagged in the following formats, including the Pelias repository, the branch name, the date the image was built, and the full git commit hash.
+
+```
+pelias/repository:branch
+pelias/repository:branch-date-commit
+```
+
+## Future work: full end to end testing
+
+In the future, we hope to build systems for running complete end to end tests of a small Pelias build using our [docker projects](https://github.com/pelias/docker). For some ideas, see our discussion on [city acceptance tests](https://github.com/pelias/pelias/issues/718).

development/tagging_releases.md

@@ -0,0 +1,82 @@
+## Tagging releases with Semantic Release
+
+We use the wonderful [Semantic Release](https://github.com/semantic-release/semantic-release)
+project to manage releasing new versions of all our projects.
+
+Semantic Release looks for a [certain commit message format](https://github.com/semantic-release/semantic-release#commit-message-format) to decide when to release.
+
+To help you decide which one to use, we'll list our general guidelines below. Feel free to include
+these in your commit messages if you feel confident. If not, the core team will take care of it.
+
+The general format is:
+
+```
+type(component): Message
+```
+
+Where `type` is one of the below commit message types, `component` is a description of what area of
+the codebase affects, and `Message` is a normal commit message.
+
+#### Chores
+
+[Chores](https://blog.carbonfive.com/2020/02/24/what-are-these-chores-doing-in-my-backlog/) are
+anything that don't affect the behavior of Pelias code, but help keep the project a healthy and
+efficient place to work.
+
+Another way to look at chores is they are changes to the codebase that would not warrant a new
+release at all.
+
+Good examples include:
+
+- Fixing typos in documentation
+- Updating settings in our CI infrastructure
+- Removing dead or unused code
+
+A commit message for a chore looks like this:
+
+```
+chore(CI): Remove deprecated Travis CI configuration option
+```
+
+#### Bug fixes
+
+Bug fixes correct existing errors in the behavior of code.
+
+A bugfix commit looks something like this:
+
+```
+fix(query): Ensure Elasticsearch queries handle empty value
+```
+
+#### Features
+
+Features are the good stuff! They add new behavior that helps the project or improve how it behaves.
+
+A feature commit usually looks like this:
+
+```
+feat(sql): Improve performance of SQL queries
+```
+
+#### Breaking Changes
+
+Any of the above types of commits can also be marked as a breaking change. Breaking changes are used
+to signal to users or modules that depend on the software that a backwards-incompatible change has
+been made. In Pelias, this will trigger a new _major version_ release.
+
+Breaking changes are created by ensuring that the term `BREAKING CHANGE:` occurs in the _body_ of
+the commit message. This means you can't trigger a breaking change with something like `git commit
+-m "my commit message"`, you have to run `git commit` and let it open your editor to write the
+commit message.
+
+Also, note the colon at the end of the string. We've often missed breaking changes because we only
+wrote `BREAKING CHANGE`.
+
+A breaking change commit looks like this:
+
+
+```
+feat(services): Add support for libpostal service
+
+BREAKING CHANGE: this project now requires the Libpostal Service to be present in order to work
+```