Service Broker API: making ch ch ch changes

The Service Broker API for CloudFoundry is documented at https://docs.cloudfoundry.org/services/api.html. This document can be confusing to new developers to CloudFoundry, so here's a rundown of what it is that this document describes and the workflow by which we make changes to it.

What is the Service Broker API?

This is a contract that CloudController expects all service brokers to adhere to. This is not a description of endpoints within CloudController, but a description of actions that CloudController takes against a service broker.

Each operation described is a request that CloudController makes, with a route, request body JSON, response status codes, and response body JSON.

As such, changes to the API documentation should be careful to avoid ambiguity. For instance, in some versions of the documentation, request fields are labeled as "optional." This is ambiguous, and false. If a field is documented for a request body, then when we make that request, that field is always included. A service broker may choose not to care about that, but that is outside the scope of our responsibility.

CloudFoundry has a change policy for service brokers, as follows:

• Existing endpoints and fields will not be removed or renamed.
• New optional endpoints, or new HTTP methods for existing endpoints, may be added to enable support for new features.
• New fields may be added to existing request/response messages. These fields must be optional and should be ignored by clients and servers that do not understand them.

Who owns the Service Broker API documentation?

The documentation lives in its own repo in GitHub. It is supposedly owned by the "docs team," the identities of whom are shrouded in mystery. In actuality, the content is owned by the CAPI team; we just have to go through an arcane workflow in order to make changes according to changes we make to CloudController.

The Service Broker API says that it has a version. This version is arbitrary, and bumps to the version are decided upon by the developer who remembers to update the documentation after making a change to how CloudController calls out to service brokers. The version is for documentation only, since it does not exist anywhere in code.

Service Broker API acceptance tests

In CloudController, there are acceptance tests for each version of the Service Broker API. This is to ensure that we do not make any backwards-incompatible changes to functionality that existing service brokers depend upon.

These live in CloudController.

Tests for existing functionality may be overly specific to query parameters or JSON fields at the time of writing. Making changes to the code may therefor cause old tests to fail. If the changes are backwards compatible, then it may be tempting to fix the tests by including the changes. Unfortunately, doing so means that we would effectively be changing our contract for an old API version, and our tests for the version would no longer match the documentation for that version. Instead, tests must be fixed by making them forwards compatible without changing the contract. For instance:

• Tests of query params that check for exact match of the request URI, and new functionality adds a query param
  • Consider changing the test to parse the URI and check only for the query params extant in the old API version
  • The test for the new API version can check for the existence of the new query param
• Tests of JSON body that compare one hash to another, and a new field is added
  • Consider changing the test to compare the values of a specific enumeration of keys
  • The test for the new API version can check for the existence of the new field in the body
• Tests may check for exact string matches, and a backwards-compatible change adds to the text
  • Consider changing the test to use a regular expression rather than string equality

Workflow for making changes to the API documentation

• CAPI engineer changes the service broker interaction in CC (or merges a pull request that does so)
  • Old service broker acceptance tests are fixed, without making any changes to the contract for any old version
• CAPI engineer bumps the version of CloudController. This is an arbitrary number that doesn't really mean anything.
• CAPI engineer creates an acceptance test for the new Service Broker API version
• CAPI engineer makes a new version in the docs-services repo (described below)
  • in the Dependencies section for the new version, make some guesses at the release versions of cf-release, CC and CLI
  • Changes are pushed to a branch, and a pull request is started
  • In the pull request, include the fact that this pull request should not be merged
  • At some point in the future, Nick will ask the docs team to merge the pull request

Updating the version of the docs

Each version of the API is its own markdown file in docs-services, with file name pattern api-vX.X.html.md.erb. As of this pull request, the current version of the API docs are a symlink to the most recent file.

• Copy api.html.md.erb to api-vX.X.html.md.erb
• Change _version_table.html.erb
  • Add a link to the previous version
  • Change the current version
• Change the symlink of api.html.md.erb to point to your new file
• Edit the new file
  • Add your new stuff
  • Change references to the previous version
  • Replace the Changes Since vX.X section
  • Guess at the versions this change will be released in. Update the Dependencies section.
• Edit the changelog at v2-api-changelog.html.md.erb