Update README

http-tests · Dec 27, 2023 · 1008b03 · 1008b03
1 parent a5dfc7c
commit 1008b03
Showing 1 changed file with 68 additions and 44 deletions.
diff --git a/README.md b/README.md
@@ -7,17 +7,18 @@ including browsers, proxy caches and CDNs. Its public results are available at
 <!-- START doctoc generated TOC please keep comment here to allow auto update -->
 <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
 
-
 - [Goals](#goals)
 - [Installation](#installation)
   - [Installing from NPM](#installing-from-npm)
-- [Running the Test Server](#running-the-test-server)
-- [Testing Reverse Proxies and CDNs](#testing-reverse-proxies-and-cdns)
-  - [Testing from the Command Line](#testing-from-the-command-line)
-  - [Testing with Docker](#testing-with-docker)
-- [Testing Browser Caches](#testing-browser-caches)
-- [Interpreting the Results](#interpreting-the-results)
-  - [Test Results FAQ](#test-results-faq)
+- [Running tests with scripts](#running-tests-with-scripts)
+  - [Testing with docker](#testing-with-docker)
+  - [Testing browser caches](#testing-browser-caches)
+  - [Testing forward proxies](#testing-forward-proxies)
+- [Running tests with npm](#running-tests-with-npm)
+  - [Starting the test server](#starting-the-test-server)
+  - [Running the client](#running-the-client)
+- [Interpreting results](#interpreting-results)
+  - [Test results FAQ](#test-results-faq)
 - [Getting your results onto cache-tests.fyi](#getting-your-results-onto-cache-testsfyi)
 - [Creating new tests](#creating-new-tests)
 
@@ -32,12 +33,12 @@ The underlying aim is to provide a basis for discussion about how HTTP caches --
 
 In other words, **passing all of the tests currently means nothing** -- this is not a conformance test suite, it's just the start of a conversation, and a **tool to assess how a cache behaves**.
 
-Therefore, if you believe a test should change (based upon common behaviour or your interpretation of the specifications), or have additional tests, please [contribute](.github/CONTRIBUTING.md).
+Therefore, if you believe a test should change (based upon common behaviour or your interpretation of the specifications), or have additional tests, please [contribute](CONTRIBUTING.md).
 
 
 ## Installation
 
-The tests require a recent version of [NodeJS](https://nodejs.org/) (10.8.0 or greater), which includes the `npm` package manager.
+The tests require a recent version of [NodeJS](https://nodejs.org/) (14.8.0 or greater), which includes the `npm` package manager.
 
 To install the most recent source from GitHub (*recommended; things are moving fast*):
 
@@ -53,72 +54,95 @@ Alternatively, for the most recent release:
 
 > npm i --legacy-bundling http-cache-tests
 
+Note that the version in the registry is not necessarily up-to-date.
 
-## Running the Test Server
 
-First, start the server-side by running:
+## Running tests with scripts
 
-> npm run server
+A number of scripts are supplied to simplify common test scenarios.
 
-inside the directory (the repository's directory if you cloned from git, or `node_modules/http-cache-tests` if you installed from npm).
+### Testing with docker
 
-By default, the server runs on port 8000; to choose a different port, use the `--port` argument; e.g.,
+There's also a docker image, `mnot/proxy-cache-tests`, that can be used to test reverse proxy caches. Once you have docker running, you can run the CLI tests against a given proxy like this:
 
-> npm run server --port=8080
+> ./test-docker.sh squid
 
-If you want to run an HTTPS origin, you'll need to specify the `protocol`, `keyfile` and `certfile`:
+Omit the proxy name to test all available in the Docker image. To run an individual test case, try:
 
-> npm run server --protocol=https --keyfile=/path/to/key.pem --certfile=/path-to-cert.pem
+> ./test-docker.sh -i freshness-none nginx
 
-Note that the default port for HTTPS is still 8000.
 
-Make sure that the browser is not configured to use a proxy cache, and that the network being tested upon does not use an intercepting proxy cache.
+### Testing browser caches
 
+To test a browser, just point it at `https://{hostname:port}/test-browser.html` after setting up the server.
 
-## Testing Reverse Proxies and CDNs
+On OSX, you can use `test-browser.sh` to automate this:
 
-### Testing from the Command Line
+> ./test-browser.sh safari
 
-To test a reverse proxy or CDN from the command line::
+Again, omit the browser name to test all. Run a single case with:
 
-> npm run --silent cli --base=http://server-url.example.org:8000/
+> ./test-browser.sh -i freshness-none safari
 
-... using the URL of the server you want to test. This will output the test results in JSON to STDOUT, suitable for inclusion in the `results` directory. See `lib/summary.mjs` for details of how to interpret that.
+Make sure that your browsers are not configured to use a proxy cache, and that the network being tested upon does not use an intercepting proxy cache.
 
-To run a single test, use:
 
-> npm run cli --base=http://server-url.example.org:8000/ --id=test-id
+### Testing forward proxies
 
-... where `test-id` is the identifier for the test. This will output the request and response headers as seen by the client and server, along with the results. This is useful for debugging a particular failure.
+To test a forward proxy which listens on 127.0.0.1:8082, start the server:
 
+> npm run server
 
-### Testing with Docker
+and then run:
 
-There's also a docker image, `mnot/proxy-cache-tests`, that can be used to test reverse proxy caches. Once you have docker running, you can run the CLI tests against a given proxy like this:
+> HTTP_PROXY=http://127.0.0.1:8082 npm run --silent cli --base=http://127.0.0.1:8000
 
-> ./test-docker.sh squid
+or:
 
-To run an individual test case, try:
+> ./test-host.sh 127.0.0.1:8002
 
-> ./test-docker.sh -i freshness-none nginx
+Again, pass `-i` to run a specific test.
 
 
-## Testing Browser Caches
+## Running tests with npm
 
-To test a browser, just point it at `https://{hostname:port}/test-browser.html` after setting up the server.
+If you don't want to run the test scripts (see above), this section documents how to run tests directly with NPM. First, you'll need to start the test server; then, you'll need to run the client against it.
 
-On OSX, you can use `test-browser.sh` to automate this.
+### Starting the test server
 
-## Testing Forward Proxies
-To test a forward proxy which listens on 127.0.0.1:8082, start the server:
+First, start the server-side by running:
 
 > npm run server
 
-and then run:
+inside the directory (the repository's directory if you cloned from git, or `node_modules/http-cache-tests` if you installed from npm).
+
+By default, the server runs on port 8000; to choose a different port, use the `--port` argument; e.g.,
+
+> npm run server --port=8080
+
+If you want to run an HTTPS origin, you'll need to specify the `protocol`, `keyfile` and `certfile`:
+
+> npm run server --protocol=https --keyfile=/path/to/key.pem --certfile=/path-to-cert.pem
+
+Note that the default port for HTTPS is still 8000.
+
+
+### Running the client
+
+To test a reverse proxy or CDN from the command line:
+
+> npm run --silent cli --base=http://server-url.example.org:8000/
+
+... using the URL of the server you want to test. This will output the test results in JSON to STDOUT, suitable for inclusion in the `results` directory. See `lib/summary.mjs` for details of how to interpret that.
+
+To run a single test, use:
+
+> npm run cli --base=http://server-url.example.org:8000/ --id=test-id
+
+... where `test-id` is the identifier for the test. This will output the request and response headers as seen by the client and server, along with the results. This is useful for debugging a particular failure.
 
-> HTTP_PROXY=http://127.0.0.1:8082 npm run --silent cli --base=http://127.0.0.1:8000
 
-## Interpreting the Results
+## Interpreting results
 
 HTTP caching by its nature is an optimisation; implementations aren't required to cache everything. However, when they do cache, their behaviour is constrained by [the specification](https://httpwg.org/specs/rfc9111.html).
 
@@ -140,11 +164,11 @@ Some additional results might pop up from time to time:
 When you're testing with a browser, each test has a `uuid` that identifies that specific test run; this can be used to find its requests in the browser developer tools or proxy logs. Click ⚙︎ to copy it to the clipboard.
 
 
-### Test Results FAQ
+### Test results FAQ
 
 If you see a lot of failures, it might be one of a few different issues:
 
-* If you see lots of grey circles at the top (dependency failures), it's probably because the cache will store and reuse a response without explicit freshness or a validator. While this is technically legal in HTTP, it interferes with the tests. Disabling "default caching" or similar usually fixes this.
+* If you see lots of grey circles at the top (dependency failures), it's probably because the cache will store and reuse a response without explicit freshness or a validator; see the very first test (`freshness-none`). While this is technically legal in HTTP, it interferes with the tests. Disabling "default caching" or similar usually fixes this.
 
 * If you see lots of blue diamonds (setup failures), it's likely that the cache is refusing `PUT` requests. Enable them to clear this; the tests use PUT to synchronise state between the client and the server.
 
@@ -160,4 +184,4 @@ Right now, all of the reverse proxy and CDN implementations are run by a script
 
 ## Creating new tests
 
-See [CONTRIBUTING.md](.github/CONTRIBUTING.md)
+See [CONTRIBUTING.md](CONTRIBUTING.md)