v0.37.0

k6 v0.37.0 is here! 🎉 Mainly it contains fixes and ongoing efforts with refactoring.

New Features!

Added experimental basic event loop (#882)

We added basic event loop support in k6 (#2228 and #2373) 🎉 This was just the first step and isn't used by any of the existing k6 JS APIs yet. For now, it is only available to xk6 extensions like this one that adds support for setTimeout(), setInterval(), clearTimeout() and clearInterval().

Expect to see more changes related to event loops in the next k6 releases, where event loops will start being used by some core k6 modules! For example, by improving some existing JavaScript APIs to have support for callbacks or return Promise values, so they can be used asynchronously. We expect this change will unlock a lot of previously difficult use cases (see #882), though we'll likely iterate on these new APIs as experimental extensions for a while, to stabilize them before we merge them into the core.

ℹ️ If you are an extension developer, please use it and give your feedback. But take into consideration that it's likely that the current Go API may change.

Added an option to output k6 logs to a file through --log-output (#2285)

This is on top of the already supported options for sending logs to stdout/stderr and to Grafana Loki. This new feature speaks for itself with simple usage examples:

k6 run --log-output file=./k6.log --logformat json ./examples/stages.js

And one more with a defined minimal log level:

k6 run --log-output file=./k6.log,level=info --logformat json ./examples/stages.js

Thanks, @alyakimenko for the contribution!

Docs: Using file output

Breaking changes

Introduced stricter thresholds parsing (#2400)

In the past, thresholds were evaluated using a JavaScript runtime. For a multitude of reasons, this wasn't satisfying. As of v0.37.0, thresholds are now parsed directly in Go. As a result, k6 will now return an error message on thresholds that do not strictly match the documented specification, instead of just silently ignoring them. Another change is that when a non syntactically correct threshold expression is detected, k6 will immediately interrupt its execution before even starting the load test run.

Below you can find examples of the thresholds expressions that won't work anymore:

export const options = {
    thresholds: {
        "http_req_duration": [
            // although the aggregation method and values are correct, 
            // the equal sign is invalid; use == or ===
            "rate=200",
            // thresholds do not support javascript expressions anymore
            "throw new Error('wat')",
            // it fails, as foo is not a valid threshold expression's aggregation method keyword
            "foo>500",
        ],
    },
};

Extensions

v0.37.0 finalizes (#2376) the switching of our internal modules (gRPC module refactoring) to a new Go/JavaScript module API.

⚠️ It's important to highlight that the old API (e.g. methods like context.WithRuntime, common.Bind and others #2384) is deprecated and will be removed in the next k6 release (v0.38.0). For this release, every extension that isn't using the new API will get a warning message like this:

WARN[0000] Module 'k6/x/sql' is using deprecated APIs that will be removed in k6 v0.38.0, for more details on how to update it see https://k6.io/docs/extensions/guides/create-an-extension/#advanced-javascript-extension

We did migrations for some xk6 extensions (see connected issues to the task #2344). The pull requests can serve as examples on how to transition your extension to the new API.

Docker Repository

We migrated our Docker Hub repository from loadimpact/k6 to grafana/k6 (#2377).

docker run -i grafana/k6 run - <script.js

We will continue publishing our docker image releases as both loadimpact/k6 and grafana/k6 for several more releases, but if you use the old one in your local or CI environments, please plan the migration.

Enhancements and UX improvements

• We continued work on the source map feature: add samples (#2339) and improve loading and parsing (#2355).
• Updated installation from source instructions (#2359). Thanks, @karitham, for making the change.
• Enabled more TC39 tests: optional-chaining (#2338) and async (#2396).
• We updated our Code of Conduct and a few GitHub URLs (#2416).

Bugs fixed!

• http.head started taking a body as a second argument (#2402) by mistake in v0.36.0. It is now back to its old signature http.head(url, [params]).
• Fixed options.scenarios' JSON marshaling (#2392).
• Metrics' names display again in k6's Rest API (#2421).
• We improved argument validation for check(#2387), http.batch(#2415) and metrics' constructors (#2427).

Internals

• We updated our CI to improve developer experience. Dependency and linter checks now run only for pull requests (#2403).
• This release also contains a few refactoring PRs that fix linters errors (#2334, #2331 and #2341), remove global variable usage (#2336, #2358, #2353 and #2357) and remove an unnecessary dependency (#2313) which makes our codebase more consistent and maintainable.
• The headers parameter in k6's GRPC module is marked as deprecated (#2370).
• Switched envconfig to our own fork (#2337) in order to abstract the os package and improve testability.

v0.36.0

k6 v0.36.0 is here! 🎉 It introduces a couple of new features which enhance its usability, includes a number of fixes, and the result of ongoing refactoring efforts.

New Features!

Source Maps support (#2082)

Following #2082, k6 now has support for Source Maps. k6 will try to load source maps either from the file system(s) or from inside the script, based on the standard(-ish) //#sourceMappingURL= comments. Furthermore, as k6 internally uses Babel to transform ES6+ scripts to ES5.1+, it will now make use of its ability to generate source maps, including combining with previously generated ones, to report correct line numbers. This should fix #1804; however, we anticipate some more issues will arise, and further tweaking will be necessary.

Thus, given an imported.js module such as:

export function f1() {
  throw "line 2";
}

  throw "line 6";
}

export function f3() {
  throw "line 10";
}

and a k6 test script importing it as such:

import { f2 } from "./imported.js"

export default function() {
  f2();
}

Previous versions of k6 would report an error stack trace indicating an invalid line number in imported.js (10):

ERRO[0000] line 6
        at f2 (file:///some/path/imported.js:10:61(2))
        at file:///some/path/sourcemap.js:4:20(4) executor=per-vu-iterations scenario=default source=stacktrace

Starting with v0.36.0 and source maps support, k6 would now report the exception at the correct line in imported.js:

ERRO[0000] line 6
        at f2 (file:///some/path/imported.js:6:2(2))
        at file:///some/path/loadtest.js:4:2(4)
        at native executor=per-vu-iterations scenario=default source=stacktrace

Temporary warning

Note that if a file size is greater than 250kb and the internal Babel is needed, Babel will not generate source map. This is because during internal testing it was found this takes 3x to 4x more memory, potentially leading to OOM (standing for "Out Of Memory", a state in which the OS kills a process for using too much memory) on bigger inputs. If required, you can control the accepted file size limit via the temporary K6_DEBUG_SOURCEMAP_FILESIZE_LIMIT=524288 environment variable; which will be removed after we no longer rely on Babel (#2296). A pre-generated source map will always be loaded. For more details, check #2345.

Ability to abort tests (#2093)

Thanks to the contribution of @gernest (#2093), k6 now has the ability to abort a test run from within the test script. The newly added test.abort() function in the k6/execution module allows k6 scripts to immediately abort the test execution - the VU that called it will abort immediately and any other VUs in the same or other instances (in the case of k6 cloud) will also be interrupted and abort soon after. Local k6 run tests will exit with a code of 108, so this event can also be easily detected in a CI script.

Aborting is possible during initialization:

import exec from "k6/execution";
exec.test.abort();

As well as inside the default function:

import exec from "k6/execution";

export default function() {
  // Note that you can abort with a specific message too
  exec.test.abort("this is the reason");
}

export function teardown() {
  console.log("teardown will still be called after test.abort()");
}

k6 inspect extended output (#2279)

Following #2279, the k6 inspect command now supports an --execution-requirements flag. When used, the command's output will include fields related to the execution requirements, by deriving k6's configuration from the execution context, and including the maxVUs and totalDuration fields in the output.

Forcing HTTP/1 protocol (#2222)

Thanks to the work of @sjordhani22, #2222 made it possible to force k6 to use version 1.1 of the protocol when firing HTTP requests.

It can be done by setting the http2client=0 value in the GODEBUG environment variable:

GODEBUG=http2client=0 k6 run testscript.js

N.B: the usage of the GODEBUG variable is considered temporary, and expected to change in the future. If you start using this feature, keep an eye out for potential future changes.

Extensions

v0.36.0 marks the switch of some of our internal modules to a new Go/JavaScript module API. We expect this change to make the process of developing internal JavaScript modules and advanced JavaScript extensions easier and more streamlined in the future. Although this switch to a new API does not introduce breaking changes for existing extensions yet, we anticipate deprecating the old extension API (e.g. common.Bind(), lib.WithState(), etc.) at an undecided point in the future.

For more details, see: #2243, #2241, #2239, #2242, #2226, and #2232.

Breaking changes

Restricting file opening to init context

VUs are now restricted to only open() files that were also opened in the init context of the first VU - the one that was initialized to get the exported options from the JS script (__VU==0). While it was somewhat possible to open files only in other VUs (e.g __VU==2) in the past, it was unreliable. #2314 ensures that k6 would now throw an error in a similar scenario. This means that you can still open files only for some VUs, but you need to have opened all of those files in the initial VU (__VU==0).

let file;

if (__VU == 0) {
  open("./file1.bin")
  open("./file2.bin")
} else if (__VU % 2 == 0) {
  file = open("./file1.bin")
} else {
  file = open("./file2.bin")
}

export default () => {
  // use file for something
}

Bugs Fixed!

• We addressed an issue uncovered by our community, which kept our users from using GRPC with multiple services definition in a single proto file. This issue was solved in #2265.
• Thanks to the contribution of @Resousse, we've now updated k6's go-ntlmssp dependency. The updating PR #2290 indeed fixes issues with NTLM Authentication backends returning two authorization headers.

Maintenance

• We have refactored our implementation of the RampingVU executor, for better clarity and maintainability. See #2155.
• #2316 relaxed quite a few of the code linting rules we applied to k6's code. It also revamped our Makefile, so the new make ci-like-lint target will run the exact same golangci-lint version that will be used in our GitHub Actions CI pipeline.
• #2304 prepared the removal of external dependencies from k6's JSONAPI compliant REST API, and deprecated the api.v1's client.Call method in favor of its newer client.CallAPI counterpart. It allows us to both reduce our reliance on external dependencies and improve its maintainability.
• We have updated our Goja dependency, our JS interpreter, to its latest available version. Unfortunately, some of the new features are not always usable, yet. Namely, Goja now supports the optional chaining syntax, but the Babel version we use presently does not. Which means that if Babel needs to be used, optional chaining can't be. See #2317 and #2238.
• Thanks to @knittl, #2312 upgraded loadimpact/k6 docker image base to Alpine 3.15.

Known Bugs

• #2226 introduced an unintended breaking change to http.head(). The signature in k6 v0.35.0 was http.head(url, [params]) and was inadvertently changed to http.head(url, [body], [params]) in v0.36.0. That change will be reverted in k6 v0.37.0, but until then, we suggest users use the stable http.request('HEAD', url, null, params) API for HTTP HEAD requests that need to specify custom parameters. Thanks, @grantyoung, for reporting the problem (#2401)!

v0.35.0

k6 v0.35.0 is here! 🎉 It introduces several new features that nicely enhance its usability and also contains a whole lot of fixes and ongoing efforts with refactoring.

In total, we have closed 14 issues. We have also branched out three new xk6 extensions during this release ⭐

New features

Ability to set VU-wide custom metric tags (#2172)

k6 now supports setting tags for VUs as part of the Execution API with an easy key-value interface. These tags are attached to the metrics emitted by the VU. Example usage:

import http from 'k6/http';
import exec from 'k6/execution';

export const options = {
    duration: '10s',
    vus: 3,
};

export default function () {
    exec.vu.tags['mytag'] = 'value';
    exec.vu.tags['vuId'] = exec.vu.idInTest;

    console.log(`mytag is ${exec.vu.tags['mytag']} and my VU's ID in tags ${exec.vu.tags['vuId']}`);

    // the metrics these HTTP requests emit will get tagged with `mytag` and `vuId`:
    http.batch(['https://test.k6.io', 'https://test-api.k6.io']);
}

One of the most requested use cases for this feature is that now we can tag all metrics with the current stage number. With a bit of JS code it is possible to calculate which stage of a ramping-vus or ramping-arrival-rate scenario the VU is currently in. This in turn allows the setting of thresholds only on the metrics that were emitted in specific stages of the test run! 🎉

There are some caveats, however: values can be only of String, Number or Boolean type, while values of other types will result either in a warning or an exception if throw option is enabled. Additionally, given that k6 has a whole bunch of system tags, one should be careful with using them as keys. You can read complete information about VU tags in k6/execution docs.

Initial basic support for JS promises

With the goja update in #2197, you can now make a Promise and chain it in your k6 scripts:

export default function () {
    var p = new Promise((resolve, reject) => {
        console.log('do something promising!');
        reject('here');
    });

    p.then(
        (s) => { console.log('fulfilled with', s) },
        (s) => { console.log('rejected with', s) },
    );
}

It must be noted that Promises are not used by k6 itself yet but this addition is a stepping stone for implementing async functionality in future releases. Thanks, @dop251, for your awesome work in developing goja! ❤️

Support for gRPC server reflection (#2160)

k6's gRPC capabilities were extended with a support for server reflection which allows one to use gRPC even without a proto file at hand. In other words, the following script is now possible:

import grpc from 'k6/net/grpc';
import { check } from "k6";

let client = new grpc.Client();

export default () => {
	client.connect("127.0.0.1:10000", {plaintext: true, reflect: true})
	const response = client.invoke("main.RouteGuide/GetFeature", {
		latitude: 410248224,
		longitude: -747127767
	})

	check(response, {"status is OK": (r) => r && r.status === grpc.StatusOK});
	console.log(JSON.stringify(response.message))

	client.close()
}

You can read more about the protocol here. Thanks, @joshcarp, for putting a lot of effort into this feature!

Other changes and UX improvements

• Support for cookie jars in k6/ws (#2193).
• Forbid metric.Add calls to let NaN values through. Instead, k6 will log nice warnings or throw an exception if --throw is enabled (#1876, #2219).
• Support for compression in websockets (#2162). Thanks, @cooliscool!
• Switch to camel case for CLI options to the outputs (#2150). Thanks, @josephwoodward!
• Much neater error message on nil response body (#2195). Thanks, @daniel-shuy!

New xk6 extensions

xk6-browser

xk6-browser is a browser automation extension which relies on Chrome Devtools Protocol. With xk6-browser, you can interact with the browser to test your web applications end-to-end while accessing all of the k6 core features, including protocol-level APIs and other k6 extensions. It’s a single tool for both protocol and browser-level testing.

The browser extension comes with an API that aims for rough compatibility with the Playwright API for NodeJS, meaning k6 users do not have to learn an entirely new API.

xk6-output-remote-write

Prometheus is now officially supported in k6 OSS with a xk6-output-remote-write extension. This is an output extension with implementation for Prometheus Remote-Write protocol which means that beyond Prometheus, any compatible remote-write solution can be used with it. You can read the full guide to using the extension in the relevant tutorial.

xk6-output-influxdb

After hard work at working out how to integrate InfluxDB v2 API, it was decided to pull that integration into a new xk6-output-influxdb extension for now. The built-in influxdb output in k6 still supports only InfluxDB v1, as before, with some minor optimizations (#2190).

Please try out the new extensions and tell us what you think!

Breaking changes

• The addition of common metrics registry (#2071) no longer allows defining custom metrics with the same name as one of the builtin metrics, e.g. new Counter("http_req_duration") will now abort. Similarly, an attempt to redefine a metric with the same name but with different type will error out. Builtin metrics may no longer be referenced as global objects in xk6 extensions either.
• Fix inconsistency in environment variables' names: use K6_NO_SETUP and K6_NO_TEARDOWN options instead of NO_SETUP and NO_TEARDOWN (#2140).
• Module interfaces were changed as part of refactoring efforts. Any JS module that needs access to the VU must now implement the new interfaces. This change can impact some xk6 extensions (#2234).

Bugs fixed!

• Fix of a misleading sorting of custom submetrics in the default end-of-test summary (#2198). Thanks, @knittl!
• Fix for extensions depending on afero.FS: implement a newer version of the afero.FS interface for internal filesystems so that extension depending on that or newer version can be built (#2216).
• Fix for websockets: websockets now use the global User-Agent setting (#2151). Thanks, @cooliscool!
• Fixes for tests, Github actions, and Loki integration (#2205, #2153, #2220).

Maintenance

• Update a whole bunch of dependencies (#2159, #2170, #2165).
• Switch to Go 1.17 (#2156). Thanks, @b5710546232!
• Get rid of mapstructure (#2223).
• Minor but necessary cleanups (#2164, #2192).

hacktoberfest

k6 participated in this year's hacktoberfest and we would like to thank all contributors! Here're some additional improvements made by the community members:

• Add multi-message WebSockets tests (#2184).
• Try out the new and shiny Github forms which are already improving the formatting of k6's new issues (#2174,#2179).
• An improved writing style and correctness in our README (#2189, #2152, #2169, #2181) and in some other places (#2182).

Thank you, @knittl, @cooliscool, @josephwoodward, @b5710546232, @nontw, @divshacker, @daniel-shuy, @Sayanta66, @marooncoder09, @idivyanshbansal, @saintmalik, @EricSmekens, for helping make k6 better 😄

v0.34.1

k6 v0.34.1 is a patch release with a few minor bugfixes:

• There was a minor bug in the new k6/execution API added to k6 v0.34.0 - some of its properties weren't usable with the externally-controlled executor (#2132).
• goja, the JavaScript runtime that k6 uses, was updated to its latest version (#2135), which fixed a couple of bugs:
  • A newly introduced JS bug from k6 v0.34.0, where rest parameters were undefined when the functions also had an internal lambda (#2131). Thanks for reporting, @efdknittlfrank!
  • An old JS bug, first introduced in k6 v0.28.0, which caused Response.json() to not have a length property when the response was a JSON array, i.e.response.json().hasOwnProperty('length') was returning false (#2133). Thanks for reporting, @julien-sugg!

v0.34.0

k6 v0.34.0 is here! 🎉 It introduces the already announced k6/execution API and includes some more enhancements and a bunch of minor bug fixes.

New k6 JavaScript API - k6/execution

The k6/execution module allows the fetching of information about the current VU, instance (mostly relevant in distributed/cloud) or scenario execution state through a series of exported properties.

How to use

Some of the previously suggested solutions with the globally available __VU and __ITER values, such as for getting a unique object per iteration from an array or SharedArray, can be done by using the scenario.iterationInTest property, which is guaranteed to be unique across VUs, even for distributed or cloud tests. For example:

import exec from "k6/execution";
import { SharedArray } from "k6/data";

const data = new SharedArray("my dataset", function(){
  return JSON.parse(open('my-large-dataset.json'));
})

export const options = {
  scenarios :{
    "use-all-the-data": {
      executor: "shared-iterations",
      vus: 100,
      iterations: data.length,
      maxDuration: "1h"
    }
  }
}

export default function() {
  // this is unique even in the cloud
  var item = data[exec.scenario.iterationInTest];
  http.post("https://httpbin.test.k6.io/anything?endpoint=amazing", item)
}

You can read the full documentation here.

Enhancements and UX improvements

• Warn Windows users on importing dependencies or opening files as absolute paths (#2078).
• Pass setup data object into handleSummary callback (#2103). Thanks, @SamuelJohnson01997!

Breaking changes

• The deprecated outputs Datadog and Kafka have been removed (#2081).

Bugs fixed!

• Use the POST HTTP request method instead of GET for pushing logs to Loki (#2100).
• Encode the blacklistIPs option using the CIDR notation in JSON (#2083).
• ext.loadimpact option has the same precedence as the script configuration during the consolidation process (#2099).
• The WebSocket connection used for tailing logs from the k6 Cloud is reestablished in the case of an unexpected error (#2090).

Internals

• A simpler and clearer API has been added as an alternative to common.Bind, which also gives JS modules and extensions easy access to some useful internal objects and runtime information (#2108). This API is not yet stable, it's very likely to change more in future k6 versions.
• Speeding ups TC39 tests using a pool of Babel compilers (#1839).
• Goja and some internal dependencies have been updated adding the native support for Arrow functions, Destructuring, Default arguments and Computed properties features. For the same reason, the relative Babel's plugins supporting those features are not required anymore so they have been disabled (#2109, #2092).

v0.33.0

k6 v0.33.0 is here! 🎉 It's a small release that includes a bunch of minor bugfixes and enhancements, but is also laying the groundwork for some major new features like the upcoming k6/execution API in k6 v0.34.0.

Acquired by Grafana Labs

Load Impact, the company behind k6, was acquired by Grafana Labs! 🎉 Nothing changes regarding the k6 development for now, and any changes in the future will only be in the direction of accelerating our existing roadmap and plans, as well as better integration between k6 and the awesome Grafana tools. For more details, see the official Grafana press release.

Enhancements and UX improvements

• The --verbose help message and the statsd warning message were improved (#2005). Thanks, @vishalkuo!
• The noColor k6 option and the current UI state are now propagated to the handleSummary() function. The state object has the isStdErrTTY, isStdOutTTY and testRunDurationMs keys (#1975).
• The error message when an HTTP request times out was improved (previously it was context deadline exceeded) and it now has an error_code value of 1050 (#2008). Thanks, @vishalkuo!
• Script errors will no longer have the confusing GoError prefix in their error messages (#1775).
• All custom metric objects now have a name property (#2058 and #2076). Thanks, @olimpias and @david-gourde!
• Top-level JS arrays will now be properly encoded when sent in the body of a application/x-www-form-urlencoded request (#2060). Thanks, @noelzubin!

Bugs fixed!

• The minIterationDuration option was uninterruptible and could delay the stopping of a scenario even after gracefulStop had expired. (#2035).
• The error_code detection for HTTP/2, x509 and TLS (and potentially others) was unreliable (#2025).
• k6 used to panic when responseType was binary, but there was no response body actually returned, e.g. when there was an HTTP error (#2041).
• The throw option was not respected when there was an invalid URL (#2045). Thanks, @gchaincl!
• k6 would return an exit code of 103 instead of 107 for script errors when initializing non-service VUs (#2046).
• Deleted library versions from cdnjs could previously cause a panic (#2047).
• The correct error message for missing files was not shown when the filename contained spaces (#1973).
• The regular expressions for the github and cdnjs "magic" loaders were slightly wrong (#2066).
• A potential (harmless) data race could have been caused by an unintentional copying of a data struct (#2067).
• The segmentation of small ramping-arrival-rate scenarios was not optimal (#1863).

Internals

• The default end-of-test summary is now completely generated by the same k6-summary JS code that is hosted on jslib.k6.io (#1975). That PR also improved the k6 TTY detection and removed a few Go dependencies and code hacks, though it also caused us to bump the minimum required Go version for compiling k6 to Go 1.16 (because of its usage of go:embed).
• Arrival-rate executors will no longer create a new goroutine for every new iteration (#1957 and #2038).
• We have enabled GitHub's CodeQL checks for the Go parts of the repo (#1961). Thanks, @jfcg!
• We have added the necessary k6 core changes for providing execution information to scripts #1863! This was the groundwork for the extended replacement of the __VU and __ITER execution context variables we plan to introduce. The new API will be able to return other information as well, for example which scenario the current iteration is in, what's the number of the current VU/iteration globally across all k6 instances, or in the current scenario, etc. These APIs are still not available to JS scripts, but we plan to expose them via the k6/x/execution xk6 extension for now and iterate on them in the following weeks, releasing a stable version in k6 v0.34.0.

Breaking changes

• The k6 cloud exit code for a failed cloud test was changed from 99 to 97 (#2046).
• The default value of K6_STATSD_TAG_BLOCKLIST and K6_DATADOG_TAG_BLACKLIST is now vu,iter,url (#2063).
• The __ITER execution context variable is no longer set in setup() and teardown() due to #1863. This might be better classified as removing a previously undefined behavior instead of a breaking change, but it's still worth mentioning.

v0.32.0

k6 v0.32.0 is here! 🎉 It's a smaller release, featuring mostly chores that we never got to or that needed to be done in this release cycle, but also includes a significant performance improvement for arrival-rate executors and some breaking changes.

Move out of Bintray

Bintray has stopped servicing users on 1st of May, which meant that we needed to move the deb and rpm repositories out of there before then. Please follow the new installation instructions and the "Note about Bintray" to find out how to remove the old repository if you have used them.

Notable changes

Move all outputs to new Output interface introduced in v0.31.0 and other related changes

We started on this in the previous v0.31.0 release and now all internal outputs implement the new Output interface we provided for xk6 output extensions. Additionally one of the old built-in outputs is being deprecated and one renamed:

1. The kafka output has been somewhat neglected since it was added 3 years ago. Additionally it brings a lot of complexity and isn't well understood by anyone on the team. Given that output extensions are possible since the last version, the Kafka output has been moved to one. The built-in output will continue to work for a few more k6 versions, emitting a deprecation warning when used, so that everyone has time to transition to the extension. All future improvements will happen only in the extension, the built-in output is frozen until it's dropped.
2. We are also deprecating/renaming the datadog output. It should've probably always been just a configuration of the statsd output and now in k6 v0.32.0, it is going to be just that. We've added a new K6_STATSD_ENABLE_TAGS option to the statsd output, which, when enabled (it's false by default), will send metric tags the same way the datadog output did before. That is, instead of using the datadog output, you should use the statsd one with K6_STATSD_ENABLE_TAGS=true. Additionally, the new K6_STATSD_TAG_BLOCKLIST option can be used to not send tags that the user doesn't want to, similar to the old K6_DATADOG_TAG_BLACKLIST option.
   This makes it cleaner to also emit metrics to other services that accept the same data and tag formats, such as New Relic, Amazon Cloudwatch, and statsd >v0.9.0. The old datadog output will still work for a few k6 versions, emitting a warning to switch to statsd when used.

Apart from a message about them being deprecated, nothing should be changed from the actual refactoring yet, but we advise users to use the proposed alternatives starting with this release.

json output emits thresholds defined on the metrics (#1886)

Previous to this change thresholds were not included in the json output. Now the Metric JSON object will get its thresholds field properly populated.

Thanks to @codebien for this contribution!

cloud output has an option to abort the test if aborted from the cloud (#1965)

In v0.26.0 we made the cloud output stop emitting metrics if it gets a particular error from the backend, as that meant that the test was aborted in the cloud. Now we added K6_CLOUD_ABORT_ON_ERROR to be able to say that it should not only stop emitting metrics, but also stop the execution of the local k6 test run. The default configuration is false, so it is backwards compatible. This also works when the test is aborted by the user or if cloud execution limits are reached, which would also lead to the test being aborted.

Full stack traces for init context and setup/teardown exceptions (#1971)

For a long time, if there was an exception in either the init context or in the setup() or teardown() invocations, the stack trace would be just the last line, making it really hard to debug issues there. Now there is a full stack trace and, as such errors will result in aborted k6 execution, k6 run will also exit with exit code 107, signalling a script error.

Considerable performance improvements for arrival rate executors (#1955)

Due to a wrong re-usage for a particular internal data structure, the arrival rate executors were having a much worse performance than expected. With this release they should be a lot more performant, especially with large numbers of VUs.

Updating the majority of our dependencies and dropping some

The list is too long and we have been postponing updating for quite some time now, but we have finally updated all our dependencies that we don't want to drop. This could lead to some stability problems, which is why it was done early on in the cycle. While the team has not found any regressions, given all the updates we could have missed something so please open a issue if you find anything.

Some notable updates:

• goja, the JS engine we use got support for let/const which allowed us to disable a Babel plugin. Previous to this if you had a particularly long script sometimes Babel took upwards of 30 minutes to transpile. Now even our worst contender that previously took 51 minutes is transpiled in less than a minute 🎉. Also globalThis is now available.
• updating the gRPC libraries fixed bug (#1928) and probably others. (#1937)

ArrayBuffer is now supported in all JS APIs dealing with binary data, including in WebSocket messages (#1841)

Besides the minor breaking changes (see the "Breaking changes" section below), it's now possible to send binary WS messages with the socket.sendBinary() function and to receive binary messages with the binaryMessage event handler:

const binFile = open('./file.pdf', 'b');

export default function () {
  ws.connect('http://wshost/', function(socket) {
    socket.on('open', function() {
      socket.sendBinary(binFile);
    });

    socket.on('binaryMessage', function(msg) {
      // msg is an ArrayBuffer, so we can wrap it in a typed array directly.
      new Uint8Array(msg);
    });
  });
}

Official arm64 releases for macOS and Linux (#2000)

We will now publish binary arm64 releases for macOS and Linux for new k6 releases. Support for these new architectures should be stable, given Go's cross-platform compatibility, but please report any issues you experience.

Other enhancements and UX improvements

• Options: k6 will now warn you on unrecognised configuration JS options in most cases instead of just silently ignoring them. (#1919)
• error_code: the tag error_code should now be set more accurately in some cases. (#1952)
• TLS: dropped support for SSLv3 encryption. This was dropped by Go, but now we no longer consider ssl3.0 a valid value for the tlsVersion k6 option. Thanks @codebien! (#1897)

Bugs fixed!

• Arrival-rate executors could in some cases report twice as many used VUs than what was actually true. (#1954 fixed by #1955)
• In cases of an error while reading the response body, the newly added responseCallback in v0.31.0 would be evaluated with the returned status code, while the reported one would be 0, as the response errored out and k6 does not return incomplete responses. Now responseCallback will also receive a 0 status. (#1962)
• Fix Kafka output not being usable with the InfluxDB format after v0.31.0 changes. (#1914)
• Error out with a user friendly message if ramping-vus executor would've not run a single iteration instead of just doing nothing. (#1942)

Internals

• JS: Added a way for JS modules to have per VU initialization and object. This can also be used by xk6 modules by implementing go.k6.io/k6/js/modules#HasModuleInstancePerVU. (Part of #1911)

Breaking changes

Support for ArrayBuffer in all k6 JS APIs (#1841)

Continuing from k6 v0.31.0, we're finalizing the transition to ArrayBuffer values for working with binary data. This release introduces some changes that might break scripts that relied on the previous array of integers or string result types, returned by some of our JS APIs. Specifically these cases now return ArrayBuffer instead: open(..., 'b'), HTTP response bodies for requests that specified responseType: 'binary' (including when http.batch() is used), crypto.randomBytes(), hasher.digest('binary') and encoding.b64decode().
The previous default behavior of returning string from encoding.b64decode() can be replicated with a new optional format argument and a value of "s": encoding.b64decode("5bCP6aO85by-Li4=", "url", "s").
Most of these shouldn't cause issues if the script is simply passing the values to another k6 API (e.g. opening a file as binary and passing it to http.post()), b...

v0.31.1

k6 v0.31.1 is a patch release with a single bugfix.

The bugfix is about the cloud output and the new http_req_failed metric in k6 v0.31.0. Due to additional state being used for its transport to the k6 cloud, and a misunderstanding of what a functional call from a library dependency does, the http_req_failed values were always set to 1. This did not affect any other output or the end of test summary. (#1908)

v0.31.0

k6 v0.31.0 is here! 🎉 It's a smaller release with some significant performance improvements, a new http_req_failed metric and changes to the output subsystem that enable output extensions with xk6!

New features

Output cleanup and extensions (#1874)

The state of k6's output packages has been a development pain point for a long time, which made it difficult to add new outputs in a consistent way. In the refactor done in v0.31.0, this has been mostly addressed and outputs now implement a simpler and cleaner Output interface.

In addition to this, it is now possible to implement custom k6 output extensions in Go with xk6! This is very useful if you use a system that's currently not supported by the built-in outputs, or need some custom handling of the metrics k6 produces.

Writing output extensions is done very similarly to how JS module extensions are currently written, though instead of calling js/modules.Register(), you should implement the new Output interface and call output.RegisterExtension() with your constructor.

We are working on the proper documentation for this, as well as the overdue xk6 documentation about JS extensions, so keep a lookout for those on k6.io/docs.

Marking requests as failed (#1856)

It's now possible to declare expected HTTP response statuses for either the entire test or for individual HTTP requests, and k6 will emit a new http_req_failed metric as well as tag HTTP metrics with expected_response: <bool>. By default, k6 will now fail requests that return HTTP 4xx/5xx response codes.

For example:

import http from 'k6/http';

// Set expected statuses globally for all requests.
http.setResponseCallback(http.expectedStatuses({min: 200, max: 399}, 418));

export default function () {
  // This request will be marked as failed.
  http.get('https://httpbin.test.k6.io/status/400');
  // This request will be considered as "passed" because of the responseCallback override.
  http.get('https://httpbin.test.k6.io/status/400', { responseCallback: http.expectedStatuses(400) });
}

Running this script will produce a summary like:

http_req_duration..............: avg=204.57ms min=203.31ms med=204.57ms max=205.82ms p(90)=205.57ms p(95)=205.7ms
  { expected_response:true }...: avg=203.31ms min=203.31ms med=203.31ms max=203.31ms p(90)=203.31ms p(95)=203.31ms
http_req_failed................: 50.00% ✓ 1   ✗ 1

Note the new http_req_duration sub-metric for expected responses only, and the new http_req_failed Rate metric. This new metric and metric tag have many potential use cases, and one of the most important ones is the ability to set better thresholds. For example:

• 'http_req_failed': ['rate<0.1'], i.e. fail the test if more than 10% of requests fail.
• 'http_req_duration{expected_response:true}': ['p(95)<300', 'p(99.9)<500'] - fail the test if the the 95th percentile HTTP request duration is above 300ms or the 99.9th percentile is above 500ms; specifying expected_response:true here may be important, because a lot of times failed requests may return more quickly than normal ones, thus skewing the results and wrongly satisfying the threshold.

If the response callback is not specified, the default expected statuses will be {min: 200, max: 399}. The previous behavior of not emitting anything can be achieved by setting the callback to null, i.e. http.setResponseCallback(null). Additionally, the expected_response tag can be disabled by removing it from the default list of system tags, e.g. k6 run --system-tags 'proto,subproto,status,method,url,name,group,check,error,error_code,tls_version,scenario,service'.

The http.setResponseCallback() is planned to allow arbitrary JS functions to process responses in the future, but for now only the http.expectedStatuses() callback is supported.

Other enhancements and UX improvements

• JS: Because of the awesome improvements to goja, the JS runtime k6 uses, it's no longer necessary for k6 to load core.js to polyfill missing JS features when using the default --compatibility-mode=extended. So in v0.31.0 core.js has been dropped entirely, yielding some significant CPU and memory usage improvements. The actual numbers will depend on the use case, but for simple tests users can expect a memory drop of about 2MB per VU (from ~2.7MB to ~600KB), and a slight CPU decrease of about 5-10%. For more complex tests with a lot of JS code this benefit won't be as pronounced. Another benefit of this change is that initializing VUs and starting a test is substantially faster than before! (#1824)
• JS: Also because of goja improvements, some unused Babel plugins were disabled which should have minor performance benefits as well. (#1822)
• JS: Expanded ArrayBuffer support in most internal modules, so now you can pass ArrayBuffer to http.file(), in k6/encoding and k6/crypto functions. This makes working with binary files more efficient as it doesn't require string translations. In upcoming versions we plan to expand this to the WebSocket module, as well as make some potentially breaking changes for APIs that currently return an array of integers or string (see the details in the Breaking Changes announcement below). (#1800)
• The Docker image base was updated to Alpine 3.13. Thanks @andriisoldatenko! (#1821)
• The Debian package now includes ca-certificates as a dependency. Thanks @Bablzz! (#1854)

Bugs fixed!

• Execution: Aborting a test during VU initialization (e.g. with ^C) will now properly propagate to any used outputs. (#1869)
• Execution: A race condition between the Engine and the outputs' finalization code was fixed, ensuring that all metrics are properly emitted before exiting. (#1869)
• Execution: Another race condition in the Engine was fixed, which may have resulted in the end-of-test summary missing some of the last test metric data. (#1888)
• Cloud: the test name is now properly validated and will raise an error if not set via the ext.loadimpact.name JS option or config, or the K6_CLOUD_NAME environment variable. (#1870)
• JS: Babel is now also run on compilation errors, which improves support of some obscure language features. (#1861)
• JS: SharedArray introduced in v0.30.0 can now be iterated with forEach. (#1848)

Internals

• JS: SharedArray was rewritten using goja.DynamicArray making it more performant and easier to reason about. (#1848)
• JS: Some TC39 tests for unsupported features were disabled, improving the runtime of the test suite. (#1816)
• CI: Some more tests were enabled on Windows. (#1855)

Breaking changes

• JS: While we don't expect the core.js removal and Babel changes to impact the vast majority of users, those were substantial changes in how k6 interprets JS and a minority of users might experience issues with their tests. Please report any unexpected JavaScript errors by creating a GitHub issue. In particular Promise is now undefined, and some unused Babel plugins like transform-es2015-for-of and transform-regenerator were also removed. This means that some workarounds like the ones mentioned here and here also won't work as is and will need additional polyfills and plugins to work properly.

Planned future breaking changes

The following are not breaking changes in this release, but we'd like to announce them so users can prepare for them in upcoming releases (likely k6 v0.32.0).

• JS: The ArrayBuffer changes in this release are backwards compatible and shouldn't cause any issues, but in v0.32.0 some JS APIs that currently return an array of integers or string for binary data will return ArrayBuffer instead. This is the case for open() when used with the 'b' argument, response bodies for requests that specify responseType: 'binary', crypto.randomBytes(), hasher.digest('binary'), and encoding.b64decode(). Response.json() and Response.html() will also probably stop working when used with requests that specify responseType: 'binary'. These changes shouldn't be a problem for most users that were simply using these values to pass them to other internal modules (e.g. opening a binary file and passing it to http.post()), but if the scripts modified the binary data or depended on the current array of integers or string values they will need to be adapted to use [typed arrays](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_...

v0.30.0

k6 v0.30.0 is here! 🎉 It was a bit of a slow after-holiday release, but it still packs a few major new features and improvements that users have been requesting for a long time!

New features

Share memory between VUs using read-only arrays (#1739)

k6 has long had an issue with the handling of big data files with test fixtures. For example, if you have a huge users.json file with test users for your application:

[
  {"username": "user1", "password": "password1", "other": "some-long-data-....-1"},
  {"username": "user2", "password": "password2", "other": "some-long-data-....-2"},
  // ... ~1 million more users or more... :D
  {"username": "user999999", "password": "password999999", "other": "some-long-data-....-999999"}
]

If you just use JSON.parse(open('users.json')) in your script, then every VU will have a copy of the whole huge data set. Every VU in k6 is a separate JavaScript runtime, so there wasn't a thread-safe way to share data between them. Until now, that is!

We've added a new built-in SharedArray object in the new k6/data module that allows VUs to share read-only data:

import { SharedArray } from 'k6/data';
import { sleep } from 'k6';
import http from 'k6/http';

let users = new SharedArray('someName', function () {
    // This function will be called only once, in the first init context
    // execution. Every other VU will just get a memory-safe read-only reference
    // to the already loaded data.
    console.log('Loading users.json, this happens only once...');
    // You are not restricted to JSON, you can do anything - parse a CSV or XML
    // file, generate random data, etc. - as long as you return an array.
    return JSON.parse(open('users.json'));
});

export let options = { vus: 10, duration: '30s' };
export default function () {
    let randomUserID = Math.floor(Math.random() * users.length);
    let user = users[randomUserID]; // alternatively, we can also use __VU and/or __ITER
    console.log(`VU ${__VU} is running iteration ${__ITER} with user ${user.username}...`);
    http.post('https://httpbin.test.k6.io/post', JSON.stringify(user));
    sleep(Math.random() * 2); // or, better yet, use arrival-rate
}

Notice how Loading users.json is logged only once, but each VU uses the users variable like a normal JS array. The data is read only once and we have just a single copy of the huge array in memory! Behind the scenes, k6 uses a JS Proxy to transparently copy only the row each VU requests in users[randomUserID] to it. This on-demand copying is a bit inefficient, but it's still leagues better than having a copy of the huge array in every VU! And you can avoid the copying in every iteration by pinning the data used by every VU and having let user = users[randomUserID] in the init context!

And yes, you can have multiple SharedArray objects in the same script, just make sure to give them unique names - this is what someName in the script above was for. Because VUs are independent JS runtimes, we need some way to differentiate between the different shared memory objects, so we require them to have unique names. These names are also the IDs that any xk6 extensions would need to use to access them.

This works both locally and in the cloud. We advise everyone who deals with large data files to wrap them in a SharedArray and give this new feature a try. The required script changes should be minimal, while the memory usage should be significantly lower. Hopefully, we can finally consider one of the biggest blockers k6 users have had for a long time solved! 🎉

Support a handleSummary() callback at the end of the test (#1768)

You can now export a function called handleSummary() and k6 will call it at the end of the test run, after even teardown(). handleSummary() will be called with a JS object containing the same information that is used to generate the end-of-test summary and --summary-export, and allows users to completely customize how the end-of-test summary looks like.

Besides customizing the end-of-test CLI summary (if handleSummary() is exported, k6 will not print the default), you can also transform the summary data to various machine or human-readable formats and save it to files. This allows the creation of JS helper functions that generate JSON, CSV, XML (JUnit/xUnit/etc.), HTML, etc. files from the summary data. Even binary formats like PDF are not out of reach, potentially, with an appropriate JS library that works in k6! You can also send the generated reports to a remote server by making an HTTP request with them (or using any of the other protocols k6 already supports)! Here's a simple example:

import http from 'k6/http';
import k6example from 'https://raw.githubusercontent.com/loadimpact/k6/master/samples/thresholds_readme_example.js';
export default k6example; // use some predefined example to generate some data
export const options = { vus: 5, iterations: 10 };

// These are still very much WIP and untested, but you can use them as is or write your own!
import { jUnit, textSummary } from 'https://jslib.k6.io/k6-summary/0.0.1/index.js';

export function handleSummary(data) {
    console.log('Preparing the end-of-test summary...');

    // Send the results to some remote server or trigger a hook
    let resp = http.post('https://httpbin.test.k6.io/anything', JSON.stringify(data));
    if (resp.status != 200) {
        console.error('Could not send summary, got status ' + resp.status);
    }

    return {
        'stdout': textSummary(data, { indent: ' ', enableColors: true}), // Show the text summary to stdout...
        'junit.xml': jUnit(data), // but also transform it and save it as a JUnit XML...
        'summary.json': JSON.stringify(data), // and a JSON with all the details...
        // And any other JS transformation of the data you can think of,
        // you can write your own JS helpers to transform the summary data however you like!
    }
}

k6 expects handleSummary() to return a {key1: value1, key2: value2, ...} map. The values can be string or ArrayBuffer and represent the generated summary report contents. The keys should be strings and determine where the contents will be displayed or saved: stdout for standard output, stderr for standard error, or a path to a file on the system (which will be overwritten).

The format of the data parameter is similar but not identical to the data format of --summary-export. The format of --summary-export remains unchanged, for backwards compatibility, but the data format for this new k6 feature was made more extensible and had some of the ambiguities and issues from the previous format fixed. We can't cover the new format in the release notes, though you can easily see what it contains by using return { 'stdout': JSON.stringify(data)}; in handleSummary()! 😄

This feature is only available for local k6 run tests for now, though we plan to support k6 cloud tests eventually. And, as mentioned in the snippet above, the JS helper functions that transform the summary in various formats are far from final, so keep an eye on jslib.k6.io for updates. Or, better yet, submit PRs with improvements and more transformations at https://github.com/loadimpact/jslib.k6.io 😄

Other enhancements and UX improvements

• CI: k6 releases for Windows will now be digitally signed, which should reduce the number and severity of warnings Windows users see; the warnings would hopefully disappear altogether once Microsoft sees enough usage of the signed k6 releases to trust us (#1746). The installer and binary were also enhanced with more metadata and had their look updated with the new k6 logo and styling (#1727).
• JS: goja, the JS runtime k6 uses, was updated to its latest master version. This includes a few bugfixes and support for several new features, so --compatibility-mode=base is even more feature-rich at no additional runtime cost. We are contributing patches to goja in an effort to completely drop core.js and have the benefit of lower CPU and memory usage per VU even with --compatibility-mode=extended in the next k6 version! 🎉
• Config: integer values for duration and similar time values in the exported script options and environment variables are now treated as milliseconds. Similarly, the timeout option in http.Params can now be "stringy", e.g. "30s", "1m10s", etc. (#1738).
• HTTP: k6 now accepts ArrayBuffer values for the HTTP request body (#1776). This is a prelude/MVP for us gradually adopting ArrayBuffer for all binary data in k6 (#1020).
• Docker: We've added WORKDIR /home/k6 to our official Dockerfile (#1794).

Bugs fixed!

• HTTP: updated the golang.org/x/crypto and golang.org/x/net dependencies, which should have resolved some corner case issues with HTTP/2 connections, since k6 depends on golang.org/x/net/http2 (#1734).
• HTTP: fixed a couple of issues with blockHostnames that prevented zero-length matches for wildcards, as well as the ex...