Forward thresholds execution results when running k6 -o cloud scripts.js

[oleiade]

Feature Description

As of this writing, when k6 is run with its output set to the cloud (k6 run -o cloud script.js), it observes the same behavior as when k6 run is executed, and does not forward its thresholds results to the cloud backend. It results in thresholds results not being displayed in k6 cloud's web interface because their result were simply not sent.

In an ideal state, we would prefer for k6 run -o cloud to evaluate thresholds locally as it is the case currently, but we would also like it to include the results of the thresholds' execution in the output and forward it to the cloud to be able to interpret and display them.

Suggested Solution (optional)

It is yet unclear how we would address this, and will probably involve some non OSS work behind closed doors.

Already existing or connected issues / PRs (optional)

• Part of the initial scope of #1443 was to actually define a shared representation format for thresholds, with the long-term idea to first unify execution, and then allow k6 to stream thresholds results to the cloud?

[na--]

Hmm is this a regression? IIRC this used to work with old k6 versions, and the code still seems to be in the cloud output 😕 See:

k6/output/cloud/output.go

Line 296 in 1e84682

err := out.testFinished()

k6/output/cloud/output.go

Lines 682 to 710 in 1e84682

	func (out *Output) testFinished() error {
	if out.referenceID == "" || out.config.PushRefID.Valid {
	return nil
	}
	testTainted := false
	thresholdResults := make(cloudapi.ThresholdResult)
	for name, thresholds := range out.thresholds {
	thresholdResults[name] = make(map[string]bool)
	for _, t := range thresholds {
	thresholdResults[name][t.Source] = t.LastFailed
	if t.LastFailed {
	testTainted = true
	}
	}
	}
	out.logger.WithFields(logrus.Fields{
	"ref": out.referenceID,
	"tainted": testTainted,
	}).Debug("Sending test finished")
	runStatus := lib.RunStatusFinished
	if out.runStatus != lib.RunStatusQueued {
	runStatus = out.runStatus
	}
	return out.client.TestFinished(out.referenceID, thresholdResults, testTainted, runStatus)
	}

k6/cloudapi/api.go

Line 141 in 1e84682

Thresholds ThresholdResult `json:"thresholds"`

[na--]

Ah, so the actual task is about the metric values that are compared to the threshold constraints, since we only send the threshold names and their pass/fail status at the moment:

k6/cloudapi/api.go

Line 40 in 1e84682

type ThresholdResult map[string]map[string]bool

But then this is a duplicate of this older issue, so I'll close it: #1956