Skip to content

Commit

Permalink
Merge pull request #18 from dailymotion/before_after_feature
Browse files Browse the repository at this point in the history 
Implemented setup/teardown feature
  • Loading branch information
@ubermensch01
ubermensch01 committed Jan 24, 2020
2 parents b778d93 + 25a235b commit e894c24
Show file tree
Hide file tree
Showing 12 changed files with 628 additions and 72 deletions.
335 changes: 289 additions & 46 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,88 +1,331 @@
# Allure Golang Integrations
# Allure Golang Integration

allure-go is a Go package that provides support for Allure reports in Golang : https://github.com/allure-framework/allure2
`allure-go` is a Go package that provides all required tools to transform your tests into [Allure](https://github.com/allure-framework/allure2) reports.

## Installing
## Table of Contents

To start using allure-go, install Go and run `go get`:
1. [Installation](#installation)
2. [Usage](#usage)
1. [Specifying `allure-results` folder location](#specifying-allure-results-folder-location)
2. [Optional environment variable](#optional-environment-variable)
3. [Features](#features)
1. [Test Wrapper](#test-wrapper)
2. [Nested Steps](#nested-steps)
3. [Option Slices](#option-slices)
1. [Description](#description)
2. [Action](#action)
3. [Parameter](#parameter)
4. [Parameters](#parameters)
4. [Test-specific options](#test-specific-options)
1. [Lead](#lead)
2. [Owner](#owner)
3. [Epic](#epic)
4. [Severity](#severity)
5. [Story](#story)
6. [Feature](#feature)
4. [Parameters](#parameters-as-a-feature)
5. [Attachments](#attachments)
6. [Setup/Teardown](#setupteardown)
7. [Environment files](#environment-files)
4. [Feature Roadmap](#feature-roadmap)


## Installation
In order to retrieve `allure-go` simply run the following command (assuming `Go` is installed):
```sh
$ go get -u github.com/dailymotion/allure-go
```

This will retrieve the library.

## Usage

### Specifying allure-results folder location
Golang's approach to evaluating test scripts does not allow to establish a single location for test results
programmatically, the following environment variable is required for `allure-go` to work as expected.
In order to specify the location of a report run the following:
```shell script
export ALLURE_RESULTS_PATH=</some/path>
```

`allure-results` folder will be created in that location.

### Optional environment variable
Allure-go will retrieve the absolute path of your testing files (for example, /Users/myuser/Dev/myProject/tests/myfile_test.go) and will display this path in the reports.

First of all, you need to set an environment variable to define the `allure-results` folder location:
To make it cleaner, you can trim prefix the path of your project by defining the `ALLURE_WORKSPACE_PATH` with the value of your project root path :
```shell script
export ALLURE_WORKSPACE_PATH=/another/path
```
export ALLURE_RESULTS_PATH=/some/path

You will now get the relative path of your test files from your project root level.

## Features

### Test Wrapper
`allure-go` test reporting relies on wrapping test scripts inside a call to `allure.Test()` function:
```go
allure.Test(t *testing.T, testOptions ...Option)
```
This should be the path where the `allure-results` folder exists or should be created, not the path of the folder itself.
This function allows adding required modules representing a test along with additional non-essential modules like labels.
For basic usage refer to the following example:
```go
package test

The `allure-results` folder is automatically created if it doesn't exist with `drwxr-xr-x` file system permission.
import (
"fmt"
"github.com/dailymotion/allure-go"
"testing"
)

func TestStep(t *testing.T) {
allure.Test(t, allure.Action(func() {
fmt.Println("This block of code is a test")
}))
}
```

### Using environment file
### Nested Steps
`allure-go` takes advantage of Allure Reports feature of nested steps and allows marking portions of the code as steps
even if said portions are found outside test scripts in functions that are eventually called by a test script.
This is a great feature for enabling levels of detail for test reporting.
```go
package test

Allure reporting tool allows putting run-specific properties in the report provided
by `environment.properties` or `environment.xml` file.
import (
"fmt"
"github.com/dailymotion/allure-go"
"testing"
)

func TestStep(t *testing.T) {
allure.Test(t, allure.Action(func() {
allure.Step(allure.Description("Action"),
allure.Action(func() {
fmt.Println("This block of code is a step")
}))
PerformAction()
}))
}

In order to take advantage of this feature, set an environment variable to specify the location of the file:
func PerformAction() {
allure.Step(allure.Description("Action"),
allure.Action(func() {
fmt.Println("This block of code is also a step")
}))
}
```
export ALLURE_ENVIRONMENT_FILE_PATH=/path/to/file/environment.properties
```

### Examples
### Option Slices
Option slices allow providing as much or as little detail for the test scripts as needed.

To implement this library in your tests, follow the [examples](example/example_test.go).
#### Description
```go
allure.Description("Description of a test or a step")
```
Provides a name for the step and a description for the test. This option is required

#### Action
```go
allure.Action(func() {})
```
This option is a wrapper for the portion of code that is either your test or your step. This option is required.

Execute tests with the usual go test command :
#### Parameter
```go
allure.Parameter(name string, value interface{})
```
go test ./example
This option specifies parameters for the test or step accordingly.
This particular option can be called multiple times to add multiple parameters.

#### Parameters
```go
allure.Parameters(parameters map[string]interface{})
```
This option allows specifying multiple parameters at once.
This option can be called multiple times as well and will not interfere with previous parameter assignments.

This will automatically generate an `allure-results` folder in the path defined by ALLURE_RESULTS_PATH.
### Test-specific options

To see the report in html, generate it with the allure command line :
#### Lead
```go
allure.Lead(lead string)
```
allure serve $ALLURE_RESULTS_PATH/allure-results
This option specifies a lead of the feature.

#### Owner
```go
allure.Owner(owner string)
```
This will automatically generate and open the HTML reports in a browser.
This option specifies an owner of the feature.

The results file are compatible with the [Jenkins plugin](https://wiki.jenkins.io/display/JENKINS/Allure+Plugin).
#### Epic
```go
allure.Epic(epic string)
```
This option specifies an epic this test belongs to.

## Optional environment variable
#### Severity
```go
allure.Severity(severity severity.Severity)
```
This option specifies a severity level of the test script to allow better prioritization.

Allure-go will retrieve the absolute path of your testing files (for example, /Users/myuser/Dev/myProject/tests/myfile_test.go) and will display this path in the reports.
#### Story
```go
allure.Story(story string)
```
This option specifies a story this test script belongs to.
This particular option can be called multiple times to connect this test script to multiple stories.

To make it cleaner, you can trim prefix the path of your project by defining the `ALLURE_WORKSPACE_PATH` with the value of your project root path :
#### Feature
```go
allure.Feature(feature string)
```
export ALLURE_WORKSPACE_PATH=/another/path
This option specifies a feature this test script belongs to.
This particular option can be called multiple times to connect this test script to multiple features.

### Parameters as a feature
`allure-go` allows specifying parameters on both test and step level to further improvement informativeness of your
test scripts and bring it one step closer to being documentation.
In order to specify a parameter refer to the following example:
```go
package test

import (
"github.com/dailymotion/allure-go"
"testing"
)

func TestParameter(t *testing.T) {
allure.Test(t,
allure.Description("Test that has parameters"),
allure.Parameter("testParameter", "test"),
allure.Action(func() {
allure.Step(allure.Description("Step with parameters"),
allure.Action(func() {}),
allure.Parameter("stepParameter", "step"))
}))
}
```
Allure parameters integrate neatly with Golang's parameterized tests too:
```go
package test

You will now get the relative path of your test files from your project root level.
import (
"github.com/dailymotion/allure-go"
"testing"
)

func TestParameterized(t *testing.T) {
for i := 0; i < 5; i++ {
t.Run("", func(t *testing.T) {
allure.Test(t,
allure.Description("Test with parameters"),
allure.Parameter("counter", i),
allure.Action(func() {}))
})
}
}
```

### Attachments
`allure-go` providing an ability to attach content of various MIME types to test scripts and steps.
The most common MIME types are available as constants in `allure-go`
(`image/png`, `application/json`, `text/plain` and `video/mpeg`).
In order to attach content to a test or step refer to the following example:
```go
package test

## Features available
import (
"errors"
"github.com/dailymotion/allure-go"
"log"
"testing"
)

This project is still in progress. Here is the list of available features :
- It is possible to create a test object in the report with the `allure.Test()` method.
- It is possible to create a step object inside a test object of the report with the `allure.Step()` method.
- It is possible to pass parameters to a test object or a step object with the `allure.TestWithParameters()` and `allure.StepWithParameters()` methods.
- It is possible to add attachments to a test object or a step object with the `allure.AddAttachment()` method (depending if it is called inside an `allure.Test()` wrapper or an `allure.Step()` wrapper).
func TestImageAttachmentToStep(t *testing.T) {
allure.Test(t,
allure.Description("testing image attachment"),
allure.Action(func() {
allure.Step(allure.Description("adding an image attachment"),
allure.Action(func() {
err := allure.AddAttachment("image", allure.ImagePng, []byte("<byte array representing an image>"))
if err != nil {
log.Println(err)
}
}))
}))
}
```

### Setup/Teardown
Golang does not directly follow the setup/teardown approach of other languages like Java, C# and Python.
This does not prevent us from logically separating said phases in test scripts and taking
advantage of separating these phases in the test reports too.
`allure.BeforeTest` and `allure.AfterTest` functions have to be called in sequence with the test wrapper.

`allure.BeforeTest` goes first, then `allure.Test` and finally `allure.AfterTest`.
This is done to allow various logical conclusions, such as upon failure of `allure.BeforeTest` actual test body
will be skipped, etc.

Please refer to the following example of setup/teardown usage:
```go
package test

import (
"github.com/dailymotion/allure-go"
"testing"
)
func TestAllureSetupTeardown(t *testing.T) {
allure.BeforeTest(t,
allure.Description("setup"),
allure.Action(func() {
allure.Step(
allure.Description("Setup step 1"),
allure.Action(func() {}))
}))

allure.Test(t,
allure.Description("actual test"),
allure.Action(func() {
allure.Step(
allure.Description("Test step 1"),
allure.Action(func() {}))
}))

allure.AfterTest(t,
allure.Description("teardown"),
allure.Action(func() {
allure.Step(
allure.Description("Teardown step 1"),
allure.Action(func() {}))
}))
}
```
In case of utilizing setup/teardown in a parameterized test, all of the test phases have to be called
inside `t.Run(...)` call.

### Environment Files
`allure-go` allows taking advantage of the environment file feature of Allure reports. You can specify report-specific
variables that you want to appear in the report such as browser kind and version, OS, environment name, etc.
In order to do that create an `environment.xml` or `environment.properties` file as instructed [here](https://docs.qameta.io/allure/#_environment) and define an
environment variable for `allure-go` to incorporate in the results:
```shell script
export ALLURE_ENVIRONMENT_FILE_PATH=<path to environment file>
```

## Improvements needed
## Feature Roadmap
`allure-go` is still in active development and is not yet stabilized or finalized.

Here is the list of improvements that are needed for existing features :
Here is a list of improvements that are needed for existing features :
- [X] Manage failure at step level : the project must be able to mark a Step as failed (red in the report) when an assertion fails. The assertion stacktrace must appear in the report.
- [ ] Manage error at test and step level : the project must be able to mark a Test and Step as broken (orange in the report) when an error happens. The error stacktrace must appear in the report.
- [ ] Add support of Links
- [X] Manage error at test and step level : the project must be able to mark a Test and Step as broken (orange in the report) when an error happens. The error stacktrace must appear in the report.
- [X] Add support of Links
- [ ] Add support of Unknown status for Test object
- [ ] Add support of Set up in Execution part of a Test
- [ ] Add support of Tear down in Execution part of a Test
- [ ] Add support of Severity
- [X] Add support of Set up in Execution part of a Test
- [X] Add support of Tear down in Execution part of a Test
- [X] Add support of Severity
- [ ] Add support of Flaky tag
- [ ] Add support of Categories
- [ ] Add support of Features
- [X] Add support for environment files
- [X] Add support of Categories
- [X] Add support of Features
- [X] Add support for environment files
- [ ] Add support for history

0 comments on commit e894c24

Please sign in to comment.