Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cucumber v5 error in steps file from supporting file results in "Unknown error" (no stack trace) #4314

Closed
pyxis828 opened this issue Aug 5, 2019 · 4 comments

Comments

@pyxis828
Copy link

pyxis828 commented Aug 5, 2019

[//]: # NOTE: This repository only maintains packages that are listed in the Readme. Please make sure that your issue is directly caused by one of these packages and if not file an issue in the correct 3rd party package repository.

Environment (please complete the following information):

  • WebdriverIO version: 5.11.6
  • Mode: Standalone mode or WDIO Testrunner -- tried both ways
  • **If WDIO Testrunner sync
  • Node.js version: 10.16.0
  • NPM version: 6.9.0
  • Browser name and version: Chrome 76.0.3809.87
  • Platform name and version: Windows 10
  • Additional wdio packages used (if applicable): @wdio/selenium-standalone service, @wdio/allure-reporter, @wdio/cucumber-framework (others installed but not in use)

Config of WebdriverIO

        //feature files
        "./features/login.feature"
    ],
    "capabilities": [{
        "browserName": "chrome",
        "goog:chromeOptions": {
            args: ['--disable-infobars', '--start-maximized']
        },
        "maxInstances": 1,
    //trace, debug, info, warn, error, silent
    "logLevel": "warn",
    waitforTimeout: 60000,
    connectionRetryTimeout: 90000,
    connectionRetryCount: 3,
    hostname: 'localhost',
    port: 4444,
    path: '/wd/hub',
    services: ['selenium-standalone'],
    seleniumLogs: 'logs',
    seleniumInstallArgs: {
        version : "3.9.1",
        baseURL : "https://selenium-release.storage.googleapis.com",
        drivers : {
          chrome : {
            version : "76.0.3809.68",
            arch    : process.arch,
            baseURL : "https://chromedriver.storage.googleapis.com",
          }
        }
      },
    framework: 'cucumber',
    cucumberOpts: {
        require: ['./steps/*.js'],        // <string[]> (file/dir) require files before executing features
        backtrace: false,   // <boolean> show full backtrace for errors
        compiler: [],       // <string[]> ("extension:module") require files with the given EXTENSION after requiring MODULE (repeatable)
        dryRun: false,      // <boolean> invoke formatters without executing steps
        failFast: false,    // <boolean> abort the run on first failure
        format: ['pretty'], // <string[]> (type[:path]) specify the output format, optionally supply PATH to redirect formatter output (repeatable)
        colors: true,       // <boolean> disable colors in formatter output
        snippets: true,     // <boolean> hide step definition snippets for pending steps
        source: true,       // <boolean> hide source URIs
        profile: [],        // <string[]> (name) specify the profile to use
        strict: false,      // <boolean> fail if there are any undefined or pending steps
        name: [],           // Only execute the scenarios with name matching the expression (repeatable)
        // tagExpression: [],   // <string> (expression) only execute the features or scenarios with tags matching the expression
        tagExpression: "@run",
        timeout: 500000,      // <number> timeout for step definitions
        ignoreUndefinedDefinitions: false, // <boolean> Enable this config to treat undefined definitions as warnings.
    },
    reporters: [ 'dot',
        ['allure', {
        outputDir: './reports/allure-results',
        disableWebdriverStepsReporting: true,
        disableWebdriverScreenshotsReporting: true,
        useCucumberStepReporter: true
    }]
],

Describe the bug
When running a test, if there is a failure is a step that occurs in a file that is required by the steps file (but not in the steps file itself), there is no stack trace or other real failure returned to the console or error logs. The only error message is "Unknown error" -->

[0-0] ::::failure::::
Screenshot saved to: C:\Workspace\webdriverio-test\sf3_tests\shots\shots_1565013543736.png
[0-0] Error in "Login and Logout: User Forgot Password Button, Retrieve Email, Then back out: And the user receives a "Forgot Password" email on the "email inbox" page"
**Uknown error.**
[0-0] FAILED in chrome - C:\Workspace\webdriverio-test\sf3_tests\features\login.feature

To Reproduce
Feature File

Given the user is on the "some" page
Then the "title" is "displayed" on "some" page

Steps File

const World = require (process.cwd() + '/share/world.js');
const { Given, When, Then } = require('cucumber')
const assert = require('assert');

let _page;

// go to a page
Given('the user is on the {string} page', (pageName) => {
    _page = World.getPage(pageName);
    _page.open();
});

// verify some element is in a particular state
Then('the {string} is {string} on the {string} page', (elementName, elementState, pageName) => {
    _page = World.getPage(pageName);
    elementName = World.camelizeString(elementName);
    elementState = World.pascalString(elementState);
    // example: userNameIsVisible
    let method = `${elementName}Is${elementState}`;
    _page[method]();
});

Create the page and World files to support the above steps, but make an error in the code somewhere on one of those pages (like forget to import assert for the then step, or make an error in the getPage, camelizeString, etc. functions). In the console, the only error that you get back will be "Unknown error"

Expected behavior
When there is a failure, I expect to get a stack trace with an error message stating what the error in the code was that caused the code to fail, with some file(s) and line number(s) indicating where in the code the failure occurred.

Log
from allure reporter -->

            <name>And the user receives a "Forgot Password" email on the "email inbox" page</name>
            <title>And the user receives a "Forgot Password" email on the "email inbox" page</title>
            <labels>
                <label name='language' value='javascript'/>
                <label name='framework' value='wdio'/>
                <label name='thread' value='0-0'/>
            </labels>
            <parameters>
                <parameter kind='argument' name='browser' value='chrome'/>
            </parameters>
            <steps/>
            <attachments/>
            <failure>
                <message>this.emailTitles(...).$ is not a function</message>
                <stack-trace>TypeError: this.emailTitles(...).$ is not a function
                    at EmailInbox.getEmail (C:\Workspace\webdriverio-test\sf3_tests\pages\emailInbox.page.js:26:44)
                    at World.Then (C:\Workspace\webdriverio-test\sf3_tests\steps\check_email.steps.js:12:11)
                    at Promise (C:\Workspace\webdriverio-test\sf3_tests\node_modules\@wdio\sync\build\index.js:65:22)
                    at World.executeSync (C:\Workspace\webdriverio-test\sf3_tests\node_modules\@wdio\sync\build\index.js:63:10)
                    at C:\Workspace\webdriverio-test\sf3_tests\node_modules\@wdio\sync\build\runFnInFiberContext.js:21:27</stack-trace>
            </failure
        </test-case>

Additional context
Add any other context about the problem here.

@mgrybyk
Copy link
Member

mgrybyk commented Aug 5, 2019

So, there is a log in allure reporter? Error like that are not printed out to console, this is an expected behavior.
You can use some reporter to see error message details, like stack.

Regarding that error is called unknown error, it is known issue and is fixed in #4288

@pyxis828
Copy link
Author

pyxis828 commented Aug 6, 2019

@mgrybyk So I am a little confused by your response. Are you saying that "Unknown error" will no longer be output to the console due to fixes from #4288? So will the actual stack trace then be printed to the console? I don't understand your statement that it is an expected behavior that the stack trace would not be printed to the console. In v4 the stack trace was always printed to the console, no matter where the error occurred. It is not reasonable (IMO) to expect the dev to always have to refer to the output of a reporter service (which is an add-on) to view stack trace and see why and where there was an error in their code. Additionally, allure reporter (in this instance) names their reports with what appears to be a rather random string (perhaps it is the session id), and it is hard to figure out which report is specifically associated with any given test run to pin down the stack trace that is relevant.

@mgrybyk
Copy link
Member

mgrybyk commented Aug 6, 2019

@pyxis828
f you want to print your errors, use reporter, ex spec

Stack traces are not printed out like this because of mess in console.

Logs are stored in files and are printed out by reporters

@mgrybyk
Copy link
Member

mgrybyk commented Aug 6, 2019

Closing this one as it refers to expected behavior and issue that gonna be fixed with another PR soon.

If you like, you make rise a feature request for printing stack traces

@mgrybyk mgrybyk closed this as completed Aug 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants