Closure Style BDD (Jest and Ginkgo)

[nickolasrm]

Hey guys I was looking into pytest-bdd and I found really interesting the way things work here!

I'm used to Jest and Ginkgo BDD way of testing and I found their way of implementing really easy to write, and hard to avoid coupling functions as well. Which in my opinion is a plus 1 for pytest-bdd of separating code from the Gherkin definition.

Note
These are only my thoughts and they might not reflect the real problem regarding the closure like implementation of BDD

Closure-Like BDD

Tools like Jest and Ginkgo closures and functions to define what testing block they are executing. Take this test from Ginkgo docs for instance:

var _ = Describe("Books", func() {
  var foxInSocks, lesMis *books.Book

  BeforeEach(func() {
    lesMis = &books.Book{
      Title:  "Les Miserables",
      Author: "Victor Hugo",
      Pages:  2783,
    }

    foxInSocks = &books.Book{
      Title:  "Fox In Socks",
      Author: "Dr. Seuss",
      Pages:  24,
    }
  })

  Describe("Categorizing books", func() {
    Context("with more than 300 pages", func() {
      It("should be a novel", func() {
        Expect(lesMis.Category()).To(Equal(books.CategoryNovel))
      })
    })

    Context("with fewer than 300 pages", func() {
      It("should be a short story", func() {
        Expect(foxInSocks.Category()).To(Equal(books.CategoryShortStory))
      })
    })
  })
})

In the example above, the BDD tree is built using the closures inside closures. By using this inner function relationship, the following tests would be queued:

Books -- Categorizing books with fewer than 300 pages should be a short story

We didn't have to specify the way functions relate, it's like a parent/child relationship, while the types are merely markers for building the test title sentence.

Closure Problem

In Python it seems hard to implement such feature because we don't have the concept of closures like in Go or JS. To illustrate better, it isn't possible to do something like this

scenario("abc", lambda x: 
    code
    code
    ...
)

Nor

scenario("abc", def foo():
    code
    code
    ...
)

Inner Functions

However, I think it is possible to overcome this limitation by using decorators and inner functions. Take the code below as an example:

@scenario 
def bookstore():
    @given 
    def a_user():
        @beforeach
        def create_user():
            ...
        @given 
        def a_book():
            @beforeach
            def create_book():
                 ...
            @when
            def user_buys():
                @then 
                def book_cant_be_borrowed():
                    ...
        @given
        def a_cd():
            @beforeach
            def create_cd():
                ...
            @then 
            def cd_count_reduces():
                ...

In order to make it work, the decorated functions shouldn't execute their inner code at first sight, but they should be allocated into a tree-like structure and then parsed separately as test functions. This would result in tests like this:

def test_1():
    create_user()
    create_book()
    book_cant_be_borrowed()

def test_2():
    create_user()
    create_cd()
    cd_count_reduces()

Implementation Challenges

Of course these "before something" hook functions are only added here if it is not possible to generate test entries dynamically. They are only a way to distinguish between leaf nodes and what is the test structure.

In my opinion it would be nice to find a way to add tests dynamically, so at the same time the scenario function is executing, leaf functions are found and executed.

Considerations

I personally prefer this way of writing tests with BDD and I understand it might be complex to reproduce other frameworks behaviors having a language limitation.

I also think the proposed idea doesn't relate very well with pytest's fixtures concept, and I think this is something that should have more thoughts proceeding to an implementation stage.

This tool was designed for using Gherkin language as part of its working logic and I am not sure if the community would agree with the method proposed here, but I personally like the closure due to the writing and bug detection simplification.