setup and teardown per test function

[pepdiz]

Should be possible to have a setup and teardown functions associated to each test function?

AFAIK actually you have a setup function that is executed previously to each test and a teardown function executed after each test, but the setup and teardown functions are common to all tests.

This way you have to include the cleanup (or setup) for all tests in the same function even when certain cleanup (or setup) is not needed in certain tests.

As an example, supose I have a two tests, one require to create a file and then delete it, the other doesn't require anything:

test_test1 () {
  assert_equals 1 1
}

test_test2 () {
   F=$(mktemp -d)
   assert_status_code 0 "test -f $F"
   rm $F
}

the problem with this pattern is if exit code of assertion is not 0 it never executes the cleanup (here, the rm $F) .

The right way to do this is using setup and teardown functions, but since those functions are common for all tests you end up setting or cleaning things not needed for certain tests:

setup () {
  export F=$(mktemp -d)
}

teardown () {
  [ -f $F ] && rm $F
}

but since setup is executed for all tests it will create a file also for test 1 which is not needed and teardown will delete a file not needed to be created neither deleted.

This way I'm forced to take into account all cases in setup and teardown, for example if I add another test checking for directoy existence I would need a new variable and know in the test which one to use:

setup () {
  export F=$(mktemp)
  export D=$(mktemp -d)
}

teardown () {
  [ -f $F ] && rm $F
  [ -d $D ] && rm -rf $D 
}

test_test1 () {
  assert_equals 1 1
}

test_test2 () {
   assert_status_code 0 "test -f $F"
}

test_test3 () {
   assert_status_code 0 "test -d $D"
}

And everyting gets bloated quickly.

This would be solved if we have a setup and teardow funcion for each test function, maybe using a pattern such as "setup_$testname" and "teardown_$testname" for example:

setup_test2 () {
  export F=$(mktemp)
}

setup_test3 () {
  export F=$(mktemp -d)
}

teardown_test2 () {
  [ -f $F ] && rm $F
}

teardown_test3 () {
  [ -f $F ] && rm -rf $F
}

test_test1 () {
  assert_equals 1 1
}

test_test2 () {
   assert_status_code 0 "test -f $F"
}

test_test3 () {
   assert_status_code 0 "test -d $F"
}

In short what I asking for is setup and teardown functions associated to each individual test, similar to @beforeeach and @AfterEach in jUnit ( current setup and teardown would be similar to @BeforeAll and @afterall in jUnit )

[pgrange]

Hello @pepdiz,

Thanks for using bash_unit and opening this issue.

I'm not sure what could be the consequences of such a fine grain setup/teardown system.

If I may, let me share with you how I would handle the situation you describe with the current version of bash_unit. For this situation, I would use several test files, one for each kind of context, should they be very dissimilar, I want my tests to be run into.

But before that, please note:

• setup and teardown are executed before each test in the current test file, which is similar to what @BeforeEach and @AfterEach do in jUnit
• setup_suite and teardown_suite are run only once for all the tests in the current test file, which could be compared to what @BeforeAll and @AfterAll do in jUnit.

I can't think of any way to solve your current issue in jUnit if you put all your tests in the same class.

That's pretty much the same with bash_unit if you put all your tests in the same file, they all share the same setup and teardown functions. But if you separate them in different files then you can have a different setup.

So to use your previous example, That would end up with something (quite artificial here but we're ok that this is an example, right?)

#> cat example_tests
test_test1 () {
  assert_equals 1 1
}

#> example_tests_with_dir
test_test2 () {
   assert_status_code 0 "test -f $F"
}

setup () {
  export F=$(mktemp -d)
}

teardown () {
  [ -f $F ] && rm $F
}

If you take a look at bash_unit test code, for instance, you'll see that the tests are split between several test files, trying to represent cohesive kinds of tests. Each of these tests files can have a specific setup or teardown function.

I hope this may help.

You may also take a look at this section of the bash_unit doc that gives brief explanation of setup/teardown functions: https://github.com/pgrange/bash_unit#how-to-write-tests

[pepdiz]

Ok, thanks for your reply, sure I can solve the problem grouping tests in related classes but this is somehow related to an issue that may become an aesthetics problem, let me explain:

I have a bash script that uses functions defined in the script and I want to test the functions but I cannot include the scripts in a setup function in tests because this will execute the script, the obvious solutions is to split functions and main body in different files so I can source the functions in the test setup and test functions properly.

But this forces me to include the functions in the main script via source and I don't want to do that because functions are only used in the script and I want the script to be a monolithic script rather than splitted in different files.

I really don't see a solution to this other than splitting functions in different files, do you have an idea to avoid this?

[pgrange]

That is a brillant question indeed. One that deserves some examples that were definitely missing from bash_unit repository before.

I just added an example in bash_unit to treat this specific topic: test_hello_i18n.

As you can see, in hello_i18n we have this function hello that we wish to test. Of course, if we only source the script under test, the script is executed and we are not happy with that. So to prevent that, we encapsulate the main code of the script in a function named main and we ensure that this function is only executed when we run the script itself, not when we source it.

This is how the script ensures that it is not currently sourced:

[[ "$0" == "${BASH_SOURCE[0]}" ]] && main "$@" || true

It's a pattern which I often used in my production code to deal with the situation you just described. But I never thought of sharing it. I hope this can help you.

Moreover, but that's a different issue, your question reminded me of a mirror situation for which I also have a pattern that I never shared before. It is when you want to actually run the script in your test but want to fake some internal functions your script defines and calls. For instance if your script depends on the current time. If you only fake the function, the script will override the fake version by the actual version when executed. To prevent that, you need to adapt your script. It is described in hello_timed.

[pepdiz]

thanks a lot, it's really a smart trick that I will add to my toolcase ;-) Sadly it only works in bash so not portable, but I use bash most of the time

[pepdiz]

Looking at hello_timed example I'm a bit confused about how it works, as far as I know scripts called by other scripts do not have access to caller environment, that is called script don't see variables or functions defined in caller script. So in the test_hello_timed example the "declare -F current_hour" should return 1 and define the function, rather than return 0 as is the function is already defined, to do that you "export -f current_hour" in test_hello_timed script prior to call hello_timed.

I suppose the key here is the fake function that somehow makes the call to hello_timed to occur in the same environment than test_hello_time, similar as the call to hello timed was "source ../hello_timed" rather than "$(../hello_timed)", can you explain the mechanism used here? I'm curious.

[pgrange]

Hello @pepdiz and sorry for the delay.

You’re right. It is fake which takes care of exporting the defined function.

see https://github.com/pgrange/bash_unit/blob/master/bash_unit#L165