Run unit tests in Wine in CI

[Ericson2314]

Now that #8901 is merged, we can cross compile the unit test executables to windows, and (as was already the case for cross builds) install them in the check output.

We can't run them on the Unix platforms we cross compile them from directory, but we can run them in wine.

We should make a new hydraTest job that takes that check output and runs all the executables in it with Wine.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-on-windows/1113/109

[qknight]

@Ericson2314 i've executed the tests using /nix/store/xch07snwpzrs0ph946kmzgzp2zass2fh-wine-wow-8.0.2/bin/wine imperatively. what i noticed:

• to make it work for both linux (wine) and native windows 10: i copied all files into one directory (symlinks break everything)
• wine is like 10x slower than native
• except for libnixutil-tests.exe which crashes, all 4 tests are 1:1 to native windows
• export WINEDEBUG=-all so one does not get an interactive shell

libnixexpr-tests.exe

[0;32m[----------] [mGlobal test environment tear-down
[0;32m[==========] [m352 tests from 18 test suites ran. (12204 ms total)
[0;32m[  PASSED  ] [m330 tests.
[0;31m[  FAILED  ] [m22 tests, listed below:
[0;31m[  FAILED  ] [mNixStringContextElemTest.prop_round_rip
[0;31m[  FAILED  ] [mPrimOpTest.getEnv
[0;31m[  FAILED  ] [mPrimOpTest.baseNameOf
[0;31m[  FAILED  ] [mPrimOpTest.dirOf
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_value_set_get_int
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_value_set_get_float
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_value_set_get_bool
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_value_set_get_string
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_value_set_get_null
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_value_set_get_path
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_build_and_init_list
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_build_and_init_attr
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_expr_eval_external
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_expr_eval_from_string
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_expr_eval_add_numbers
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_expr_eval_drv
[0;31m[  FAILED  ] [mnix_api_expr_test.nix_build_drv
[0;31m[  FAILED  ] [mErrorTraceTest.filterSource
[0;31m[  FAILED  ] [mDerivedPathExpressionTest_RapidCheck.prop_opaque_path_round_trip
[0;31m[  FAILED  ] [mDerivedPathExpressionTest_RapidCheck.prop_derived_path_built_placeholder_round_trip
[0;31m[  FAILED  ] [mDerivedPathExpressionTest_RapidCheck.prop_derived_path_built_out_path_round_trip
[0;31m[  FAILED  ] [mtoString/ToStringPrimOpTest.toString/10, where GetParam() = ("builtins.toString ./test", "/test")

libnixfetchers-tests.exe

wine libnixfetchers-tests.exe
works!

libnixstore-tests.exe

wine libnixstore-tests.exe
all fail

libnixutil-tests.exe

wine libnixutil-tests.exe
...
[0;32m[----------] [m6 tests from decompress
[0;32m[ RUN      ] [mdecompress.decompressNoneCompressed
[0;32m[       OK ] [mdecompress.decompressNoneCompressed (0 ms)
[0;32m[ RUN      ] [mdecompress.decompressEmptyCompressed
[0;32m[       OK ] [mdecompress.decompressEmptyCompressed (0 ms)
[0;32m[ RUN      ] [mdecompress.decompressXzCompressed
terminate called after throwing an instance of 'nix::EndOfFile'

general

making this declarative, should the nix expression always assume 'success', i.e. ignore the output code so can check results manually?

[Ericson2314]

@qknight

libnixexpr-tests.exe

I think this is because it needs the environment variable set for the unit test data files?

except for libnixutil-tests.exe which crashes

oh no!

terminate called after throwing an instance of 'nix::EndOfFile'

This is new to me since I last ran the tests

making this declarative, should the nix expression always assume 'success', i.e. ignore the output code so can check results manually?

I think we want to run all tests no matter what, but it's OK to make the derivation fail if any one of them failed. Hopefully we aren't failing for long :).