Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression tests fail for release builds #958

Open
hagau opened this issue Dec 16, 2022 · 1 comment
Open

Regression tests fail for release builds #958

hagau opened this issue Dec 16, 2022 · 1 comment
Labels
new New issue that has not been yet acknowledged
Milestone

Comments

@hagau
Copy link

hagau commented Dec 16, 2022

Platform:

  • OMNeT++ version: 5.7
  • OS: Ubuntu 22.04.1 LTS

Description:
I noticed that the regression testsuite was hardcoded to use the debug build, and I would like to test the optimized release builds instead. I thus modified the suite to run with the release binaries, which was easy enough to do.

Unfortunately, when run with release binaries, some of the tests failed, and more tests failed with more optimizations turned on.

For O0, none of the common tests failed
For O3, two from common failed: bigdecimal_3.test bigdecimal_4.test

For the core test, the behaviour is similar:
For O0, two fail: NED_paramcrossref_5.test signals_static_registration_1.test
For O3, six fail: NED_paramcrossref_5.test SimTime_ctor_1.test SimTime_div_1.test SimTime_preciseDiv_2.test c++-doubletolongconversion.test signals_static_registration_1.test
Adding -DNDEBUG=1 to the CFLAGS increases the number of failing test as well.

The expected and actual output for bigdecimal_3.test:

expected:

0.99999999999999999,1: < <=                                                                    
0.99999999999999999,123456789012345678: < <=                                                   
-1,-0.999999999999999999: < <= 

actual:

0.99999999999999999,1: < <=
0.99999999999999999,123456789012345678: >= >
-1,-0.999999999999999999: < <=

for SimTime_ctor_1:

--- expected output
+++ actual output
@@ -23,11 +23,11 @@
 96000*10^2 = ~9.6e+06 -> 9600000 (int64:9600000000000000) ok
 96000*10^3 = ~9.6e+07 -> 96000000 (int64:96000000000000000) ok
 96000*10^4 = ~9.6e+08 -> 960000000 (int64:960000000000000000) ok
-96000*10^5 = ~9.6e+09 -> ERROR: simtime_t overflow: Cannot represent 96000*10^5, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
-96000*10^6 = ~9.6e+10 -> ERROR: simtime_t overflow: Cannot represent 96000*10^6, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
-96000*10^7 = ~9.6e+11 -> ERROR: simtime_t overflow: Cannot represent 96000*10^7, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
-96000*10^8 = ~9.6e+12 -> ERROR: simtime_t overflow: Cannot represent 96000*10^8, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
-96000*10^9 = ~9.6e+13 -> ERROR: simtime_t overflow: Cannot represent 96000*10^9, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
+96000*10^5 = ~9.6e+09 -> -8846744073.709551616 (int64:-8846744073709551616) WRONG
+96000*10^6 = ~9.6e+10 -> 3766279631.45224192 (int64:3766279631452241920) WRONG
+96000*10^7 = ~9.6e+11 -> 769308167.103315968 (int64:769308167103315968) WRONG
+96000*10^8 = ~9.6e+12 -> 7693081671.03315968 (int64:7693081671033159680) WRONG
+96000*10^9 = ~9.6e+13 -> 3143840415.493390336 (int64:3143840415493390336) WRONG
 96000*10^10 = ~9.6e+14 -> ERROR: simtime_t overflow: Cannot represent 96000*10^10, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
 96000*10^11 = ~9.6e+15 -> ERROR: simtime_t overflow: Cannot represent 96000*10^11, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
 96000*10^12 = ~9.6e+16 -> ERROR: simtime_t overflow: Cannot represent 96000*10^12, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
@@ -58,11 +58,11 @@
 -96000*10^2 = ~-9.6e+06 -> -9600000 (int64:-9600000000000000) ok
 -96000*10^3 = ~-9.6e+07 -> -96000000 (int64:-96000000000000000) ok
 -96000*10^4 = ~-9.6e+08 -> -960000000 (int64:-960000000000000000) ok
--96000*10^5 = ~-9.6e+09 -> ERROR: simtime_t overflow: Cannot represent -96000*10^5, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
--96000*10^6 = ~-9.6e+10 -> ERROR: simtime_t overflow: Cannot represent -96000*10^6, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
--96000*10^7 = ~-9.6e+11 -> ERROR: simtime_t overflow: Cannot represent -96000*10^7, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
--96000*10^8 = ~-9.6e+12 -> ERROR: simtime_t overflow: Cannot represent -96000*10^8, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
--96000*10^9 = ~-9.6e+13 -> ERROR: simtime_t overflow: Cannot represent -96000*10^9, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
+-96000*10^5 = ~-9.6e+09 -> 8846744073.709551616 (int64:8846744073709551616) WRONG
+-96000*10^6 = ~-9.6e+10 -> -3766279631.45224192 (int64:-3766279631452241920) WRONG
+-96000*10^7 = ~-9.6e+11 -> -769308167.103315968 (int64:-769308167103315968) WRONG
+-96000*10^8 = ~-9.6e+12 -> -7693081671.03315968 (int64:-7693081671033159680) WRONG
+-96000*10^9 = ~-9.6e+13 -> -3143840415.493390336 (int64:-3143840415493390336) WRONG
 -96000*10^10 = ~-9.6e+14 -> ERROR: simtime_t overflow: Cannot represent -96000*10^10, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
 -96000*10^11 = ~-9.6e+15 -> ERROR: simtime_t overflow: Cannot represent -96000*10^11, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9
 -96000*10^12 = ~-9.6e+16 -> ERROR: simtime_t overflow: Cannot represent -96000*10^12, out of range (-9223372036.854775807,9223372036.854775807) allowed by scale exponent -9

Steps To Reproduce:

  1. In test/{common,core}, in each runtest change the MODE=debug to MODE=release and each work_dbg to work to run in release mode
  2. Run the core/common tests as usually
@hagau hagau added the new New issue that has not been yet acknowledged label Dec 16, 2022
@avarga
Copy link
Contributor

avarga commented Dec 21, 2022

This is important feedback, we'll check, thank you!

@rhornig rhornig modified the milestones: 6.0.2, 6.1 Oct 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new New issue that has not been yet acknowledged
Projects
None yet
Development

No branches or pull requests

3 participants