Flaky TestShadow

[AlexanderYastrebov]

--- FAIL: TestShadow (8.11s)
    shadow_test.go:252: client observes: total=100
    shadow_test.go:256: client observes: main=0, shadow=0, clientCancel=72
    shadow_test.go:269: client observes: statusFifoFull=1, statusFifoTimeout=27, statusFifoErr=0
    --- FAIL: TestShadow/50%_shadow_with_fifo_and_100%_timing_out_main (1.25s)
        shadow_test.go:368: backends observe: counter main=63, counter shadow=36
        shadow_test.go:393: Failed to check: fifo full 1

[szuecs]

 --- FAIL: TestShadow (8.11s)
    shadow_test.go:252: client observes: total=100
    shadow_test.go:256: client observes: main=0, shadow=0, clientCancel=72
    shadow_test.go:269: client observes: statusFifoFull=1, statusFifoTimeout=27, statusFifoErr=0
    --- FAIL: TestShadow/50%_shadow_with_fifo_and_100%_timing_out_main (1.25s)
        shadow_test.go:368: backends observe: counter main=63, counter shadow=36
        shadow_test.go:393: Failed to check: fifo full 1
time="2023-10-16T15:52:43Z" level=error msg="Error connecting to backend: dial tcp 127.0.0.1:39811: connect: connection refused"
2023/10/16 15:52:43 github.com/testcontainers/testcontainers-go - Connected to docker: 

Somehow I think the problem of many tests are the ones that use testcontainers...

[szuecs]

Requests      [total, rate, throughput]  100, 100.95, 0.00
Duration      [total, attack, wait]      1.105526036s, 990.587569ms, 114.938467ms
Latencies     [mean, 50, 95, 99, max]    92.666713ms, 125.442977ms, 126.125083ms, 126.497495ms, 126.516671ms
Bytes In      [total, mean]              0, 0.00
Bytes Out     [total, mean]              0, 0.00
Success       [ratio]                    0.00%
Status Codes  [code:count]               0:72  502:20  503:8  
Error Set:
Get "http://127.0.0.1:38611/": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
502 Bad Gateway
503 Service Unavailable
time="2023-12-08T11:44:51Z" level=info msg="client canceled after 125.276965ms, route main with backend network http://127.0.0.1:37933, status code 499: dialing failed false: context canceled, remote host: 127.0.0.1, request: \"GET / HTTP/1.1\", host: 127.0.0.1:38611, user agent: \"Go-http-client/1.1\""
--- FAIL: TestShadow (7.87s)
    --- FAIL: TestShadow/50%_with_fifo_and_slow_main (1.06s)
        shadow_test.go:368: backends observe: counter main=98, counter shadow=43
        shadow_test.go:372: client observes: total=98, expected=100
        shadow_test.go:376: client observes: main=98, shadow=0, clientCancel=0
        shadow_test.go:384: client observes: Qfull=0, Qtimeout=0, Qerr=0
        shadow_test.go:386: 98 != 98 or 98 != 100
    shadow_test.go:252: client observes: total=100
    shadow_test.go:256: client observes: main=0, shadow=0, clientCancel=72
    shadow_test.go:269: client observes: statusFifoFull=8, statusFifoTimeout=20, statusFifoErr=0
time="2023-12-08T11:44:51Z" level=error msg="Error connecting to backend: dial tcp 127.0.0.1:42473: connect: connection refused"

[szuecs]

Requests      [total, rate, throughput]  100, 101.07, 0.00
Duration      [total, attack, wait]      1.115360983s, 989.398602ms, 125.962381ms
Latencies     [mean, 50, 95, 99, max]    98.069562ms, 125.500835ms, 126.10726ms, 126.165714ms, 126.174501ms
Bytes In      [total, mean]              0, 0.00
Bytes Out     [total, mean]              0, 0.00
Success       [ratio]                    0.00%
Status Codes  [code:count]               0:76  502:23  503:1  
Error Set:
503 Service Unavailable
502 Bad Gateway
Get "http://127.0.0.1:33737/": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
--- FAIL: TestShadow (8.16s)
    shadow_test.go:252: client observes: total=100
    shadow_test.go:256: client observes: main=0, shadow=0, clientCancel=76
    shadow_test.go:269: client observes: statusFifoFull=1, statusFifoTimeout=23, statusFifoErr=0
    --- FAIL: TestShadow/50%_shadow_with_fifo_and_100%_timing_out_main (1.25s)
        shadow_test.go:368: backends observe: counter main=66, counter shadow=36
        shadow_test.go:393: Failed to check: fifo full 1