e2e: add e2e for canary weight #257
Conversation
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #257 +/- ##
==========================================
- Coverage 44.58% 44.46% -0.13%
==========================================
Files 32 32
Lines 5275 5290 +15
==========================================
Hits 2352 2352
- Misses 2754 2769 +15
Partials 169 169 |
| } | ||
|
|
||
| // MakeRequestAndCountExpectedResponse make 'totReq' requests and determine whether to test results according to 'fn' callback function | ||
| func MakeRequestAndCountExpectedResponse(t *testing.T, r roundtripper.RoundTripper, gwAddr string, expected Assertion, totReq int, fn func(int, int) bool) { |
There was a problem hiding this comment.
| func MakeRequestAndCountExpectedResponse(t *testing.T, r roundtripper.RoundTripper, gwAddr string, expected Assertion, totReq int, fn func(int, int) bool) { | |
| func MakeRequestAndCountExpectedResponse(t *testing.T, r roundtripper.RoundTripper, gwAddr string, expected Assertion, totReq, minSuccessCount int) { |
Why not just use minSuccessCount instead of fn? Then the expected information of the test result would be more helpful, as follows: expected minSuccessCount is 40, actual count is 10
There was a problem hiding this comment.
But in some cases, we might want the values to be in a range. eg 40 < x < 60 🤔
There was a problem hiding this comment.
why not just a success Rate ?
There was a problem hiding this comment.
I think the total requests should be managed by test suite, and it is not exposed. I think the pkg user is more care about the success or fail rate ?
There was a problem hiding this comment.
I agree. but if the expected rate is 50%, and there is always a slight deviation between the real rate and expected rate(50%). I am not sure how to control the deviation within a reasonable range, so I give control to the user
There was a problem hiding this comment.
Maybe should have a accepted error interval to set up. Like if the accepted error interval is 5%, and expected rate is 50%, then we accept 45 to 55 %
Charlie17Li
force-pushed
the
e2e/canary-weigth-50
branch
from
b426b57
to
5942909
Compare
|
@Charlie17Li Hi, the e2e test cases seems failed. |
Signed-off-by: charlie <qianglin98@qq.com>
Signed-off-by: charlie <qianglin98@qq.com>
Charlie17Li
force-pushed
the
e2e/canary-weigth-50
branch
from
e446541
to
01d5dd7
Compare
|
@Charlie17Li Is there any progress on this? |
Ⅰ. Describe what this PR did
add an e2e testcase for the canary weight pattern
Ⅱ. Does this pull request fix one issue?
Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews