Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci.yml: test result summary is confusing #14573

Open
svardew opened this issue Apr 17, 2024 · 2 comments
Open

ci.yml: test result summary is confusing #14573

svardew opened this issue Apr 17, 2024 · 2 comments
Labels

Comments

@svardew
Copy link
Contributor

svardew commented Apr 17, 2024

Steps to reproduce

  1. build vim via .github/workflow/ci.yml
  2. open job result, windows (HUGE, mingw, yes, yes, x86, stable, yes)
  3. view step log, Test and show the result of testing Vim
  4. on this step, some oldstyle tests are done.
    but summary report Execute/Skip/Fail is 0.
  if exist dostmp\test23.in del dostmp\test23.in
  ..\vim -u dos.vim -U NONE --noplugin --not-a-term --gui-dialog-file guidialog "+set ff=dos|f dostmp\test23.in|wq" test23.in
  move test23.in test23.in.bak > nul
  copy dostmp\test23.in test23.in > nul
  copy test23.ok test.ok > nul
  ..\vim -u dos.vim -U NONE --noplugin --not-a-term --gui-dialog-file guidialog -s dotest.in test23.in
  ..\vim -u dos.vim -U NONE --noplugin --not-a-term --gui-dialog-file guidialog "+set ff=unix|f test.out|wq"  dostmp\test23.out
  if not exist dostmp\NUL md dostmp
  if exist dostmp\test24.in del dostmp\test24.in
Test results:

Skipped:

-------------------------------
Executed:     0 Tests
 Skipped:     0 Tests
  Failed:     0 Tests

ALL DONE

Expected behaviour

if summary is correct, no test is done. this step shoud be skipped.
if summary is wrong, fix to show testing result correctly.

Version of Vim

9.1.0252 and 9.0.0421

Environment

runs-on: windows-2022
(from steps Check version)
VIM - Vi IMproved 9.1 (2024 Jan 02, compiled Apr 17 2024 07:54:36)
MS-Windows 32-bit GUI/console version
Included patches: 1-252
Compiled by runneradmin@fv-az731-743
Huge version with GUI.

Logs and stack traces

No response

@svardew svardew added the bug label Apr 17, 2024
@chrisbra
Copy link
Member

chrisbra commented Apr 17, 2024

I don't understand. If I go to one of those logs (HUGE, mingw, yes, yes, x86, stable, yes), it does show correct test statistics:

-------------------------------
Executed:  6212 Tests
 Skipped:   762 Tests
  Failed:     0 Tests

Please be more precise at what you did and what you saw.

@chrisbra
Copy link
Member

ah, you are talking about testing console vim.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants