Remove --verbose flag from travis-ci

[mrocklin]

Fixes #3441

I'm not sure that we should do this yet, but I thought I'd put up a PR to see how it looked.

[jrbourbeau]

LGTM. I'd say this is an improvement over having to download the raw logs on Travis.

[jakirkham]

Not sure how I feel about this. OTOH it is nice to see which functions have run in a module (particularly to know if the tests one has added passed or not). OTOH having to look at raw logs hurts usability (especially when linking specific lines for issues to share with others).

[mrocklin]

Agreed that there is a tradeoff here. I'm also not sure how I feel about it. I anticipate that there would be times where people have to turn back on verbose logs when something hangs and they need to see what, for example.

…

On Wed, May 9, 2018 at 11:16 PM, jakirkham ***@***.***> wrote: Not sure how I feel about this. OTOH it is nice to see which functions have run in a module (particularly to know if the tests one has added passed or not). OTOH having to look at raw logs hurts usability (especially when linking specific lines for issues to share with others). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3477 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASszJ7YJWKgqoRNeXGe2RMJ1rm8N1p9ks5tw7D5gaJpZM4T2eW2> .

[mrocklin]

@jakirkham any changes in thoughts here?

[mrocklin]

When tracking down errors recently (due to the requests bug) I've found myself wishing that verbose wasn't set. I think I always want to look at the error report, and that includes the names of the failed tests.

[jakirkham]

Is there some way to get just the names of the modules tested listed instead of every function? That might strike a better balance between keeping the log usable and providing early indication of where failures have occurred.

[mrocklin]

mrocklin@carbon:~/workspace/dask$ py.test  dask 
============================= test session starts ==============================
platform linux -- Python 3.6.4, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
rootdir: /home/mrocklin/workspace/dask, inifile: setup.cfg
plugins: timeout-1.2.1, cov-2.5.1
collected 5442 items / 4 skipped                                               

dask/array/tests/test_array_core.py ...............XX................... [  0%]
........................................................................ [  1%]
.....................x.................................................. [  3%]
..........................................x..s.......................... [  4%]
............ssss........s                                                [  5%]
dask/array/tests/test_chunk.py .....                                     [  5%]
dask/array/tests/test_creation.py .......x.............................. [  5%]
..........................                                                                                  [  6%]
dask/array/tests/test_fft.py ........................................... [  7%]
...............s.s.........s.s.........s.s.s.s.........s.s.s.s.......... [  8%]
........................................................................ [  9%]
........................................................................ [ 11%]
..                                                                                                          [ 11%]
dask/array/tests/test_ghost.py ......................x...                                                   [ 11%]
dask/array/tests/test_gufunc.py ........X.................                                                  [ 12%]
dask/array/tests/test_image.py ...                                                                          [ 12%]
dask/array/tests/test_linalg.py .......................................s [ 12%]
..sssssss............................................................... [ 14%]
........................................................................ [ 15%]
.................                                                                                           [ 15%]
dask/array/tests/test_linearoperator.py .                                                                   [ 15%]

[jakirkham]

Ha...very well :)

[jakirkham]

So the modules show up in one of the matrix builds, but not all of them. For example, this one is just .s. Any ideas as to how we could correct them to also have module names?

[mrocklin]

Maybe it changed in pytest versions?

…

On Sun, Jun 17, 2018 at 1:33 AM, jakirkham ***@***.***> wrote: So the modules show up in one of the matrix builds, but not all of them. For example, this one is just .s <https://travis-ci.org/dask/dask/jobs/376321534>. Any ideas as to how we could correct them to also have module names? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3477 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASszE2XIv-ej9UgdnJJkaA_pXC2Nj2nks5t9eo2gaJpZM4T2eW2> .

[jrbourbeau]

After fiddling around a little bit, it doesn't seem to matter what the pytest version is (pinned version 3.1.1 vs. the latest release 3.6.1)

(dask_dev) ➜  dask git:(master) ✗ py.test dask/bytes/
======================================================== test session starts ========================================================
platform darwin -- Python 3.6.4, pytest-3.1.1, py-1.5.3, pluggy-0.4.0
rootdir: /Users/jbourbeau/software/dask, inifile: setup.cfg
collected 82 items / 1 skipped

dask/bytes/tests/test_bytes_utils.py .........
dask/bytes/tests/test_compression.py .....
dask/bytes/tests/test_hdfs.py ssssssssssssssssssssssss
dask/bytes/tests/test_http.py ......s.
dask/bytes/tests/test_local.py .............................s.....s

=============================================== 55 passed, 28 skipped in 3.74 seconds ===============================================

But using the pytest-xdist plugin for parallel testing is where the file names go away

(dask_dev) ➜  dask git:(master) ✗ py.test -n3 dask/bytes/
======================================================== test session starts ========================================================
platform darwin -- Python 3.6.4, pytest-3.1.1, py-1.5.3, pluggy-0.4.0
rootdir: /Users/jbourbeau/software/dask, inifile: setup.cfg
plugins: xdist-1.22.2, forked-0.2
gw0 [82] / gw1 [82] / gw2 [82]
scheduling tests via LoadScheduling
............s.ssssss.sssssssssssssssss..................................s.....ss..
=============================================== 55 passed, 27 skipped in 3.39 seconds ===============================================

Some initial searching didn't turn up a way to have both parallel testing and print the module names without also printing the individual test names.

[jakirkham]

Right pytest-xdist doesn’t currently have a mechanism to run modules on specific workers. Getting the correct verbosity after that may be possible. Some effective strategies for test distribution are at the end of the issue.

xref: pytest-dev/pytest-xdist#18