Streaming CLI Output #608
Streaming CLI Output #608
Conversation
akutz
force-pushed
the
feature/streaming-cli-output
branch
from
cec92e9
to
bde5f83
Compare
|
@cduchesne Would you please test this when you get a chance? Thanks! |
Codecov Report
@@ Coverage Diff @@
## master #608 +/- ##
==========================================
- Coverage 34.16% 8.96% -25.21%
==========================================
Files 36 15 -21
Lines 2901 2299 -602
==========================================
- Hits 991 206 -785
- Misses 1807 2068 +261
+ Partials 103 25 -78
Continue to review full report at Codecov.
|
akutz
force-pushed
the
feature/streaming-cli-output
branch
from
bde5f83
to
5577d3f
Compare
This patch adds a new feature to REX-Ray's Volume CLI so that streaming
output is supported. By setting the environment variable
"REXRAY_CLI_TABWRITER_AUTOFLUSH" to a truthy value, the CLI will emit
results as they are produced rather than waiting until all results are
available. For example:
$ REXRAY_CLI_TABWRITER_AUTOFLUSH=true \
rexray volume create build1 build2 build3 --size 16
The above command will create three volumes named "build1", "build2",
and "build3" with a size of 16GB. With the above environment variable
set, however, the CLI will emit the volumes as they're created instead
of waiting until all three have been processed.
If an error occurs when processing multiple items, the error is written
to the console in the middle of the normal, tabular output. However, the
error is written to the STDERR file descriptor instead of STDOUT,
meaning it is very simple to pay attention to only the successful
results if so desired.
The streaming output is also supported for the "json" and "jsonp"
formats.
There is one caveat to using the streaming output option -- there is no
longer a guarantee that tabular data will be properly aligned. While the
data will still be separated by tabs, the data must normally be buffered
until it is all available to know how to properly produce the formatted
table. Streaming the data means padding can only be generated based upon
the current and previous entries, but not possible, future ones. For
example:
ID Name Status Size Path
vol-0 builds-0 available 10240
vol-1 builds-1 available 10240
error querying volume vol-2
vol-3 builds-3-abcdefghik available 10240
vol-4 builds-4 available 10240
The above output demonstrates two things: 1) an error emitted to STDERR
in the middle of the tabular data that has been written to STDOUT and 2)
the entry for "vol-3" is not properly aligned due to the fact that the
tabular formatting was calculated well in advance of the "vol-3" data
being taken into consideration.
|
Hi @cduchesne, Do you have some time to please verify this? Thank you. |
|
Hi @cduchesne, Do you have some time to please verify this? Thank you.
I think like the image above, @cduchesne may have gone off the deep-end with meetings. Can you possibly validate this PR? Thanks! |
|
Hi @akutz I grabbed your branch and built it, to test on CentOS7 with vbox driver. Operations like
The bigger issue I ran into, though, was that I couldn't create Volumes in vbox with rexray built from this branch, but I could from master. The debug output is large, so I put it in a gist: https://gist.github.com/codenrhoden/5f0246ea68cccd58d651efc5ea60865c The test version hangs after 'created libstorage client' -- the master version proceeds, and starts to use the executor. I do not that this PR needs to be rebased, so it may not be changes in this PR at all, but wanted to bring it up. Same config for both setups: libstorage:
service: virtualbox
virtualbox:
endpoint: http://10.0.2.2:18083
volumePath: /Users/travis/VirtualBox VMs/Volumes
controllerName: IDE Controller |
This patch adds a new feature to REX-Ray's Volume CLI so that streaming output is supported. By setting the environment variable
REXRAY_CLI_TABWRITER_AUTOFLUSHto a truthy value, the CLI will emit results as they are produced rather than waiting until all results are available.For comparison, here is what traditional, buffered output looks like:
When enabled, streaming output will appear as results are produced:
To better understand streaming output, please consider the following example:
The above command will create three volumes named
build1,build2, andbuild3with a size of 16GB. With the above environment variable set, however, the CLI will emit the volumes as they're created instead of waiting until all three have been processed.If an error occurs when processing multiple items, the error is written to the console in the middle of the normal, tabular output. However, the error is written to the
STDERRfile descriptor instead ofSTDOUT, meaning it is very simple to pay attention to only the successful results if so desired.The streaming output is also supported for the
jsonandjsonpformats.There is one caveat to using the streaming output option -- there is no longer a guarantee that tabular data will be properly aligned. While the data will still be separated by tabs, the data must normally be buffered until it is all available to know how to properly produce the formatted table. Streaming the data means padding can only be generated based upon the current and previous entries, but not possible, future ones. For example:
The above output demonstrates two things: 1) an error emitted to
STDERRin the middle of the tabular data that has been written toSTDOUTand 2) the entry forvol-3is not properly aligned due to the fact that the tabular formatting was calculated well in advance of thevol-3databeing taken into consideration.