mc sends unnecessary status messages to stdout when it's a TTY #3499

kevinlul · 2020-11-20T21:10:09Z

Expected behavior

mc should behave consistently for testing so its output is parseable and informative logs shouldn't be in stdout but stderr

Actual behavior

On a first run in a tty, mc dumps versions of this to stdout before the proper output of its command, even with --json and --quiet. This makes it difficult to test scripts reproducibly that expect structured output.

mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/root/.mc/share`.
mc: Initialized share uploads `/root/.mc/share/uploads.json` file.
mc: Initialized share downloads `/root/.mc/share/downloads.json` file.

Steps to reproduce the behavior

Example docker-compose

version: "2"

services:
  minio:
    image: minio/minio
    environment:
      MINIO_ACCESS_KEY: docker
      MINIO_SECRET_KEY: inconsistent
    command: server /data
  mc:
    image: minio/mc
    environment:
      MC_HOST_app: http://docker:inconsistent@minio:9000
    depends_on:
      - minio
    command: admin --json user list app

docker-compose up: mc exits normally as expected, without any output
docker-compose run --rm -T mc: mc exits normally as expected, without any output since there is no TTY
docker-compose run --rm mc: mc dumps the above message to stdout

mc --version

I found this to occur in

mc version RELEASE.2020-11-17T00-39-14Z

and

mc version RELEASE.2020-10-03T02-54-56Z

System information

I've confirmed this to happen in Docker environments on WSL2, macOS, and Linux.

The text was updated successfully, but these errors were encountered:

harshavardhana · 2020-11-20T21:17:19Z

~ tree /tmp/garbage1/
/tmp/garbage1/ [error opening dir]

0 directories, 0 files

~ mc --config-dir /tmp/garbage1 --json ls /tmp/2 
{
 "status": "success",
 "type": "folder",
 "lastModified": "2020-11-17T23:26:17.202902722-08:00",
 "size": 4096,
 "key": ".minio.sys/",
 "etag": "",
 "url": "/tmp/2/",
 "versionOrdinal": 1
}
{
 "status": "success",
 "type": "folder",
 "lastModified": "2020-11-17T14:52:11.063079105-08:00",
 "size": 4096,
 "key": "testbucket/",
 "etag": "",
 "url": "/tmp/2/",
 "versionOrdinal": 1
}

~ tree /tmp/garbage1/
/tmp/garbage1/
├── certs
│   └── CAs
├── config.json
├── session
└── share
    ├── downloads.json
    └── uploads.json

Doesn't seem to be the case @kevinlul ?

kevinlul · 2020-11-20T21:19:06Z

Is this a fresh environment? If the files already exist in ~/.mc then I don't expect the message to appear.

harshavardhana · 2020-11-20T21:21:14Z

Is this a fresh environment? If the files already exist in ~/.mc then I don't expect the message to appear.

Yes @kevinlul

~ mv ~/.mc ~/.mc.old
~ mc --config-dir /tmp/garbage1 --json ls /tmp/2 
{
 "status": "success",
 "type": "folder",
 "lastModified": "2020-11-17T23:26:17.202902722-08:00",
 "size": 4096,
 "key": ".minio.sys/",
 "etag": "",
 "url": "/tmp/2/",
 "versionOrdinal": 1
}
{
 "status": "success",
 "type": "folder",
 "lastModified": "2020-11-17T14:52:11.063079105-08:00",
 "size": 4096,
 "key": "testbucket/",
 "etag": "",
 "url": "/tmp/2/",
 "versionOrdinal": 1
}
~ tree ~/.mc
/home/harsha/.mc [error opening dir]

0 directories, 0 files

harshavardhana · 2020-11-20T21:23:42Z

As per the code it should never be printed

                 if !globalQuiet && !globalJSON {
                        console.Infoln("Configuration written to `" + mustGetMcConfigPath() + "`. Please update your access credentials.")
                }

Are you sure your container has pulled the latest mc ?

kevinlul · 2020-11-20T21:29:42Z

I should clarify that I'm running this against MinIO specifically and not the local file system. When I run your commands, which deal with the local file system, the output is okay. However, if I try it against a cluster then I get the message output. It seems like the position of the --json flag also matters. First-run examples below:

/ # mc admin --json user list app
mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/root/.mc/share`.
mc: Initialized share uploads `/root/.mc/share/uploads.json` file.
mc: Initialized share downloads `/root/.mc/share/downloads.json` file.

/ # mc ls --json app
mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/root/.mc/share`.
mc: Initialized share uploads `/root/.mc/share/uploads.json` file.
mc: Initialized share downloads `/root/.mc/share/downloads.json` file.

/ # mc ls --json /tmp
mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/root/.mc/share`.
mc: Initialized share uploads `/root/.mc/share/uploads.json` file.
mc: Initialized share downloads `/root/.mc/share/downloads.json` file.

/ # mc --json ls /tmp
/ # mc --json ls app
/ # mc ls --json app
/ # mc admin --json user list app

harshavardhana · 2020-11-20T21:31:09Z

@kevinlul yes position matters and we can't do much about it - its a Limitation of Go flag parsing.

harshavardhana · 2020-11-20T21:31:33Z

I should clarify that I'm running this against MinIO specifically and not the local file system. When I run your commands, which deal with the local file system, the output is okay. However, if I try it against a cluster then I get the message output. It seems like the position of the --json flag also matters. First-run examples below:

It doesn't matter mc configuration creation has nothing to do with where you are talking to.

harshavardhana · 2020-11-20T21:32:18Z

So provide the flag in the beginning and we won't show the message that's it @kevinlul

kevinlul · 2020-11-20T21:36:41Z

If this is intended behaviour then maybe this is a better topic for documentation? I learned of the flag from the admin client docs, section 6, and I really did not expect this at all. In the two ways it is used in that part of the docs, --json appears after the mc admin subcommand in two different places, which suggests to the reader that it doesn't matter.

harshavardhana · 2020-11-20T21:38:33Z

If this is intended behaviour then maybe this is a better topic for documentation? I learned of the flag from the admin client docs, section 6, and I really did not expect this at all. In the two ways it is used in that part of the docs, --json appears after the mc admin subcommand in two different places, which suggests to the reader that it doesn't matter.

Not my intention TBH, it's a fundamental issue of Go parser position matters for behavior. So use it the way I just showed you, documentation can follow later.

kevinlul · 2020-11-20T21:48:25Z

Alright. If the Go parser position issue cannot be worked around, I suggest sending diagnostic messages to stderr if possible to avoid clobbering regular output. I will keep in mind to put the --json flag first.

minio/mc#3499

Add dataset-groups relationship Add admin-only query string parameter to assume user identity for /api/datasets, /api/families, /api/participants, /api/analyses Filter the following endpoints by group permissions for the current user or assumed user identity (#117) * GET /api/datasets * GET /api/datasets/:id * PATCH /api/datasets/:id * GET /api/participants * PATCH /api/participants/:id * GET /api/families * GET /api/families/:id * PATCH /api/families/:id * GET /api/tissue_samples/:id * GET /api/analyses * GET /api/analyses/:id * PATCH /api/analyses/:id General cleanup * Fix up some endpoint names * Prefer first_or_404 to one_or_none, except if query uses contains_eager * Avoid referring to the db model as "table" in PATCH endpoints Update docs for how to run tests offline to avoid inconsistent behaviour madmin: put the --json flag first to avoid future grief with minio/mc#3499 Co-authored-by: Kevin Lu <6320810+kevinlul@users.noreply.github.com>

stale · 2021-02-19T11:25:40Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

matthewdarwin · 2021-03-07T20:31:33Z

Is the plan still to update the docs for this issue?

stale · 2021-06-05T20:47:36Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

kevinlul · 2021-06-09T17:37:12Z

👀

stale · 2021-09-08T05:49:25Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

matthewdarwin · 2021-09-08T14:10:36Z

Is the plan still to update the docs for this issue?

stale · 2021-12-07T22:49:13Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

ravindk89 · 2022-03-29T15:02:57Z

The next-gen docs have already addressed this - the newer structure prioritizes optimal flag positioning

We are building a redesigned replacement for docs.min.io - I have avoided completely dropping docs.min.io in the meantime because there is some information there that is still helpful.

Instead it seems produent to simply redirect the pages that are no longer serving a useful purpose, such as the minio client quickstart guides and the admin quickstart guide.

Rewriting the existing examples on the legacy docs would not be the best use of our resources. Please follow minio/docs#454

stale · 2022-06-29T11:41:28Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

kevinlul added community triage labels Nov 20, 2020

kevinlul added a commit to ccmbioinfo/stager that referenced this issue Nov 20, 2020

Put the --json flag first to avoid future grief

1d4733d

minio/mc#3499

stale bot added the stale label Feb 19, 2021

stale bot removed the stale label Mar 7, 2021

stale bot added the stale label Jun 5, 2021

stale bot removed the stale label Jun 9, 2021

stale bot added the stale label Sep 8, 2021

stale bot removed the stale label Sep 8, 2021

stale bot added the stale label Dec 7, 2021

harshavardhana self-assigned this Jan 9, 2022

stale bot removed the stale label Jan 9, 2022

kannappanr assigned ravindk89 Mar 29, 2022

ravindk89 mentioned this issue Mar 29, 2022

Finish migrating mc admin to new reference structure, finish quickstarts for mc/mc admin minio/docs#454

Closed

3 tasks

stale bot added the stale label Jun 29, 2022

stale bot closed this as completed Oct 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mc sends unnecessary status messages to stdout when it's a TTY #3499

mc sends unnecessary status messages to stdout when it's a TTY #3499

kevinlul commented Nov 20, 2020

harshavardhana commented Nov 20, 2020

kevinlul commented Nov 20, 2020

harshavardhana commented Nov 20, 2020

harshavardhana commented Nov 20, 2020 •

edited

kevinlul commented Nov 20, 2020

harshavardhana commented Nov 20, 2020

harshavardhana commented Nov 20, 2020

harshavardhana commented Nov 20, 2020

kevinlul commented Nov 20, 2020

harshavardhana commented Nov 20, 2020

kevinlul commented Nov 20, 2020

stale bot commented Feb 19, 2021

matthewdarwin commented Mar 7, 2021

stale bot commented Jun 5, 2021

kevinlul commented Jun 9, 2021

stale bot commented Sep 8, 2021

matthewdarwin commented Sep 8, 2021

stale bot commented Dec 7, 2021

ravindk89 commented Mar 29, 2022

stale bot commented Jun 29, 2022

mc sends unnecessary status messages to stdout when it's a TTY #3499

mc sends unnecessary status messages to stdout when it's a TTY #3499

Comments

kevinlul commented Nov 20, 2020

Expected behavior

Actual behavior

Steps to reproduce the behavior

mc --version

System information

harshavardhana commented Nov 20, 2020

kevinlul commented Nov 20, 2020

harshavardhana commented Nov 20, 2020

harshavardhana commented Nov 20, 2020 • edited

kevinlul commented Nov 20, 2020

harshavardhana commented Nov 20, 2020

harshavardhana commented Nov 20, 2020

harshavardhana commented Nov 20, 2020

kevinlul commented Nov 20, 2020

harshavardhana commented Nov 20, 2020

kevinlul commented Nov 20, 2020

stale bot commented Feb 19, 2021

matthewdarwin commented Mar 7, 2021

stale bot commented Jun 5, 2021

kevinlul commented Jun 9, 2021

stale bot commented Sep 8, 2021

matthewdarwin commented Sep 8, 2021

stale bot commented Dec 7, 2021

ravindk89 commented Mar 29, 2022

stale bot commented Jun 29, 2022

harshavardhana commented Nov 20, 2020 •

edited