Run experiment without web server

[YiChunHung]

What would you like to be added:
Support running experiment without actually launch web server/ui.

Why is this needed:
As the normal user in the shared resource, it is usually that normal users have no auth to open a specific port.

Without this feature, how does current nni work：
As the doc mentions, nni would create a web server to control the experiment.

Components that may involve changes:
nni web server

Brief description of your proposal if any:
nni can probably just log down the necessary information, such as metric lists and intermediate result. After that, the user can download the log files to the local laptop for the visualization.

[SparkSnail]

Hi, in remote/pai/kubeflow mode, NNI need another port to accept metric data from trail_keeper, so if we support this kind of running mode, NNI may need a lot of refactor change, it might be a big effort. As we discussed internal, we may not support this feature. If you have another good suggestion, please feel free to give feedback. Thanks!

[scarlett2018]

reopen, as I'm thinking whether we can save the key results info in a JSON or HTML format just for browsing. open to discussion.

[notabigfish]

Same need. any updates?

[rockcor]

maybe you can write shell script like

#!/bin/bash
for file in nni-experiments/$1/trials/*
do
    if test -d $file
    then
         log=`cat "$file/trial.log" | grep 'test_acc'`
        echo "$file""$log"
    fi
done

To sort your metric, use ./nni_best.sh <expID> | sort -n -k 4 -t :

[lukelu312]

same here +1, hope to hear your expertise on the latest progress about the feature.