Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubernetes tracker meeting Empty domain #447

Open
jzp1025 opened this issue Aug 9, 2018 · 0 comments
Open

kubernetes tracker meeting Empty domain #447

jzp1025 opened this issue Aug 9, 2018 · 0 comments

Comments

@jzp1025
Copy link

jzp1025 commented Aug 9, 2018

Hi ! I'm trying to use dmlc-submit to submit a kubernetes job , and i meet the bug following:

To check each log, try 'kubectl logs job/{{role}}-{{jobname}}-{{workerID}}'
Traceback (most recent call last):
  File "./dmlc-submit", line 9, in <module>
    submit.main()
  File "/home/tusimple/incubator-mxnet/3rdparty/dmlc-core/tracker/dmlc_tracker/submit.py", line 54, in main
    kubernetes.submit(args)
  File "/home/tusimple/incubator-mxnet/3rdparty/dmlc-core/tracker/dmlc_tracker/kubernetes.py", line 143, in submit
    pscmd="echo \"To check each log, try 'kubectl logs job/{{role}}-{{jobname}}-{{workerID}}'\"")
  File "/home/tusimple/incubator-mxnet/3rdparty/dmlc-core/tracker/dmlc_tracker/tracker.py", line 428, in submit
    fun_submit(nworker, nserver, envs)
  File "/home/tusimple/incubator-mxnet/3rdparty/dmlc-core/tracker/dmlc_tracker/kubernetes.py", line 129, in kubernetes_submit
    resp = k8s_batch.create_namespaced_job(namespace=args.kube_namespace, body=mn_sh_job)
  File "build/bdist.linux-x86_64/egg/kubernetes/client/apis/batch_v1_api.py", line 58, in create_namespaced_job
  File "build/bdist.linux-x86_64/egg/kubernetes/client/apis/batch_v1_api.py", line 143, in create_namespaced_job_with_http_info
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 321, in call_api
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 155, in __call_api
  File "build/bdist.linux-x86_64/egg/kubernetes/client/api_client.py", line 364, in request
  File "build/bdist.linux-x86_64/egg/kubernetes/client/rest.py", line 266, in POST
  File "build/bdist.linux-x86_64/egg/kubernetes/client/rest.py", line 166, in request
  File "/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 72, in request
    **urlopen_kw)
  File "/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 150, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/usr/local/lib/python2.7/dist-packages/urllib3/poolmanager.py", line 322, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 849, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python2.7/dist-packages/urllib3/connection.py", line 367, in connect
    cert = self.sock.getpeercert()
  File "/usr/local/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", line 351, in getpeercert
    'subjectAltName': get_subj_alt_name(x509)
  File "/usr/local/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", line 228, in get_subj_alt_name
    for name in ext.get_values_for_type(x509.DNSName)
  File "/usr/local/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", line 181, in _dnsname_to_stdlib
    name = idna_encode(name)
  File "/usr/local/lib/python2.7/dist-packages/urllib3/contrib/pyopenssl.py", line 179, in idna_encode
    return idna.encode(name)
  File "/usr/local/lib/python2.7/dist-packages/idna/core.py", line 350, in encode
    raise IDNAError('Empty domain')
idna.core.IDNAError: Empty domain

i'm using the command below:

./dmlc-submit     --cluster kubernetes     --jobname test     --num-workers 
3     --num-server 1     --kube-worker-image jzp1025/mxnet:test     --kube-server-image jzp1025/mxnet:test     python /incubator-mxnet/example/image-classification/train_mnist.py --network=resnet --num-layers=110 --batch-size=128 --kv-store=dist_device_sync

looking for some help , thanks !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant