Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError, Generators Being Added #25

Open
dcdanko opened this issue Jun 29, 2019 · 2 comments
Open

TypeError, Generators Being Added #25

dcdanko opened this issue Jun 29, 2019 · 2 comments

Comments

@dcdanko
Copy link
Contributor

dcdanko commented Jun 29, 2019 •
edited

Maybe an issue with numpy version. I have v1.11.3

$ PlasFlow.py --input scaff1k.fa --output scaff.plasflow.csv --threshold 0.7
...
Traceback (most recent call last):
  File "/home/dcdanko/miniconda3/envs/plasflow/bin/PlasFlow.py", line 346, in <module>
    vote_proba = vote_class.predict_proba(inputfile)
  File "/home/dcdanko/miniconda3/envs/plasflow/bin/PlasFlow.py", line 302, in predict_proba
    avg = np.average(self.probas_, axis=0, weights=self.weights)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/numpy/lib/function_base.py", line 1110, in average
    avg = a.mean(axis)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/numpy/core/_methods.py", line 70, in _mean
    ret = umr_sum(arr, axis, dtype, out, keepdims)
TypeError: unsupported operand type(s) for +: 'generator' and 'generator'
@dcdanko
Copy link
Contributor Author

dcdanko commented Jun 29, 2019

Seems to be related to this issue. Got this by calling list() on the generators.

Tensorflow version is the same listed on the README

Succesfully read previously calculated kmer frequencies for kmer 5
Predicting labels using kmer 5  frequencies
WARNING:tensorflow:From /home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/estimators/dnn.py:378 in predict_proba.: calling BaseEstimator.predict (from tensorflow.contrib.learn.python.learn.estimators.estimator) with x is deprecated and will be removed after 2016-12-01.
Instructions for updating:
Estimator is decoupled from Scikit Learn interface by moving into
separate class SKCompat. Arguments x, y and batch_size are only
available in the SKCompat class, Estimator will only accept input_fn.
Example conversion:
  est = Estimator(...) -> est = SKCompat(Estimator(...))
WARNING:tensorflow:From /home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/estimators/dnn.py:378 in predict_proba.: calling BaseEstimator.predict (from tensorflow.contrib.learn.python.learn.estimators.estimator) with batch_size is deprecated and will be removed after 2016-12-01.
Instructions for updating:
Estimator is decoupled from Scikit Learn interface by moving into
separate class SKCompat. Arguments x, y and batch_size are only
available in the SKCompat class, Estimator will only accept input_fn.
Example conversion:
  est = Estimator(...) -> est = SKCompat(Estimator(...))
WARNING:tensorflow:From /home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/estimators/dnn.py:378 in predict_proba.: calling BaseEstimator.predict (from tensorflow.contrib.learn.python.learn.estimators.estimator) with as_iterable is deprecated and will be removed after 2016-12-01.
Instructions for updating:
Estimator is decoupled from Scikit Learn interface by moving into
separate class SKCompat. Arguments x, y and batch_size are only
available in the SKCompat class, Estimator will only accept input_fn.
Example conversion:
  est = Estimator(...) -> est = SKCompat(Estimator(...))
WARNING:tensorflow:float64 is not supported by many models, consider casting to float32.
Traceback (most recent call last):
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1021, in _do_call
    return fn(*args)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1003, in _run_fn
    status, run_metadata)
  File "/home/dcdanko/miniconda3/envs/plasflow/lib/python3.5/contextlib.py", line 66, in __exit__
    next(self.gen)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 469, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.NotFoundError: Tensor name "dnn/hiddenlayer_0/biases" not found in checkpoint files /home/dcdanko/miniconda3/envs/plasflow/bin/models/kmer5_split_20_20_neurons_relu/model.ckpt-50000-?????-of-00001
	 [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dcdanko/miniconda3/envs/plasflow/bin/PlasFlow.py", line 331, in <module>
    vote_proba = vote_class.predict_proba(inputfile)
  File "/home/dcdanko/miniconda3/envs/plasflow/bin/PlasFlow.py", line 283, in predict_proba
    self.probas_ = [list(clf.predict_proba_tf(X)) for clf in self.clfs]
  File "/home/dcdanko/miniconda3/envs/plasflow/bin/PlasFlow.py", line 283, in <listcomp>
    self.probas_ = [list(clf.predict_proba_tf(X)) for clf in self.clfs]
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/estimators/dnn.py", line 380, in <genexpr>
    return (pred[key] for pred in preds)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 902, in _infer_model_as_iterable
    restore_checkpoint_path=checkpoint_path):
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/graph_actions.py", line 858, in run_feeds_iter
    _restore_from_checkpoint(session, g, restore_checkpoint_path)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/graph_actions.py", line 95, in _restore_from_checkpoint
    saver.restore(session, checkpoint_path)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1388, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 766, in run
    run_metadata_ptr)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 964, in _run
    feed_dict_string, options, run_metadata)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1014, in _do_run
    target_list, options, run_metadata)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1034, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Tensor name "dnn/hiddenlayer_0/biases" not found in checkpoint files /home/dcdanko/miniconda3/envs/plasflow/bin/models/kmer5_split_20_20_neurons_relu/model.ckpt-50000-?????-of-00001
	 [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

Caused by op 'save/RestoreV2', defined at:
  File "/home/dcdanko/miniconda3/envs/plasflow/bin/PlasFlow.py", line 331, in <module>
    vote_proba = vote_class.predict_proba(inputfile)
  File "/home/dcdanko/miniconda3/envs/plasflow/bin/PlasFlow.py", line 283, in predict_proba
    self.probas_ = [list(clf.predict_proba_tf(X)) for clf in self.clfs]
  File "/home/dcdanko/miniconda3/envs/plasflow/bin/PlasFlow.py", line 283, in <listcomp>
    self.probas_ = [list(clf.predict_proba_tf(X)) for clf in self.clfs]
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/estimators/dnn.py", line 380, in <genexpr>
    return (pred[key] for pred in preds)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 902, in _infer_model_as_iterable
    restore_checkpoint_path=checkpoint_path):
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/graph_actions.py", line 858, in run_feeds_iter
    _restore_from_checkpoint(session, g, restore_checkpoint_path)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/graph_actions.py", line 93, in _restore_from_checkpoint
    saver = saver or _make_saver(graph)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/graph_actions.py", line 86, in _make_saver
    max_to_keep=keep_checkpoint_max)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1000, in __init__
    self.build()
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1030, in build
    restore_sequentially=self._restore_sequentially)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 620, in build
    restore_sequentially, reshape)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 404, in _AddShardedRestoreOps
    name="restore_shard"))
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 361, in _AddRestoreOps
    tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 200, in restore_op
    [spec.tensor.dtype])[0])
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/ops/gen_io_ops.py", line 441, in restore_v2
    dtypes=dtypes, name=name)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
    op_def=op_def)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2240, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/dcdanko/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1128, in __init__
    self._traceback = _extract_stack()

NotFoundError (see above for traceback): Tensor name "dnn/hiddenlayer_0/biases" not found in checkpoint files /home/dcdanko/miniconda3/envs/plasflow/bin/models/kmer5_split_20_20_neurons_relu/model.ckpt-50000-?????-of-00001
	 [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

@dcdanko
Copy link
Contributor Author

dcdanko commented Jun 30, 2019

Going through this more thoroughly I believe this issue is similar to Issue #6

Manually inspecting the checkpoint files suggests that the layers in the files are named hiddenlayer_0/biases without the leading dnn/. Probably an arcane issue relating to very old TensorFlow versions.

I haven't yet been able to find any resources on migrating checkpoints to newer versions of TensorFlow but it is hard to imagine that this cannot be done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@dcdanko