Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

training error #75

Open
noranali opened this issue Apr 25, 2022 · 1 comment
Open

training error #75

noranali opened this issue Apr 25, 2022 · 1 comment

Comments

@noranali
Copy link

noranali commented Apr 25, 2022

thank you for sharing your code i have an error while trying to train using colab notebook can you help me please?
the error is
img_in, img_gt (16, 256, 256, 3) (16, 256, 256, 3)
g_net/enc1_1/weights:0
g_net/enc1_1/biases:0
g_net/enc1_2/conv1/weights:0
g_net/enc1_2/conv1/biases:0
g_net/enc1_2/conv2/weights:0
g_net/enc1_2/conv2/biases:0
g_net/enc1_3/conv1/weights:0
g_net/enc1_3/conv1/biases:0
g_net/enc1_3/conv2/weights:0
g_net/enc1_3/conv2/biases:0
g_net/enc1_4/conv1/weights:0
g_net/enc1_4/conv1/biases:0
g_net/enc1_4/conv2/weights:0
g_net/enc1_4/conv2/biases:0
g_net/enc2_1/weights:0
g_net/enc2_1/biases:0
g_net/enc2_2/conv1/weights:0
g_net/enc2_2/conv1/biases:0
g_net/enc2_2/conv2/weights:0
g_net/enc2_2/conv2/biases:0
g_net/enc2_3/conv1/weights:0
g_net/enc2_3/conv1/biases:0
g_net/enc2_3/conv2/weights:0
g_net/enc2_3/conv2/biases:0
g_net/enc2_4/conv1/weights:0
g_net/enc2_4/conv1/biases:0
g_net/enc2_4/conv2/weights:0
g_net/enc2_4/conv2/biases:0
g_net/enc3_1/weights:0
g_net/enc3_1/biases:0
g_net/enc3_2/conv1/weights:0
g_net/enc3_2/conv1/biases:0
g_net/enc3_2/conv2/weights:0
g_net/enc3_2/conv2/biases:0
g_net/enc3_3/conv1/weights:0
g_net/enc3_3/conv1/biases:0
g_net/enc3_3/conv2/weights:0
g_net/enc3_3/conv2/biases:0
g_net/enc3_4/conv1/weights:0
g_net/enc3_4/conv1/biases:0
g_net/enc3_4/conv2/weights:0
g_net/enc3_4/conv2/biases:0
g_net/dec3_3/conv1/weights:0
g_net/dec3_3/conv1/biases:0
g_net/dec3_3/conv2/weights:0
g_net/dec3_3/conv2/biases:0
g_net/dec3_2/conv1/weights:0
g_net/dec3_2/conv1/biases:0
g_net/dec3_2/conv2/weights:0
g_net/dec3_2/conv2/biases:0
g_net/dec3_1/conv1/weights:0
g_net/dec3_1/conv1/biases:0
g_net/dec3_1/conv2/weights:0
g_net/dec3_1/conv2/biases:0
g_net/dec2_4/weights:0
g_net/dec2_4/biases:0
g_net/dec2_3/conv1/weights:0
g_net/dec2_3/conv1/biases:0
g_net/dec2_3/conv2/weights:0
g_net/dec2_3/conv2/biases:0
g_net/dec2_2/conv1/weights:0
g_net/dec2_2/conv1/biases:0
g_net/dec2_2/conv2/weights:0
g_net/dec2_2/conv2/biases:0
g_net/dec2_1/conv1/weights:0
g_net/dec2_1/conv1/biases:0
g_net/dec2_1/conv2/weights:0
g_net/dec2_1/conv2/biases:0
g_net/dec1_4/weights:0
g_net/dec1_4/biases:0
g_net/dec1_3/conv1/weights:0
g_net/dec1_3/conv1/biases:0
g_net/dec1_3/conv2/weights:0
g_net/dec1_3/conv2/biases:0
g_net/dec1_2/conv1/weights:0
g_net/dec1_2/conv1/biases:0
g_net/dec1_2/conv2/weights:0
g_net/dec1_2/conv2/biases:0
g_net/dec1_1/conv1/weights:0
g_net/dec1_1/conv1/biases:0
g_net/dec1_1/conv2/weights:0
g_net/dec1_1/conv2/biases:0
g_net/dec1_0/weights:0
g_net/dec1_0/biases:0
WARNING:tensorflow:From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:186: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

W0425 15:44:09.251477 140246619756416 module_wrapper.py:139] From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:186: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.

WARNING:tensorflow:From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:166: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

W0425 15:44:09.300398 140246619756416 module_wrapper.py:139] From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:166: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/math_grad.py:1375: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W0425 15:44:09.349227 140246619756416 deprecation.py:323] From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/math_grad.py:1375: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:194: The name tf.GPUOptions is deprecated. Please use tf.compat.v1.GPUOptions instead.

W0425 15:44:11.574285 140246619756416 module_wrapper.py:139] From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:194: The name tf.GPUOptions is deprecated. Please use tf.compat.v1.GPUOptions instead.

WARNING:tensorflow:From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:195: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

W0425 15:44:11.582236 140246619756416 module_wrapper.py:139] From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:195: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

WARNING:tensorflow:From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:195: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

W0425 15:44:11.587347 140246619756416 module_wrapper.py:139] From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:195: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:197: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

W0425 15:44:11.618686 140246619756416 module_wrapper.py:139] From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:197: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

WARNING:tensorflow:From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:198: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

W0425 15:44:12.318228 140246619756416 module_wrapper.py:139] From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:198: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:200: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the tf.data module.
W0425 15:44:12.557217 140246619756416 deprecation.py:323] From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:200: start_queue_runners (from tensorflow.python.training.queue_runner_impl) is deprecated and will be removed in a future version.
Instructions for updating:
To construct input pipelines, use the tf.data module.
WARNING:tensorflow:From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:203: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.

W0425 15:44:12.568301 140246619756416 module_wrapper.py:139] From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:203: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.

WARNING:tensorflow:From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:204: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

W0425 15:44:12.662126 140246619756416 module_wrapper.py:139] From /content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py:204: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.NotFoundError'>, ./training_set/GOPR0374_11_02/blur/000640.png; No such file or directory
[[{{node input/ReadFile}}]]
I0425 15:44:13.161757 140244503877376 coordinator.py:224] Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.NotFoundError'>, ./training_set/GOPR0374_11_02/blur/000640.png; No such file or directory
[[{{node input/ReadFile}}]]

OutOfRangeError Traceback (most recent call last)
/tensorflow-1.15.2/python3.7/tensorflow_core/python/client/session.py in _do_call(self, fn, *args)
1364 try:
-> 1365 return fn(*args)
1366 except errors.OpError as e:

11 frames
/tensorflow-1.15.2/python3.7/tensorflow_core/python/client/session.py in _run_fn(feed_dict, fetch_list, target_list, options, run_metadata)
1349 return self._call_tf_sessionrun(options, feed_dict, fetch_list,
-> 1350 target_list, run_metadata)
1351

/tensorflow-1.15.2/python3.7/tensorflow_core/python/client/session.py in _call_tf_sessionrun(self, options, feed_dict, fetch_list, target_list, run_metadata)
1442 fetch_list, target_list,
-> 1443 run_metadata)
1444

OutOfRangeError: FIFOQueue '_0_input/batch/fifo_queue' is closed and has insufficient elements (requested 16, current size 0)
[[{{node input/batch}}]]

During handling of the above exception, another exception occurred:

OutOfRangeError Traceback (most recent call last)
/content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/run_model.py in ()
48
49 if name == 'main':
---> 50 tf.app.run()

/tensorflow-1.15.2/python3.7/tensorflow_core/python/platform/app.py in run(main, argv)
38 main = main or _sys.modules['main'].main
39
---> 40 _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)

/usr/local/lib/python3.7/dist-packages/absl/app.py in run(main, argv, flags_parser)
310 callback()
311 try:
--> 312 _run_main(main, args)
313 except UsageError as error:
314 usage(shorthelp=True, detailed_error=error, exitcode=error.exitcode)

/usr/local/lib/python3.7/dist-packages/absl/app.py in _run_main(main, argv)
256 sys.exit(retval)
257 else:
--> 258 sys.exit(main(argv))
259
260

/content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/run_model.py in main(_)
42 deblur.test(args.height, args.width, args.input_path, args.output_path)
43 elif args.phase == 'train':
---> 44 deblur.train()
45 else:
46 print('phase should be set to either test or train')

/content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py in train(self)
209
210 # update G network
--> 211 _, loss_total_val = sess.run([train_gnet, self.loss_total])
212
213 duration = time.time() - start_time

/tensorflow-1.15.2/python3.7/tensorflow_core/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
954 try:
955 result = self._run(None, fetches, feed_dict, options_ptr,
--> 956 run_metadata_ptr)
957 if run_metadata:
958 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

/tensorflow-1.15.2/python3.7/tensorflow_core/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
1178 if final_fetches or final_targets or (handle and feed_dict_tensor):
1179 results = self._do_run(handle, final_targets, final_fetches,
-> 1180 feed_dict_tensor, options, run_metadata)
1181 else:
1182 results = []

/tensorflow-1.15.2/python3.7/tensorflow_core/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
1357 if handle is None:
1358 return self._do_call(_run_fn, feeds, fetches, targets, options,
-> 1359 run_metadata)
1360 else:
1361 return self._do_call(_prun_fn, handle, feeds, fetches)

/tensorflow-1.15.2/python3.7/tensorflow_core/python/client/session.py in _do_call(self, fn, *args)
1382 '\nsession_config.graph_options.rewrite_options.'
1383 'disable_meta_optimizer = True')
-> 1384 raise type(e)(node_def, op, message)
1385
1386 def _extend_graph(self):

OutOfRangeError: FIFOQueue '_0_input/batch/fifo_queue' is closed and has insufficient elements (requested 16, current size 0)
[[node input/batch (defined at /tensorflow-1.15.2/python3.7/tensorflow_core/python/framework/ops.py:1748) ]]

Original stack trace for 'input/batch':
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/usr/local/lib/python3.7/dist-packages/traitlets/config/application.py", line 846, in launch_instance
app.start()
File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelapp.py", line 499, in start
self.io_loop.start()
File "/usr/local/lib/python3.7/dist-packages/tornado/platform/asyncio.py", line 132, in start
self.asyncio_loop.run_forever()
File "/usr/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/usr/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/usr/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/usr/local/lib/python3.7/dist-packages/tornado/platform/asyncio.py", line 122, in _handle_events
handler_func(fileobj, events)
File "/usr/local/lib/python3.7/dist-packages/tornado/stack_context.py", line 300, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/zmq/eventloop/zmqstream.py", line 452, in _handle_events
self._handle_recv()
File "/usr/local/lib/python3.7/dist-packages/zmq/eventloop/zmqstream.py", line 481, in _handle_recv
self._run_callback(callback, msg)
File "/usr/local/lib/python3.7/dist-packages/zmq/eventloop/zmqstream.py", line 431, in _run_callback
callback(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/tornado/stack_context.py", line 300, in null_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
handler(stream, idents, msg)
File "/usr/local/lib/python3.7/dist-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/usr/local/lib/python3.7/dist-packages/ipykernel/ipkernel.py", line 208, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/usr/local/lib/python3.7/dist-packages/ipykernel/zmqshell.py", line 537, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2718, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2828, in run_ast_nodes
if self.run_code(code, result):
File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2882, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 1, in
get_ipython().magic('run run_model.py --phase=train --model=color --batch=16 --lr=1e-4 --epoch=400')
File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2160, in magic
return self.run_line_magic(magic_name, magic_arg_s)
File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2081, in run_line_magic
result = fn(*args,**kwargs)
File "", line 2, in run
File "/usr/local/lib/python3.7/dist-packages/IPython/core/magic.py", line 188, in
call = lambda f, *a, **k: f(*a, **k)
File "/usr/local/lib/python3.7/dist-packages/IPython/core/magics/execution.py", line 742, in run
run()
File "/usr/local/lib/python3.7/dist-packages/IPython/core/magics/execution.py", line 728, in run
exit_ignore=exit_ignore)
File "/usr/local/lib/python3.7/dist-packages/IPython/core/pylabtools.py", line 174, in mpl_execfile
safe_execfile(fname,*where,**kw)
File "/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py", line 2483, in safe_execfile
self.compile if kw['shell_futures'] else None)
File "/usr/local/lib/python3.7/dist-packages/IPython/utils/py3compat.py", line 186, in execfile
exec(compiler(f.read(), fname, 'exec'), glob, loc)
File "/content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/run_model.py", line 50, in
tf.app.run()
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 312, in run
_run_main(main, args)
File "/usr/local/lib/python3.7/dist-packages/absl/app.py", line 258, in _run_main
sys.exit(main(argv))
File "/content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/run_model.py", line 44, in main
deblur.train()
File "/content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py", line 183, in train
self.build_model()
File "/content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py", line 134, in build_model
img_in, img_gt = self.input_producer(self.batch_size)
File "/content/drive/MyDrive/SRN-Deblur-master/SRN-Deblur-master/models/model.py", line 61, in input_producer
batch_in, batch_gt = tf.train.batch([image_in, image_gt], batch_size=batch_size, num_threads=8, capacity=20)
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/util/deprecation.py", line 324, in new_func
return func(*args, **kwargs)
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/training/input.py", line 1020, in batch
name=name)
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/training/input.py", line 789, in _batch
dequeued = queue.dequeue_many(batch_size, name=name)
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/data_flow_ops.py", line 489, in dequeue_many
self._queue_ref, n=n, component_types=self._dtypes, name=name)
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/gen_data_flow_ops.py", line 3862, in queue_dequeue_many_v2
timeout_ms=timeout_ms, name=name)
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/tensorflow-1.15.2/python3.7/tensorflow_core/python/framework/ops.py", line 1748, in init
self._traceback = tf_stack.extract_stack()

@jiangsutx
Copy link
Owner

I think this is because the recent TensorFlow may be different and in-compatible with previous old ones.
Maybe you can try Tensorflow 1.4 as listed in README.

If you want to use new Tensorflow, I think code modifications are needed according to error message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants