Errow while save and load the model

[sachinkmohan]

I had uncommented the checkpoint and tried to save the model. It generated saved_model.pb file. Below are the warning messages when the model was saved.

callbacks = [
              tf.keras.callbacks.ModelCheckpoint("DeepLabV3plus1.ckpt", verbose=1, save_weights_only=False, save_best_only=False),
             tf.keras.callbacks.ReduceLROnPlateau(monitor="iou_score", factor=0.2, patience=6, verbose=1, mode="max"),
             tf.keras.callbacks.EarlyStopping(monitor="iou_score", patience=16, mode="max", verbose=1, restore_best_weights=True)
]

366/366 [==============================] - ETA: 0s - loss: 0.3064 - iou_score: 0.6460
Epoch 00001: saving model to DeepLabV3plus1.ckpt
WARNING:tensorflow:Skipping full serialization of Keras layer <keras.layers.merge.Concatenate object at 0x7fe5cc066350>, because it is not built.
WARNING:absl:Found untraced functions such as activation_layer_call_fn, activation_layer_call_and_return_conditional_losses, conv2d_layer_call_fn, conv2d_layer_call_and_return_conditional_losses, activation_1_layer_call_fn while saving (showing 5 of 135). These functions will not be directly callable after loading.
INFO:tensorflow:Assets written to: DeepLabV3plus1.ckpt/assets
INFO:tensorflow:Assets written to: DeepLabV3plus1.ckpt/assets
/home/mohan/.local/lib/python3.7/site-packages/keras/engine/functional.py:1410: CustomMaskWarning: Custom mask layers require a config and must override get_config. When loading, the custom mask layer must be passed to the custom_objects argument.
  layer_config = serialize_layer_fn(layer)
/home/mohan/.local/lib/python3.7/site-packages/keras/saving/saved_model/layer_serialization.py:112: CustomMaskWarning: Custom mask layers require a config and must override get_config. When loading, the custom mask layer must be passed to the custom_objects argument.
  return generic_utils.serialize_keras_object(obj)
366/366 [==============================] - 96s 258ms/step - loss: 0.3064 - iou_score: 0.6460 - lr: 0.2000

Now when loading the model, I am getting the following error.

model = tf.keras.models.load_model('/home/mohan/git/TensorFlow-Advanced-Segmentation-Models/examples/DeepLabV3plus1.ckpt/')

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-ae5de3c338c0> in <module>
----> 1 model = tf.keras.models.load_model('/home/mohan/git/TensorFlow-Advanced-Segmentation-Models/examples/DeepLabV3plus1.ckpt/')

~/.local/lib/python3.7/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb

~/.local/lib/python3.7/site-packages/keras/saving/saved_model/load.py in revive_custom_object(identifier, metadata)
   1007   else:
   1008     raise ValueError(
-> 1009         f'Unable to restore custom object of type {identifier}. '
   1010         f'Please make sure that any custom layers are included in the '
   1011         f'`custom_objects` arg when calling `load_model()` and make sure that '

ValueError: Unable to restore custom object of type _tf_keras_metric. Please make sure that any custom layers are included in the `custom_objects` arg when calling `load_model()` and make sure that all layers implement `get_config` and `from_config`.

I am not sure how to load the custom objects though. Please help.

[JanMarcelKezmann]

Hi @sachinkmohan ,

so the problem you describe requires a little larger fix, which I due to a lack of time will not be able to fix in the near term.
What I can advise you is that you simply try to set save_weights_only=True.
This will allow you to reload the weights without any problems (at least it did when I have tested it).

[sachinkmohan]

Thanks for the response! @JanMarcelKezmann Appreciate it!
Will check this another time.

[zs735]

ave_weights_only=True

Hi, I am having similar issue but with saving of the weights. Just had to redo anaconda setup and reinstall tensorflow.

model.save(os.path.join(new_folder, 'model_weights.h5'))

I am now getting this:

C:\Users\Master\anaconda3\envs\gputensorflow\lib\site-packages\tensorflow\python\keras\utils\generic_utils.py:494: CustomMaskWarning: Custom mask layers require a config and must override get_config. When loading, the custom mask layer must be passed to the custom_objects argument.
warnings.warn('Custom mask layers require a config and must override '

Python 3.10.7 (tags/v3.10.7:6cc6b13, Sep 5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import tensorflow as tf
print(tf.version.VERSION)
2.10.0
exit

Appreciate your insight what must i do next?