How can I convert this to tf.keras h5 model?

[1n1n1t3]

I have little experience with pure Tensorflow and NLP and I'm having a really hard time converting this to a keras or tf.keras h5 model so I can then convert it to Tensorflow lite or Tensorflow js. Someone managed to do that ?

[JayYip]

Hi, you can try following steps:

1. Create model
2. Load weights
3. Save as h5.

[1n1n1t3]

Hi @JayYip ,

I used the DocProductPresentationV6-0.2.0.ipynb as a starting point and used this code to do what you mentioned:

`
from docproduct.predictor import RetreiveQADoc

pretrained_path = 'BioBertFolder/biobert_v1.0_pubmed_pmc/'
ffn_weight_file = None
bert_ffn_weight_file = 'newFolder/models/bertffn_crossentropy/bertffn'
embedding_file = 'Float16EmbeddingsExpanded5-27-19.pkl'
graph = tf.Graph()
with graph.as_default():

doc = RetreiveQADoc(pretrained_path=pretrained_path,
ffn_weight_file=None,
bert_ffn_weight_file=bert_ffn_weight_file,
embedding_file=embedding_file)
doc.qa_embed.model.save("model.h5")`

I get this error though:

NotImplementedError: The savemethod requires the model to be a Functional model or a Sequential model. It does not work for subclassed models, because such models are defined via the body of a Python method, which isn't safely serializable. Consider usingsave_weights, in order to save the weights of the model.

I manage to create a file with "save_weights" but it's not usable later with the tfliteconverter. I need to save the full model.

Any idea how I can overcome this error ?

[1n1n1t3]

So, I found out when the model is subclassed I can use:

tf.keras.experimental.export_saved_model(doc.qa_embed.model,saved_model_path="model.h5",serving_only=True)
But I get an error again:

ValueError: Model <docproduct.models.MedicalQAModelwithBert object at 0x7fd0a7533ac8> cannot be saved because the input shapes have not been set. Usually, input shapes are automatically determined from calling .fit() or .predict(). To manually set the shapes, call model._set_inputs(inputs).

Not sure how to proceed from here... Calling the predict method of the doc object does nothing to help. I'm lost at finding out how to set the inputs here. Please help.

[JayYip]

The reason is that in order to save the computation graph, you need to build the graph first. And without the input shape, TF cannot build the graph. Please try calling model._set_inputs(inputs) before you export the model as the error message said.

If you can successfully export the model and deploy using tf.js or tf.lite, please consider making a PR because it seems lots of people encountered the same problem while using our model.

[Santosh-Gupta]

Interesting. I am assuming that the full computation graph is already built for training and inference. Couldn't you save the computation graph then? Why do you need _set_inputs

[Chrome3912]

Try the api : https://pocket-derm.firebaseapp.com/

[Santosh-Gupta]

Try the api : https://pocket-derm.firebaseapp.com/

Pretty neat but it looks like it costs money. How did you make this?

[Chrome3912]

@Santosh-Gupta a simple post request w/ json.
Python example:

import requests
r = requests.post(url='http://li1027-34.members.linode.com/predict/{APIKEY}', json = {"question":"my skin is itchy"} )

This should work once you get the API key

[JayYip]

@pumpkinband Try this?

from docproduct.predictor import RetreiveQADoc

pretrained_path = 'BioBertFolder/biobert_v1.0_pubmed_pmc/'
ffn_weight_file = None
bert_ffn_weight_file = 'newFolder/models/bertffn_crossentropy/bertffn'
embedding_file = 'Float16EmbeddingsExpanded5-27-19.pkl'
graph = tf.Graph()
with graph.as_default():

  doc = RetreiveQADoc(pretrained_path=pretrained_path,
    ffn_weight_file=None,
    bert_ffn_weight_file=bert_ffn_weight_file,
    embedding_file=embedding_file)
    doc.qa_embed.model.save("model.h5")

[JayYip]

Ahh, sorry, some bug in the that code. Try this:

from docproduct.predictor import RetreiveQADoc

pretrained_path = 'BioBertFolder/biobert_v1.0_pubmed_pmc/'
ffn_weight_file = None
bert_ffn_weight_file = 'newFolder/models/bertffn_crossentropy/bertffn'
embedding_file = 'Float16EmbeddingsExpanded5-27-19.pkl'
graph = tf.Graph()
with graph.as_default():

  doc = RetreiveQADoc(pretrained_path=pretrained_path,
    ffn_weight_file=None,
    bert_ffn_weight_file=bert_ffn_weight_file,
    embedding_file=embedding_file)
  doc.qa_embed.model.save("model.h5")

PS, this is a very basic python syntax error. You may need to get to know more about python.

[EphChem]

@JayYip I tried the above which was similar to what I've tried before and still get:

NotImplementedError: The save method requires the model to be a Functional model or a Sequential model. It does not work for subclassed models, because such models are defined via the body of a Python method, which isn't safely serializable. Consider using save_weights, in order to save the weights of the model.

I also tried: tf.keras.experimental.export_saved_model(doc.qa_embed.model,saved_model_path="model.h5", serving_only=True) instead of doc.qa_embed.model.save("model.h5") and I get:

AttributeError: 'str' object has no attribute 'shape'

[Santosh-Gupta]

I am unable test these yet but in the meanwhile try instantiating the model object and then trying to save it

thisModel = doc.qa_embed.model
thisModel.save("model.h5")

let me know what the result is. If that doesn't work, try to instantiate the Keras model object directly.

from docproduct.models import MedicalQAModelwithBert

medical_qa_model = MedicalQAModelwithBert(
        config_file=os.path.join(
            pretrained_path, 'bert_config.json'),
        checkpoint_file=os.path.join(pretrained_path, 'biobert_model.ckpt'))

medical_qa_model.save("model.h5")

[dr-rathod]

After trying

thisModel = doc.qa_embed.model thisModel.save("model.h5")

i am getting the previous error
NotImplementedError: The savemethod requires the model to be a Functional model or a Sequential model. It does not work for subclassed models, because such models are defined via the body of a Python method, which isn't safely serializable. Consider usingsave_weights, in order to save the weights of the model. @Santosh-Gupta

[jeet-ambaliya]

after trying this out

doc.qa_embed.model._set_inputs(inputs) tf.keras.experimental.export_saved_model(doc.qa_embed.model,saved_model_path="model.h5",serving_only=True)

it is asking for inputs

NameError: name 'inputs' is not defined

can you please help me with this @Santosh-Gupta

[Santosh-Gupta]

It says you didn't define inputs.

What did you set inputs as? It just may be a python bug.

[jeet-ambaliya]

here is link to my colab can you please check what inputs i have to put ?

https://colab.research.google.com/drive/1PadA5HDsnydt6juZP7f8lVo_sLYCwOGn

[Santosh-Gupta]

It looks like you didn't define any inputs, so that's what was giving you the NameError: name 'inputs' is not defined error. It's an error when you reference a variable that's not defined yet.

[behinoo]

@JayYip @Santosh-Gupta
Any one came up with a solution to this?

I am struggling with the same issue, and hasn't been able to solve the issue. I have tried all the suggestion in this post but couldn't resolve it. I really appreciate any help that I can get.
here is my colab:
https://colab.research.google.com/drive/1ZON8lvha8sI9ZCJEF0Ad8au2NNc9sUkU

`from docproduct.models import MedicalQAModelwithBert

medical_qa_model = MedicalQAModelwithBert(
config_file=os.path.join(
pretrained_path, 'bert_config.json'),
checkpoint_file=os.path.join(pretrained_path, 'biobert_model.ckpt'))

medical_qa_model.save("model.h5")`

The error is
NotImplementedError: The save method requires the model to be a Functional model or a Sequential model. It does not work for subclassed models, because such models are defined via the body of a Python method, which isn't safely serializable. Consider using save_weights, in order to save the weights of the model.`

Thanks

[Santosh-Gupta]

For

from docproduct.models import MedicalQAModelwithBert

medical_qa_model = MedicalQAModelwithBert(
        config_file=os.path.join(
            pretrained_path, 'bert_config.json'),
        checkpoint_file=os.path.join(pretrained_path, 'biobert_model.ckpt'))

medical_qa_model.save("model.h5")

The error message mentions

"Consider using save_weights, in order to save the weights of the model"

So I would try that.

For

tf.keras.experimental.export_saved_model(doc.qa_embed.model,saved_model_path="model.h5",serving_only=True)

The error message mentions

"ValueError: Model <docproduct.models.MedicalQAModelwithBert object at 0x7f765a1e6da0> cannot be saved because the input shapes have not been set. Usually, input shapes are automatically determined from calling .fit() or .predict(). To manually set the shapes, call model._set_inputs(inputs)."

So I would try that as well.

Let me know if either of those work.

[behinoo]

@Santosh-Gupta Thank you for prompt reply. I did try using "save_weights" but it's not usable later with the tfliteconverter becuase it is not the full model.

Regarding the error for
tf.keras.experimental.export_saved_model(doc.qa_embed.model,saved_model_path="model.h5",serving_only=True),
what is the "inputs" for this model? I am not sure how I should define it.

Thank you for your time

[Santosh-Gupta]

The error message also says

Usually, input shapes are automatically determined from calling .fit() or .predict().

So you could also trying calling the predict function first, but if that doesn't work any random text should work. You may also have to supply the rest of the args.

Either way, check out the last cell which has does prediction.

[behinoo]

@Santosh-Gupta I tried different variation of what you suggested but no luck, here are the error for each:
I used random text and also similar input as the input for the fit function in the last cell but i get the same error.

medical_qa_model._set_inputs('cncnnc') tf.keras.experimental.export_saved_model(medical_qa_model,saved_model_path="model.h5",serving_only=True)
error:
` 2727 if self.inputs:
-> 2728 raise ValueError('Model inputs are already set.')
2729
2730 if self.class.name == 'Sequential' and not self.built:

ValueError: Model inputs are already set.`

if I use medical_qa_model.predict(question_text )
I will get
ValueError: ('Error when checking model input: expected no data, but got:', 'I’ve had chronic migraines for four years')

if I use medical_qa_model.predict( )
then the error will be
TypeError: predict() missing 1 required positional argument: 'x'

[Santosh-Gupta]

For the first method you are getting an error on the first line, you're trying to set the input when the input has been already set. Remove that line and try again.

medical_qa_model doesn't have it's predictor function directly defined, so I would just stick with RetreiveQADoc

https://github.com/re-search/DocProduct/blob/master/docproduct/models.py

[shravankumar9892]

Is the model bertffn_crossentropy/bertffn already been exported to SavedModel ?

[Santosh-Gupta]

@shravankumar9892 I do not believe it has been exported. I am just skimming through that link but it looks like that is for Estimator? We are using Keras.

[behinoo]

@Santosh-Gupta It seems the issue is that when model subclass was defined, the save() wasn't wasn't implemented
https://github.com/re-search/DocProduct/blob/master/docproduct/models.py#L62

[Santosh-Gupta]

@Santosh-Gupta It seems the issue is that when model subclass was defined, the save() wasn't wasn't implemented
https://github.com/re-search/DocProduct/blob/master/docproduct/models.py#L62

Since it inherits from tf.keras.Model, I imagine that tf.keras.Model's save function would still be in tact. Any thoughts @JayYip ?

[JayYip]

@behinoo I think you're right. Since model subclassing might contains dynamic graph which is defined in runtime by python. That makes exporting to SavedModel difficult. Currently, we don't have a solution.

[Santosh-Gupta]

@JayYip I am wondering where could the architecture could possibly be dynamic. I believe Tensorflow is completely static? Perhaps with the variable length input, which may explain why we keep getting the input not defined errors.

[getabhishekified]

I am unable test these yet but in the meanwhile try instantiating the model object and then trying to save it

thisModel = doc.qa_embed.model
thisModel.save("model.h5")

let me know what the result is. If that doesn't work, try to instantiate the Keras model object directly.

from docproduct.models import MedicalQAModelwithBert

medical_qa_model = MedicalQAModelwithBert(
        config_file=os.path.join(
            pretrained_path, 'bert_config.json'),
        checkpoint_file=os.path.join(pretrained_path, 'biobert_model.ckpt'))

medical_qa_model.save("model.h5")

I am getting this error

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\framework\tensor_util.py in make_tensor_proto(values, dtype, shape, verify_shape, allow_broadcast)
    557     try:
--> 558       str_values = [compat.as_bytes(x) for x in proto_values]
    559     except TypeError:

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\framework\tensor_util.py in <listcomp>(.0)
    557     try:
--> 558       str_values = [compat.as_bytes(x) for x in proto_values]
    559     except TypeError:

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\util\compat.py in as_bytes(bytes_or_text, encoding)
     64     raise TypeError('Expected binary or unicode string, got %r' %
---> 65                     (bytes_or_text,))
     66 

TypeError: Expected binary or unicode string, got Dimension(768)

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\network.py in build(self, input_shape)
    717         try:
--> 718           self.call(x, **kwargs)
    719         except (errors.InvalidArgumentError, TypeError):

F:\Documents\Biobert\AbhiBert\docproduct\bert.py in call(self, inputs)
    168                 dropout_rate=self.dropout_rate,
--> 169                 trainable=self.trainable)
    170             last_layer = _wrap_layer(

F:\Documents\Biobert\AbhiBert\docproduct\bert.py in _wrap_layer(name, input_layer, build_func, norm_layer, dropout_rate, trainable)
    140             """
--> 141             build_output = build_func(input_layer)
    142             if dropout_rate > 0.0:

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in __call__(self, inputs, *args, **kwargs)
    590           # overridden).
--> 591           self._maybe_build(inputs)
    592 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in _maybe_build(self, inputs)
   1880       with tf_utils.maybe_init_scope(self):
-> 1881         self.build(input_shapes)
   1882     # We must set self.built since user defined build functions are not

F:\Documents\Biobert\AbhiBert\keras_bert\keras_multi_head\multi_head_attention.py in build(self, input_shape)
     93             constraint=self.kernel_constraint,
---> 94             name='%s_Wq' % self.name,
     95         )

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in add_weight(self, name, shape, dtype, initializer, regularizer, trainable, constraint, partitioner, use_resource, synchronization, aggregation, **kwargs)
    383         synchronization=synchronization,
--> 384         aggregation=aggregation)
    385     backend.track_variable(variable)

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\training\tracking\base.py in _add_variable_with_custom_getter(self, name, shape, dtype, initializer, getter, overwrite, **kwargs_for_getter)
    662         initializer=initializer,
--> 663         **kwargs_for_getter)
    664 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\base_layer_utils.py in make_variable(name, shape, dtype, initializer, trainable, caching_device, validate_shape, constraint, use_resource, collections, synchronization, aggregation, partitioner)
    154       aggregation=aggregation,
--> 155       shape=variable_shape if variable_shape.rank else None)
    156 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\variables.py in __call__(cls, *args, **kwargs)
    258     if cls is VariableV1:
--> 259       return cls._variable_v1_call(*args, **kwargs)
    260     elif cls is Variable:

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\variables.py in _variable_v1_call(cls, initial_value, trainable, collections, validate_shape, caching_device, name, variable_def, dtype, expected_shape, import_scope, constraint, use_resource, synchronization, aggregation, shape)
    219         aggregation=aggregation,
--> 220         shape=shape)
    221 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\variables.py in <lambda>(**kwargs)
    197     """Call on Variable class. Useful to force the signature."""
--> 198     previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
    199     for _, getter in ops.get_default_graph()._variable_creator_stack:  # pylint: disable=protected-access

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\variable_scope.py in default_variable_creator(next_creator, **kwargs)
   2494         aggregation=aggregation,
-> 2495         shape=shape)
   2496   else:

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\variables.py in __call__(cls, *args, **kwargs)
    262     else:
--> 263       return super(VariableMetaclass, cls).__call__(*args, **kwargs)
    264 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\resource_variable_ops.py in __init__(self, initial_value, trainable, collections, validate_shape, caching_device, name, dtype, variable_def, import_scope, constraint, distribute_strategy, synchronization, aggregation, shape)
    459           aggregation=aggregation,
--> 460           shape=shape)
    461 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\resource_variable_ops.py in _init_from_args(self, initial_value, trainable, collections, caching_device, name, dtype, constraint, synchronization, aggregation, shape)
    603             initial_value = ops.convert_to_tensor(
--> 604                 initial_value() if init_from_fn else initial_value,
    605                 name="initial_value", dtype=dtype)

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\base_layer_utils.py in <lambda>()
    134         initializer = initializer()
--> 135       init_val = lambda: initializer(shape, dtype=dtype)
    136       variable_dtype = dtype.base_dtype

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\init_ops.py in __call__(self, shape, dtype, partition_info)
    525       return random_ops.truncated_normal(
--> 526           shape, 0.0, stddev, dtype, seed=self.seed)
    527     elif self.distribution == "untruncated_normal":

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\random_ops.py in truncated_normal(shape, mean, stddev, dtype, seed, name)
    172   with ops.name_scope(name, "truncated_normal", [shape, mean, stddev]) as name:
--> 173     shape_tensor = _ShapeTensor(shape)
    174     mean_tensor = ops.convert_to_tensor(mean, dtype=dtype, name="mean")

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\random_ops.py in _ShapeTensor(shape)
     43     dtype = None
---> 44   return ops.convert_to_tensor(shape, dtype=dtype, name="shape")
     45 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\framework\ops.py in convert_to_tensor(value, dtype, name, preferred_dtype, dtype_hint)
   1086       "dtype_hint", dtype_hint, "preferred_dtype", preferred_dtype)
-> 1087   return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
   1088 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\framework\ops.py in convert_to_tensor_v2(value, dtype, dtype_hint, name)
   1144       preferred_dtype=dtype_hint,
-> 1145       as_ref=False)
   1146 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\framework\ops.py in internal_convert_to_tensor(value, dtype, name, as_ref, preferred_dtype, ctx, accept_symbolic_tensors, accept_composite_tensors)
   1223     if ret is None:
-> 1224       ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
   1225 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\framework\constant_op.py in _constant_tensor_conversion_function(v, dtype, name, as_ref)
    304   _ = as_ref
--> 305   return constant(v, dtype=dtype, name=name)
    306 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\framework\constant_op.py in constant(value, dtype, shape, name)
    245   return _constant_impl(value, dtype, shape, name, verify_shape=False,
--> 246                         allow_broadcast=True)
    247 

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\framework\constant_op.py in _constant_impl(value, dtype, shape, name, verify_shape, allow_broadcast)
    283           value, dtype=dtype, shape=shape, verify_shape=verify_shape,
--> 284           allow_broadcast=allow_broadcast))
    285   dtype_value = attr_value_pb2.AttrValue(type=tensor_value.tensor.dtype)

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\framework\tensor_util.py in make_tensor_proto(values, dtype, shape, verify_shape, allow_broadcast)
    561                       "Contents: %s. Consider casting elements to a "
--> 562                       "supported type." % (type(values), values))
    563     tensor_proto.string_val.extend(str_values)

TypeError: Failed to convert object of type <class 'tuple'> to Tensor. Contents: (Dimension(768), Dimension(768)). Consider casting elements to a supported type.

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-5-78b1df8a6b61> in <module>
      6         config_file=os.path.join(
      7             pretrained_path, 'bert_config.json'),
----> 8         checkpoint_file=os.path.join(pretrained_path, 'biobert_model.ckpt'))
      9 
     10 medical_qa_model.save("model.h5")

F:\Documents\Biobert\AbhiBert\docproduct\models.py in __init__(self, hidden_size, dropout, residual, config_file, checkpoint_file, bert_trainable, layer_ind, name)
     77             training=False,
     78             trainable=bert_trainable,
---> 79             build=build)
     80         if checkpoint_file is not None:
     81             load_model_weights_from_checkpoint(

F:\Documents\Biobert\AbhiBert\docproduct\bert.py in build_model_from_config(config_file, training, trainable, seq_len, build)
    212     )
    213     if build:
--> 214         model.build(input_shape=[(None, None), (None, None), (None, None)])
    215     return model, config

c:\users\gautam-pc\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\network.py in build(self, input_shape)
    718           self.call(x, **kwargs)
    719         except (errors.InvalidArgumentError, TypeError):
--> 720           raise ValueError('You cannot build your model by calling `build` '
    721                            'if your layers do not support float type inputs. '
    722                            'Instead, in order to instantiate and build your '

ValueError: You cannot build your model by calling `build` if your layers do not support float type inputs. Instead, in order to instantiate and build your model, `call` your model on real tensor data (of the correct dtype).

[Santosh-Gupta]

It looks like the old error, where we have to initialize using inputs. I am wondering if anyone has been able to solve this yet. Perhaps we should ask in the keras github or stackover flow on how to resolve this issue.

[getabhishekified]

I am trying to run the exact code of colab notebook on my local machine(windows 64 bit) but its giving the above error, is this because of tensorflow version?
I was trying to build a django web server wrapper around this app and call it from mobile app by using HTTP calls.

[Santosh-Gupta]

What version of TF do you have? Ours is build on tf 2.0

[getabhishekified]

yes, that was the issue. I was using stable version of tensorflow.

[Fireballlion]

Has anyone managed to obtain the .h5 model or a .tflite model?

[Santosh-Gupta]

Not that I know of, but it may be easier now since TF 2.0 is officially in beta.