Predict value error Between keras model and pb model

[youyuge34]

I tried both on tf-gpu1.4+keras2.1.3 and on tf-gpu1.12+keras2.2.4 and the problem always happens.

The problem is: After I converted the keras.application.ResNet50() model into freeze graph model in .pb format, I feed in the same picture into the converted .pb model but the output value changes just a little.

Firstly I save the keras.application.ResNet50() model with .h5 format using keras API:

from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input, ResNet50
import keras.backend as K
K.set_learning_phase(0)
img = image.load_img('images/34rews.jpg', target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x_input = preprocess_input(x)

def save_h5(h5_path):
    model = ResNet50(weights='imagenet', include_top=False, pooling='avg')
    preds = model.predict(x_input)
    print('keras model.predict:', np.array(preds).squeeze()[:10])
    model.save(os.path.join(h5_path,'model.h5'), include_optimizer=False)

And the first 10 elements of the predict vector is:

'keras model.predict:' [**0.99536467** 0.31807986 2.0998483  0.9077819  0.10606026 0.93215793
 0.04187933 0.10000334 1.1727284  1.0535308 ]

Then I used this repo to convert the .h5 file into .pb file:

python keras_to_tensorflow.py --input_model="h5/model.h5" --output_model="pb/model_1103. pb"

Finally I load the pb model and predict the same image again:

def test_amir_converted_pb(pb_path):
    from tensorflow.python.platform import gfile
    with tf.Session() as sess:
        with gfile.FastGFile(pb_path, 'rb') as f:
            graph_def = tf.GraphDef()
            graph_def.ParseFromString(f.read())

        input, result = tf.import_graph_def(graph_def, return_elements=['input_1:0', "global_average_pooling2d_1/Mean:0"], name='')
        preds = sess.run(result, feed_dict={input: x_input})
        print('test_amir_converted_pb:', np.array(preds).squeeze()[:10])

But the predict vector value changes a little, which is really strange:

test_amir_converted_pb: [0.99536514 0.3180797  2.0998483  0.90778273 0.10606024 0.9321572
 0.04187941 0.10000295 1.1727289  1.0535315 ]

I also rewrite my own freeze graph tool to convert .h5 into .pb, but the problem is just the same.
PLZ help!

[youyuge34]

I removed the K.set_learning_phase(0) and the problem is solved. Maybe it could be better to let Keras to handle with the K.learning_phase() by iteself.
For my experiment, use the value of Keras.model.predict() as the correct outcome:

with K.set_learning_phase(0) and without convert_variables_to_constants:  value is the same.
without K.set_learning_phase(0) and without convert_variables_to_constants:  value is the same.
without K.set_learning_phase(0) and with convert_variables_to_constants:  value is the same.
with K.set_learning_phase(0) and with convert_variables_to_constants:  value is changed!

[amir-abdi]

Thank you @youyuge34
Feel free to send an MR if you are happy with your solution.

[youyuge34]

@amir-abdi I researched more, and I found the problem was casused by using function convert_variables_to_constants() on BN layer in Keras, which could be a tough bug inside the source codes. Due to the BN layer was totally code refactoring after Keras 2.1.3, the solution will be really tough.

To be compatible with most cases, maybe keep the K.set_learning_phase(0) is a wise choice, which will enforce creating a model with only eval nodes in current graph.

If someone has the same problem as me, PLZ try to remove K.set_learning_phase(0) and see what will happen.

[lizq-git]

@amir-abdi I researched more, and I found the problem was casused by using function convert_variables_to_constants() on BN layer in Keras, which could be a tough bug inside the source codes. Due to the BN layer was totally code refactoring after Keras 2.1.3, the solution will be really tough.

To be compatible with most cases, maybe keep the K.set_learning_phase(0) is a wise choice, which will enforce creating a model with only eval nodes in current graph.

If someone has the same problem as me, PLZ try to remove K.set_learning_phase(0) and see what will happen.

Hi, I am experiencing the same issue, the result I have for hdf5/h5 is really different from the result I have for the pb file.

On the hdf5 end (via anaconda python 3.7, tf 1.13.1), I have resized the test images, then normalized it (float 255)
I have also resized the image for the pb testing (via anaconda python 3.7, tf 1.13.1) as well.

I have also tested the pb at the windows application (via visual studio), the result is funny as well.

I have tried your solution, but it does not help to correct the model behavior.

[youyuge34]

@lizq-git make sure the input arrays of model are totally the same~