tensorflow 1.x and 2.x produce different results during inferencing based on the same parameters

[jamesYu365]

Issue type

Bug

Have you reproduced the bug with TensorFlow Nightly?

No

Source

source

TensorFlow version

tf 1.15 and tf 2.13

Custom code

Yes

OS platform and distribution

Windows

Mobile device

No response

Python version

3.8

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

None

GPU model and memory

None

Current behavior?

I trained a LSTM in Tensorflow1.15, then load the saved model parameters on Tensorflow2.13, the prediction is significantly different from Tensorflow1.15.

Standalone code to reproduce the issue

https://drive.google.com/file/d/1spHn4waDE3vNQzslA7AD06E0XQ1i-qL_/view?usp=sharing

Relevant log output

No response

[sushreebarsa]

@jamesYu365 Could you have a look at the TensorFlow Compatibility Checker to identify deprecated or removed operations from your TensorFlow 1.x model. The tool suggests alternative operations for migration. Thank you!

[jamesYu365]

Hello, thank you for your prompt response. Following your instructions, I added compat.v1 when importing the TensorFlow-related modules:

import tensorflow.compat.v1 as tf
from tensorflow.compat.v1.keras.models import Sequential
from tensorflow.compat.v1.keras.layers import LSTM, Dense

The result now is the same between two versions.
However, I'm still curious why this happened. Since the LSTM model involves only basic arithmetic operations, I wonder why TensorFlow 1 and TensorFlow 2 give different results.

[sushreebarsa]

@jamesYu365 Using compat.v1 likely mitigated the issue by forcing TF2 to use a more TF1-like behavior, potentially including non-deterministic behavior. This might have made the results from both versions more comparable.
Thank you!

[jamesYu365]

Thank you for your prompt response again. I'm still puzzled by the non-deterministic behavior you mentioned, considering that all the parameters and inputs are fixed. Where did this non-deterministic behavior come from? Could you please elaborate more on it? Perhaps you could provide some links related to this question, as I'm new to using TensorFlow. Thanks.