Doc(Transfer learning and fine-tuning) is quite different from real executive result.

[lida2003]

Issue type

Support

Have you reproduced the bug with TensorFlow Nightly?

No

Source

binary

TensorFlow version

2.15.0+nv24.03 GPU version, check this link: https://forums.developer.nvidia.com/t/multiple-executive-warnings-after-switching-tensorflow-from-2-16-1-cpu-to-v60dp-tensorflow-2-15-0-nv24-03-gpu-version/291208

Custom code

No

OS platform and distribution

Jetson Orin Nano ubuntu 22.04 Jammy

Mobile device

No response

Python version

3.10.12

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

CUDA12.2.140/cuDNN8.9.4.25

GPU model and memory

sm90 8GB

Current behavior?

The executive result (trend of curve and abosolute value) is quite different from document.

• My result: transfer_learning.zip

• Document (https://github.com/tensorflow/docs/blob/master/site/en/tutorials/images/transfer_learning.ipynb) says:

Standalone code to reproduce the issue

100% reproducable

Relevant log output

Also tried Colab, which is consistent with documentation:

[tilakrayal]

@lida2003,
Could you please confirm whether the difference in Accuracy & Loss has happened in GPU/CPU with both tensorflow v2.15, v2.16? Also I will also try to debug more on this issue and provide the resolution. Thank you!

[lida2003]

@lida2003,
Could you please confirm whether the difference in Accuracy & Loss has happened in GPU/CPU with both tensorflow v2.15, v2.16?

• NVIDIA 2.15.0+nv24.03 GPU version faild 100%
• Colab is consistent with tensorflow document (But I don't know the version)

PS: Jetson Orin Nano 8GB, CPU&GPU shared memory

v2.16

Well, on the very begining, I have installed (pip binary installation) 2.16 CPU version on Jetson Orin Nano. Runing Keras-Fine-Tuning-Pre-Trained-Models without any resource warning. It might be the way CPU using swap area.

The result is also different from the document, See link below:

• Multiple executive warnings after switching tensorflow from 2.16.1 CPU to v60dp tensorflow==2.15.0+nv24.03 GPU version

When I switched to NVIDIA 2.15.0+nv24.03 GPU version: Tensorflow v2.16.1 GPU version local build on Jetson Orin Nano failed

• The resource issue, we have confirmed with NVIDIA.

Also I will also try to debug more on this issue and provide the resolution. Thank you!

There are also a copule of other things might be a clue for you. Here is a link on NVIDIA forum:

• Tensorflow v2.16.1 GPU version local build on Jetson Orin Nano failed
• Multiple executive warnings after switching tensorflow from 2.16.1 CPU to v60dp tensorflow==2.15.0+nv24.03 GPU version

Please take a look at those warnings and memory issue. I think we need a sanity check before software is packed for release (put on repo).

EDIT: Keep sync with NVIDIA feedback.

• NVIDIA claimed their binary compiled following tensorflow guide.
• Those warnings seems OK with NVIDIA 2.15 binary version.
• Inconsistency of NVIDIA 2.15.0+nv24.03 v.s. Colab v.s. Tensorflow Documentation