which version of keras is used by latest kws_streaming

[andrewmabc]

I'm trying to run kws_streaming's 00_check_data.ipynb/01_train.ipynb/02_inference.ipynb.

docker image tensorflow/tensorflow:nightly-gpu-jupyter is used to run the demo. And the following packages are installed via pip:
tf-model-optimization-nightly
tf-keras-nightly
tfa-nightly
scipy
pydot
graphviz

The following package is install via apt-get:
apt-get install graphviz

tf._keras_internal.utils.control_flow_util.smart_cond is replaced by tensorflow.python.keras.utils.control_flow_util.smart_cond.
tf._keras_internal.models._clone_layers_and_model_config is replaced by tensorflow.python.keras.models._clone_layers_and_model_config.
tf._keras_internal.models._clone_layer is replaced by tensorflow.python.keras.models._clone_layer.
tf._keras_internal.engine.functional.reconstruct_from_config is replaced by tensorflow.python.keras.engine.functional.reconstruct_from_config.

If TF_USE_LEGACY_KERAS=1, 01_train.ipynb can be run successfully. But the following calling is always running infinitely, i.e. don't stop. Why?

model_non_stream = utils.to_streaming_inference(model_non_stream_batch, flags, Modes.NON_STREAM_INFERENCE)

For the latest kws_streaming, tf.keras.backend.learning_phase() is called in several python files. It seems tf_keras should be used instead of keras 3.

If TF_USE_LEGACY_KERAS=0, 01_train.ipynb/02_inference.ipynb report error.

which version is used, keras 3 or tf_keras?

kws_streaming @rybakov

[sushreebarsa]

@andrewmabc Could you refer to this for more information on kws_streaming.
Thank you!

[andrewmabc]

Thanks for your reply. I have read the webpage of 'Streaming Aware neural network models'.

I just want to run jupyter sample code of 'Quick onboarding with toy demo'. I think I should install correct packages. But I can't get the reason why utils.to_streaming_inference() can't return in 02_inference.ipynb.

[sushreebarsa]

@andrewmabc Could you try with Keras 3 as tf_keras will be legacy Keras?
Thank you!

[andrewmabc]

I use keras 3 and don't install tf_keras. 00_check_data.ipynb report error when executing fourth cell.

The fourth cell is :

import tensorflow as tf
import tensorflow.compat.v1 as tf1
import logging
from kws_streaming.models import model_flags
from kws_streaming.models import models
from kws_streaming.layers.modes import Modes
from kws_streaming.train import test
from kws_streaming.models import utils
from kws_streaming.data import input_data
from kws_streaming.data import input_data_utils as du
from kws_streaming.models import model_params

Reported error message :

import tensorflow as tf
import tensorflow.compat.v1 as tf1
import logging
from kws_streaming.models import model_flags
from kws_streaming.models import models
from kws_streaming.layers.modes import Modes
from kws_streaming.train import test
from kws_streaming.models import utils
from kws_streaming.data import input_data
from kws_streaming.data import input_data_utils as du
from kws_streaming.models import model_params
2024-04-25 07:03:24.703360: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
WARNING:tensorflow:Your environment has TF_USE_LEGACY_KERAS set to True, but you do not have the tf_keras package installed. You must install it in order to use the legacy tf.keras. Install it via: pip install tf_keras

ImportError Traceback (most recent call last)
Cell In[4], line 5
3 import logging
4 from kws_streaming.models import model_flags
----> 5 from kws_streaming.models import models
6 from kws_streaming.layers.modes import Modes
7 from kws_streaming.train import test

File /export/Intel_ssd/debug_kws/colab/./google-research/kws_streaming/models/models.py:17
1 # coding=utf-8
2 # Copyright 2024 The Google Research Authors.
3 #
(...)
13 # See the License for the specific language governing permissions and
14 # limitations under the License.
16 """Supported models."""
---> 17 import kws_streaming.models.att_mh_rnn as att_mh_rnn
18 import kws_streaming.models.att_rnn as att_rnn
19 import kws_streaming.models.bc_resnet as bc_resnet

File /export/Intel_ssd/debug_kws/colab/./google-research/kws_streaming/models/att_mh_rnn.py:18
16 """BiRNN model with multihead attention."""
17 from kws_streaming.layers import modes
---> 18 from kws_streaming.layers import speech_features
19 from kws_streaming.layers.compat import tf
20 import kws_streaming.models.model_utils as utils

File /export/Intel_ssd/debug_kws/colab/./google-research/kws_streaming/layers/speech_features.py:26
24 from kws_streaming.layers import random_shift
25 from kws_streaming.layers import random_stretch_squeeze
---> 26 from kws_streaming.layers import spectrogram_augment
27 from kws_streaming.layers import spectrogram_cutout
28 from kws_streaming.layers import windowing

File /export/Intel_ssd/debug_kws/colab/./google-research/kws_streaming/layers/spectrogram_augment.py:20
17 from typing import Any, Dict
19 import tensorflow as tf
---> 20 import tensorflow_model_optimization as tfmot
22 from kws_streaming.layers.compat import tf
23 from tensorflow.python.ops import array_ops # pylint: disable=g-direct-tensorflow-import

File /export/Intel_ssd/debug_kws/colab/venv3/lib/python3.11/site-packages/tensorflow_model_optimization/init.py:86
82 # To ensure users only access the expected public API, the API structure is
83 # created in the api directory. Import all api modules.
84 from tensorflow_model_optimization.python.core import version
---> 86 from tensorflow_model_optimization.python.core.api import clustering
87 from tensorflow_model_optimization.python.core.api import experimental
88 from tensorflow_model_optimization.python.core.api import quantization

File /export/Intel_ssd/debug_kws/colab/venv3/lib/python3.11/site-packages/tensorflow_model_optimization/python/core/api/init.py:16
1 # Copyright 2021 The TensorFlow Authors. All Rights Reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
13 # limitations under the License.
14 # ==============================================================================
15 """Import API modules for Tensorflow Model Optimization."""
---> 16 from tensorflow_model_optimization.python.core.api import clustering
17 from tensorflow_model_optimization.python.core.api import experimental
18 from tensorflow_model_optimization.python.core.api import quantization

File /export/Intel_ssd/debug_kws/colab/venv3/lib/python3.11/site-packages/tensorflow_model_optimization/python/core/api/clustering/init.py:16
1 # Copyright 2020 The TensorFlow Authors. All Rights Reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
13 # limitations under the License.
14 # ==============================================================================
15 """Module containing code for clustering."""
---> 16 from tensorflow_model_optimization.python.core.api.clustering import keras

File /export/Intel_ssd/debug_kws/colab/venv3/lib/python3.11/site-packages/tensorflow_model_optimization/python/core/api/clustering/keras/init.py:19
16 # pylint: disable=g-bad-import-order
17 from tensorflow_model_optimization.python.core.clustering.keras import experimental
---> 19 from tensorflow_model_optimization.python.core.clustering.keras.cluster import cluster_scope
20 from tensorflow_model_optimization.python.core.clustering.keras.cluster import cluster_weights
21 from tensorflow_model_optimization.python.core.clustering.keras.cluster import strip_clustering

File /export/Intel_ssd/debug_kws/colab/venv3/lib/python3.11/site-packages/tensorflow_model_optimization/python/core/clustering/keras/cluster.py:22
19 import tensorflow as tf
21 from tensorflow_model_optimization.python.core.clustering.keras import cluster_config
---> 22 from tensorflow_model_optimization.python.core.clustering.keras import cluster_wrapper
23 from tensorflow_model_optimization.python.core.clustering.keras import clustering_centroids
24 from tensorflow_model_optimization.python.core.keras.compat import keras

File /export/Intel_ssd/debug_kws/colab/venv3/lib/python3.11/site-packages/tensorflow_model_optimization/python/core/clustering/keras/cluster_wrapper.py:23
21 from tensorflow_model_optimization.python.core.clustering.keras import cluster_config
22 from tensorflow_model_optimization.python.core.clustering.keras import clusterable_layer
---> 23 from tensorflow_model_optimization.python.core.clustering.keras import clustering_centroids
24 from tensorflow_model_optimization.python.core.clustering.keras import clustering_registry
25 from tensorflow_model_optimization.python.core.keras.compat import keras

File /export/Intel_ssd/debug_kws/colab/venv3/lib/python3.11/site-packages/tensorflow_model_optimization/python/core/clustering/keras/clustering_centroids.py:22
20 from tensorflow.python.ops import clustering_ops
21 from tensorflow_model_optimization.python.core.clustering.keras import cluster_config
---> 22 from tensorflow_model_optimization.python.core.keras.compat import keras
25 k = keras.backend
26 CentroidInitialization = cluster_config.CentroidInitialization

File /export/Intel_ssd/debug_kws/colab/venv3/lib/python3.11/site-packages/tensorflow_model_optimization/python/core/keras/compat.py:41
37 keras_internal = tf.keras
38 return keras_internal
---> 41 keras = _get_keras_instance()
43 def assign(ref, value, name=None):
44 if hasattr(tf, 'assign'):

File /export/Intel_ssd/debug_kws/colab/venv3/lib/python3.11/site-packages/tensorflow_model_optimization/python/core/keras/compat.py:33, in _get_keras_instance()
30 os.environ['TF_USE_LEGACY_KERAS'] = '1'
32 # Use Keras 2.
---> 33 version_fn = getattr(tf.keras, 'version', None)
34 if version_fn and version_fn().startswith('3.'):
35 import tf_keras as keras_internal # pylint: disable=g-import-not-at-top,unused-import

File /export/Intel_ssd/debug_kws/colab/venv3/lib/python3.11/site-packages/tensorflow/python/util/lazy_loader.py:181, in KerasLazyLoader.getattr(self, item)
179 return super(types.ModuleType, self).getattribute(item)
180 if not self._tfll_initialized:
--> 181 self._initialize()
182 if self._tfll_keras_version == "keras_3":
183 if (
184 self._tfll_mode == "v1"
185 and not self._tfll_submodule
186 and item.startswith("compat.v1.")
187 ):

File /export/Intel_ssd/debug_kws/colab/venv3/lib/python3.11/site-packages/tensorflow/python/util/lazy_loader.py:173, in KerasLazyLoader._initialize(self)
169 super().init(
170 self._tfll_name, self._tfll_parent_module_globals, package_name
171 )
172 else:
--> 173 raise ImportError( # pylint: disable=raise-missing-from
174 "Keras cannot be imported. Check that it is installed."
175 )

ImportError: Keras cannot be imported. Check that it is installed.

I notice some tf_keras functions are used in latest kws_streaming, such as tf.keras.backend.learning_phase(). backend.learning_phase() can't be found in keras 3.

[SuryanarayanaY]

Hi @andrewmabc ,

I have checked the requirements.txt file from kws_streaming and it is using tf_nightly==2.3.0.dev20200515 version which is quiet older. Hence it will not work with Keras3. Enabling tf_keras (i.e.Keras2) might work.

We are not supporting kws_training repo here. If you have any issues on same please report at concern repo. Thanks!

[andrewmabc]

what/where is concern repo?

From the issue(google-research/google-research#783, rybakov said, 'use tf-nightly. kws_streaming is always in sync with the latest version of tf-nightly,' But in the file of requirements.txt, the version of tf_nightly is too old.

[SuryanarayanaY]

what/where is concern repo?

From the issue(google-research/google-research#783, rybakov said, 'use tf-nightly. kws_streaming is always in sync with the latest version of tf-nightly,' But in the file of requirements.txt, the version of tf_nightly is too old.

I think you need to raise an issue there itself. The attached issue is quiet older and may be current issue is fresh. Kindly make a note that if other libraries has dependency on TF or keras and its not working then it should be addressed there itself. May be they will communicate internally or do required changes.

I suggest you raise an issue there and let hear the reply from there.

[andrewmabc]

Thanks for your reply. But I still don't understand what is concern repo. How can I raise an issue on the concern repo?

[SuryanarayanaY]

what/where is concern repo?

From the issue(google-research/google-research#783, rybakov said, 'use tf-nightly. kws_streaming is always in sync with the latest version of tf-nightly,' But in the file of requirements.txt, the version of tf_nightly is too old.

Please note that kws_streaming is not a dependency of keras.May be it is using keras but not vice versa. You may please raise the issue in google-research repo and let's hear from there.

[andrewmabc]

Thanks for your reply. I raise the same issue in google-research repo.(google-research/google-research#2051)