Skip to content

Releases: tensorflow/tensorflow

TensorFlow 2.7.4

02 Sep 20:03
@tensorflow-jenkins tensorflow-jenkins
v2.7.4
a73cc22
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
TensorFlow 2.7.4

Release 2.7.4

Note: This is the last release in the 2.7.x series

This releases introduces several vulnerability fixes:

Assets 2

TensorFlow 2.10.0-rc3

29 Aug 18:01
@tensorflow-jenkins tensorflow-jenkins
v2.10.0-rc3
f082fa9
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
TensorFlow 2.10.0-rc3 Pre-release
Pre-release

Release 2.10.0

Breaking Changes

  • Causal attention in keras.layers.Attention and keras.layers.AdditiveAttention is now specified in the call() method via the use_causal_mask argument (rather than in the constructor), for consistency with other layers.
  • Some files in tensorflow/python/training have been moved to tensorflow/python/tracking and tensorflow/python/checkpoint. Please update your imports accordingly, the old files will be removed in Release 2.11.
  • tf.keras.optimizers.experimental.Optimizer will graduate in Release 2.11, which means tf.keras.optimizers.Optimizer will be an alias of tf.keras.optimizers.experimental.Optimizer. The current tf.keras.optimizers.Optimizer will continue to be supported as tf.keras.optimizers.legacy.Optimizer, e.g.,tf.keras.optimizers.legacy.Adam. Most users won't be affected by this change, but please check the API doc if any API used in your workflow is changed or deprecated, and make adaptions. If you decide to keep using the old optimizer, please explicitly change your optimizer to tf.keras.optimizers.legacy.Optimizer.
  • RNG behavior change for tf.keras.initializers. Keras initializers will now use stateless random ops to generate random numbers.
    • Both seeded and unseeded initializers will always generate the same values every time they are called (for a given variable shape). For unseeded initializers (seed=None), a random seed will be created and assigned at initializer creation (different initializer instances get different seeds).
    • An unseeded initializer will raise a warning if it is reused (called) multiple times. This is because it would produce the same values each time, which may not be intended.

Deprecations

  • The C++ tensorflow::Code and tensorflow::Status will become aliases of respectively absl::StatusCode and absl::Status in some future release.
    • Use tensorflow::OkStatus() instead of tensorflow::Status::OK().
    • Stop constructing Status objects from tensorflow::error::Code.
    • One MUST NOT access tensorflow::errors::Code fields. Accessing tensorflow::error::Code fields is fine.
      • Use the constructors such as tensorflow::errors:InvalidArgument to create status using an error code without accessing it.
      • Use the free functions such as tensorflow::errors::IsInvalidArgument if needed.
      • In the last resort, use e.g.static_cast<tensorflow::errors::Code>(error::Code::INVALID_ARGUMENT) or static_cast<int>(code) for comparisons.
  • tensorflow::StatusOr will also become in the future alias to absl::StatusOr, so use StatusOr::value instead of StatusOr::ConsumeValueOrDie.

Major Features and Improvements

  • tf.lite:

    • New operations supported:
      • tflite SelectV2 now supports 5D.
      • tf.einsum is supported with multiple unknown shapes.
      • tf.unsortedsegmentprod op is supported.
      • tf.unsortedsegmentmax op is supported.
      • tf.unsortedsegmentsum op is supported.
    • Updates to existing operations:
      • tfl.scatter_nd now supports I1 for update arg.
    • Upgrade Flatbuffers v2.0.5 from v1.12.0

  • tf.keras:

    • EinsumDense layer is moved from experimental to core. Its import path is moved from tf.keras.layers.experimental.EinsumDense to tf.keras.layers.EinsumDense.
    • Added tf.keras.utils.audio_dataset_from_directory utility to easily generate audio classification datasets from directories of .wav files.
    • Added subset="both" support in tf.keras.utils.image_dataset_from_directory,tf.keras.utils.text_dataset_from_directory, and audio_dataset_from_directory, to be used with the validation_split argument, for returning both dataset splits at once, as a tuple.
    • Added tf.keras.utils.split_dataset utility to split a Dataset object or a list/tuple of arrays into two Dataset objects (e.g. train/test).
    • Added step granularity to BackupAndRestore callback for handling distributed training failures & restarts. The training state can now be restored at the exact epoch and step at which it was previously saved before failing.
    • Added tf.keras.dtensor.experimental.optimizers.AdamW. This optimizer is similar as the existing keras.optimizers.experimental.AdamW, and works in the DTensor training use case.
    • Improved masking support for tf.keras.layers.MultiHeadAttention.
      • Implicit masks for query, key and value inputs will automatically be used to compute a correct attention mask for the layer. These padding masks will be combined with any attention_mask passed in directly when calling the layer. This can be used with tf.keras.layers.Embedding with mask_zero=True to automatically infer a correct padding mask.
      • Added a use_causal_mask call time arugment to the layer. Passing use_causal_mask=True will compute a causal attention mask, and optionally combine it with any attention_mask passed in directly when calling the layer.
    • Added ignore_class argument in the loss SparseCategoricalCrossentropy and metrics IoU and MeanIoU, to specify a class index to be ignored during loss/metric computation (e.g. a background/void class).
    • Added tf.keras.models.experimental.SharpnessAwareMinimization. This class implements the sharpness-aware minimization technique, which boosts model performance on various tasks, e.g., ResNet on image classification.

  • tf.data:

    • Added support for cross-trainer data caching in tf.data service. This saves computation resources when concurrent training jobs train from the same dataset. See (https://www.tensorflow.org/api_docs/python/tf/data/experimental/service#sharing_tfdata_service_with_concurrent_trainers) for more details.
    • Added dataset_id to tf.data.experimental.service.register_dataset. If provided, tf.data service will use the provided ID for the dataset. If the dataset ID already exists, no new dataset will be registered. This is useful if multiple training jobs need to use the same dataset for training. In this case, users should call register_dataset with the same dataset_id.
    • Added a new field, inject_prefetch, to tf.data.experimental.OptimizationOptions. If it is set to True,tf.data will now automatically add a prefetch transformation to datasets that end in synchronous transformations. This enables data generation to be overlapped with data consumption. This may cause a small increase in memory usage due to buffering. To enable this behavior, set inject_prefetch=True in tf.data.experimental.OptimizationOptions.
    • Added a new value to tf.data.Options.autotune.autotune_algorithm: STAGE_BASED. If the autotune algorithm is set to STAGE_BASED, then it runs a new algorithm that can get the same performance with lower CPU/memory usage.
    • Added tf.data.experimental.from_list, a new API for creating Datasets from lists of elements.

  • tf.distribute:

    • Added tf.distribute.experimental.PreemptionCheckpointHandler to handle worker preemption/maintenance and cluster-wise consistent error reporting for tf.distribute.MultiWorkerMirroredStrategy. Specifically, for the type of interruption with advance notice, it automatically saves a checkpoint, exits the program without raising an unrecoverable error, and restores the progress when training restarts.

  • tf.math:

    • Added tf.math.approx_max_k and tf.math.approx_min_k which are the optimized alternatives to tf.math.top_k on TPU. The performance difference range from 8 to 100 times depending on the size of k. When running on CPU and GPU, a non-optimized XLA kernel is used.

  • tf.train:

    • Added tf.train.TrackableView which allows users to inspect the TensorFlow Trackable object (e.g. tf.Module, Keras Layers and models).

  • tf.vectorized_map:

    • Added an optional parameter: warn. This parameter controls whether or not warnings will be printed when operations in the provided fn fall back to a while loop.

  • XLA:

  • CPU performance optimizations:

    • x86 CPUs: oneDNN bfloat16 auto-mixed precision grappler graph optimization pass has been renamed from auto_mixed_precision_mkl to auto_mixed_precision_onednn_bfloat16. See example usage here.
    • aarch64 CPUs: Experimental performance optimizations from Compute Library for the Arm® Architecture (ACL) are available through oneDNN in the default Linux aarch64 package (pip install tensorflow).
      • The optimizations are disabled by default.
      • Set the environment variable TF_ENABLE_ONEDNN_OPTS=1 to enable the optimizations. Setting the variable to 0 or uns...
Read more
Assets 2

TensorFlow 2.10.0-rc2

23 Aug 18:09
@tensorflow-jenkins tensorflow-jenkins
v2.10.0-rc2
4bf2119
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
TensorFlow 2.10.0-rc2 Pre-release
Pre-release

Release 2.10.0

Breaking Changes

  • Causal attention in keras.layers.Attention and keras.layers.AdditiveAttention is now specified in the call() method via the use_causal_mask argument (rather than in the constructor), for consistency with other layers.
  • Some files in tensorflow/python/training have been moved to tensorflow/python/tracking and tensorflow/python/checkpoint. Please update your imports accordingly, the old files will be removed in Release 2.11.
  • tf.keras.optimizers.experimental.Optimizer will graduate in Release 2.11, which means tf.keras.optimizers.Optimizer will be an alias of tf.keras.optimizers.experimental.Optimizer. The current tf.keras.optimizers.Optimizer will continue to be supported as tf.keras.optimizers.legacy.Optimizer, e.g.,tf.keras.optimizers.legacy.Adam. Most users won't be affected by this change, but please check the API doc if any API used in your workflow is changed or deprecated, and make adaptions. If you decide to keep using the old optimizer, please explicitly change your optimizer to tf.keras.optimizers.legacy.Optimizer.
  • RNG behavior change for tf.keras.initializers. Keras initializers will now use stateless random ops to generate random numbers.
    • Both seeded and unseeded initializers will always generate the same values every time they are called (for a given variable shape). For unseeded initializers (seed=None), a random seed will be created and assigned at initializer creation (different initializer instances get different seeds).
    • An unseeded initializer will raise a warning if it is reused (called) multiple times. This is because it would produce the same values each time, which may not be intended.

Major Features and Improvements

  • tf.lite:

    • New operations supported:
      • tflite SelectV2 now supports 5D.
      • tf.einsum is supported with multiple unknown shapes.
      • tf.unsortedsegmentprod op is supported.
      • tf.unsortedsegmentmax op is supported.
      • tf.unsortedsegmentsum op is supported.
    • Updates to existing operations:
      • tfl.scatter_nd now supports I1 for update arg.
    • Upgrade Flatbuffers v2.0.5 from v1.12.0

  • tf.keras:

    • EinsumDense layer is moved from experimental to core. Its import path is moved from tf.keras.layers.experimental.EinsumDense to tf.keras.layers.EinsumDense.
    • Added tf.keras.utils.audio_dataset_from_directory utility to easily generate audio classification datasets from directories of .wav files.
    • Added subset="both" support in tf.keras.utils.image_dataset_from_directory,tf.keras.utils.text_dataset_from_directory, and audio_dataset_from_directory, to be used with the validation_split argument, for returning both dataset splits at once, as a tuple.
    • Added tf.keras.utils.split_dataset utility to split a Dataset object or a list/tuple of arrays into two Dataset objects (e.g. train/test).
    • Added step granularity to BackupAndRestore callback for handling distributed training failures & restarts. The training state can now be restored at the exact epoch and step at which it was previously saved before failing.
    • Added tf.keras.dtensor.experimental.optimizers.AdamW. This optimizer is similar as the existing keras.optimizers.experimental.AdamW, and works in the DTensor training use case.
    • Improved masking support for tf.keras.layers.MultiHeadAttention.
      • Implicit masks for query, key and value inputs will automatically be used to compute a correct attention mask for the layer. These padding masks will be combined with any attention_mask passed in directly when calling the layer. This can be used with tf.keras.layers.Embedding with mask_zero=True to automatically infer a correct padding mask.
      • Added a use_causal_mask call time arugment to the layer. Passing use_causal_mask=True will compute a causal attention mask, and optionally combine it with any attention_mask passed in directly when calling the layer.
    • Added ignore_class argument in the loss SparseCategoricalCrossentropy and metrics IoU and MeanIoU, to specify a class index to be ignored during loss/metric computation (e.g. a background/void class).
    • Added tf.keras.models.experimental.SharpnessAwareMinimization. This class implements the sharpness-aware minimization technique, which boosts model performance on various tasks, e.g., ResNet on image classification.

  • tf.data:

    • Added support for cross-trainer data caching in tf.data service. This saves computation resources when concurrent training jobs train from the same dataset. See (https://www.tensorflow.org/api_docs/python/tf/data/experimental/service#sharing_tfdata_service_with_concurrent_trainers) for more details.
    • Added dataset_id to tf.data.experimental.service.register_dataset. If provided, tf.data service will use the provided ID for the dataset. If the dataset ID already exists, no new dataset will be registered. This is useful if multiple training jobs need to use the same dataset for training. In this case, users should call register_dataset with the same dataset_id.
    • Added a new field, inject_prefetch, to tf.data.experimental.OptimizationOptions. If it is set to True,tf.data will now automatically add a prefetch transformation to datasets that end in synchronous transformations. This enables data generation to be overlapped with data consumption. This may cause a small increase in memory usage due to buffering. To enable this behavior, set inject_prefetch=True in tf.data.experimental.OptimizationOptions.
    • Added a new value to tf.data.Options.autotune.autotune_algorithm: STAGE_BASED. If the autotune algorithm is set to STAGE_BASED, then it runs a new algorithm that can get the same performance with lower CPU/memory usage.
    • Added tf.data.experimental.from_list, a new API for creating Datasets from lists of elements.

  • tf.distribute:

    • Added tf.distribute.experimental.PreemptionCheckpointHandler to handle worker preemption/maintenance and cluster-wise consistent error reporting for tf.distribute.MultiWorkerMirroredStrategy. Specifically, for the type of interruption with advance notice, it automatically saves a checkpoint, exits the program without raising an unrecoverable error, and restores the progress when training restarts.

  • tf.math:

    • Added tf.math.approx_max_k and tf.math.approx_min_k which are the optimized alternatives to tf.math.top_k on TPU. The performance difference range from 8 to 100 times depending on the size of k. When running on CPU and GPU, a non-optimized XLA kernel is used.

  • tf.train:

    • Added tf.train.TrackableView which allows users to inspect the TensorFlow Trackable object (e.g. tf.Module, Keras Layers and models).

  • tf.vectorized_map:

    • Added an optional parameter: warn. This parameter controls whether or not warnings will be printed when operations in the provided fn fall back to a while loop.

  • XLA:

    • MWMS is now compilable with XLA.

  • oneDNN CPU performance optimizations:

    • x86 CPUs: oneDNN bfloat16 auto-mixed precision grappler graph optimization pass has been renamed from auto_mixed_precision_mkl to auto_mixed_precision_onednn_bfloat16. See example usage here.
    • aarch64 CPUs: Experimental Arm Compute Library (ACL) CPU performance optimizations through oneDNN are available in the default Linux aarch64 package (pip install tensorflow).
      • The optimizations are disabled by default.
      • Set the environment variable TF_ENABLE_ONEDNN_OPTS=1 to enable the optimizations. Setting the variable to 0 or unsetting it will disable the optimizations.
      • These optimizations can yield slightly different numerical results from when they are off due to floating-point round-off errors from different computation approaches and orders.
      • To verify that the optimizations are on, look for a message with "oneDNN custom operations are on" in the log. If the exact phrase is not there, it means they are off.

Bug Fixes and Other Changes

  • New argument experimental_device_ordinal in LogicalDeviceConfiguration to control the order of logical devices. (GPU only)

  • tf.keras:

    • Changed the TensorBoard tag names produced by the tf.keras.callbacks.TensorBoard callback, so that summaries logged automatically for model weights now include either a /histogram or /image suffix in their tag names, in order to prevent tag name collisions across summary types.

  • When running on GPU (with cuDNN version 7.6.3 or later),tf.nn.depthwise_conv2d backprop to filter (and therefore also tf.keras.layers.DepthwiseConv2D) now operate deterministically (and tf.errors.UnimplementedError is no longer thrown) when op-determinism has been enabled via `tf.co...

Read more
Assets 2

TensorFlow 2.10.0-rc1

15 Aug 18:31
@tensorflow-jenkins tensorflow-jenkins
v2.10.0-rc1
5dd05fd
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
TensorFlow 2.10.0-rc1 Pre-release
Pre-release

Release 2.10.0

Breaking Changes

  • Causal attention in keras.layers.Attention and keras.layers.AdditiveAttention is now specified in the call() method via the use_causal_mask argument (rather than in the constructor), for consistency with other layers.
  • Some files in tensorflow/python/training have been moved to tensorflow/python/tracking and tensorflow/python/checkpoint. Please update your imports accordingly, the old files will be removed in Release 2.11.
  • tf.keras.optimizers.experimental.Optimizer will graduate in Release 2.11, which means tf.keras.optimizers.Optimizer will be an alias of tf.keras.optimizers.experimental.Optimizer. The current tf.keras.optimizers.Optimizer will continue to be supported as tf.keras.optimizers.legacy.Optimizer, e.g.,tf.keras.optimizers.legacy.Adam. Most users won't be affected by this change, but please check the API doc if any API used in your workflow is changed or deprecated, and make adaptions. If you decide to keep using the old optimizer, please explicitly change your optimizer to tf.keras.optimizers.legacy.Optimizer.
  • RNG behavior change for tf.keras.initializers. Keras initializers will now use stateless random ops to generate random numbers.
    • Both seeded and unseeded initializers will always generate the same values every time they are called (for a given variable shape). For unseeded initializers (seed=None), a random seed will be created and assigned at initializer creation (different initializer instances get different seeds).
    • An unseeded initializer will raise a warning if it is reused (called) multiple times. This is because it would produce the same values each time, which may not be intended.

Major Features and Improvements

  • tf.lite:

    • New operations supported:
      • tflite SelectV2 now supports 5D.
      • tf.einsum is supported with multiple unknown shapes.
      • tf.unsortedsegmentprod op is supported.
      • tf.unsortedsegmentmax op is supported.
      • tf.unsortedsegmentsum op is supported.
    • Updates to existing operations:
      • tfl.scatter_nd now supports I1 for update arg.
    • Upgrade Flatbuffers v2.0.5 from v1.12.0

  • tf.keras:

    • EinsumDense layer is moved from experimental to core. Its import path is moved from tf.keras.layers.experimental.EinsumDense to tf.keras.layers.EinsumDense.
    • Added tf.keras.utils.audio_dataset_from_directory utility to easily generate audio classification datasets from directories of .wav files.
    • Added subset="both" support in tf.keras.utils.image_dataset_from_directory,tf.keras.utils.text_dataset_from_directory, and audio_dataset_from_directory, to be used with the validation_split argument, for returning both dataset splits at once, as a tuple.
    • Added tf.keras.utils.split_dataset utility to split a Dataset object or a list/tuple of arrays into two Dataset objects (e.g. train/test).
    • Added step granularity to BackupAndRestore callback for handling distributed training failures & restarts. The training state can now be restored at the exact epoch and step at which it was previously saved before failing.
    • Added tf.keras.dtensor.experimental.optimizers.AdamW. This optimizer is similar as the existing keras.optimizers.experimental.AdamW, and works in the DTensor training use case.
    • Improved masking support for tf.keras.layers.MultiHeadAttention.
      • Implicit masks for query, key and value inputs will automatically be used to compute a correct attention mask for the layer. These padding masks will be combined with any attention_mask passed in directly when calling the layer. This can be used with tf.keras.layers.Embedding with mask_zero=True to automatically infer a correct padding mask.
      • Added a use_causal_mask call time arugment to the layer. Passing use_causal_mask=True will compute a causal attention mask, and optionally combine it with any attention_mask passed in directly when calling the layer.
    • Added ignore_class argument in the loss SparseCategoricalCrossentropy and metrics IoU and MeanIoU, to specify a class index to be ignored during loss/metric computation (e.g. a background/void class).
    • Added tf.keras.models.experimental.SharpnessAwareMinimization. This class implements the sharpness-aware minimization technique, which boosts model performance on various tasks, e.g., ResNet on image classification.

  • tf.data:

    • Added support for cross-trainer data caching in tf.data service. This saves computation resources when concurrent training jobs train from the same dataset. See (https://www.tensorflow.org/api_docs/python/tf/data/experimental/service#sharing_tfdata_service_with_concurrent_trainers) for more details.
    • Added dataset_id to tf.data.experimental.service.register_dataset. If provided, tf.data service will use the provided ID for the dataset. If the dataset ID already exists, no new dataset will be registered. This is useful if multiple training jobs need to use the same dataset for training. In this case, users should call register_dataset with the same dataset_id.
    • Added a new field, inject_prefetch, to tf.data.experimental.OptimizationOptions. If it is set to True,tf.data will now automatically add a prefetch transformation to datasets that end in synchronous transformations. This enables data generation to be overlapped with data consumption. This may cause a small increase in memory usage due to buffering. To enable this behavior, set inject_prefetch=True in tf.data.experimental.OptimizationOptions.
    • Added a new value to tf.data.Options.autotune.autotune_algorithm: STAGE_BASED. If the autotune algorithm is set to STAGE_BASED, then it runs a new algorithm that can get the same performance with lower CPU/memory usage.
    • Added tf.data.experimental.from_list, a new API for creating Datasets from lists of elements.

  • tf.distribute:

    • Added tf.distribute.experimental.PreemptionCheckpointHandler to handle worker preemption/maintenance and cluster-wise consistent error reporting for tf.distribute.MultiWorkerMirroredStrategy. Specifically, for the type of interruption with advance notice, it automatically saves a checkpoint, exits the program without raising an unrecoverable error, and restores the progress when training restarts.

  • tf.math:

    • Added tf.math.approx_max_k and tf.math.approx_min_k which are the optimized alternatives to tf.math.top_k on TPU. The performance difference range from 8 to 100 times depending on the size of k. When running on CPU and GPU, a non-optimized XLA kernel is used.

  • tf.train:

    • Added tf.train.TrackableView which allows users to inspect the TensorFlow Trackable object (e.g. tf.Module, Keras Layers and models).

  • tf.vectorized_map:

    • Added an optional parameter: warn. This parameter controls whether or not warnings will be printed when operations in the provided fn fall back to a while loop.

  • XLA:

    • MWMS is now compilable with XLA.

  • oneDNN CPU performance optimizations:

    • x86 CPUs: oneDNN bfloat16 auto-mixed precision grappler graph optimization pass has been renamed from auto_mixed_precision_mkl to auto_mixed_precision_onednn_bfloat16. See example usage here.
    • aarch64 CPUs: Experimental oneDNN optimizations are available in the default Linux aarch64 package (pip install tensorflow).
      • The optimizations are disabled by default.
      • Set the environment variable TF_ENABLE_ONEDNN_OPTS=1 to enable the optimizations. Setting the variable to 0 or unsetting it will disable the optimizations.
      • These optimizations can yield slightly different numerical results from when they are off due to floating-point round-off errors from different computation approaches and orders.
      • To verify that the optimizations are on, look for a message with "oneDNN custom operations are on" in the log. If the exact phrase is not there, it means they are off.

Bug Fixes and Other Changes

  • New argument experimental_device_ordinal in LogicalDeviceConfiguration to control the order of logical devices. (GPU only)

  • tf.keras:

    • Changed the TensorBoard tag names produced by the tf.keras.callbacks.TensorBoard callback, so that summaries logged automatically for model weights now include either a /histogram or /image suffix in their tag names, in order to prevent tag name collisions across summary types.

  • When running on GPU (with cuDNN version 7.6.3 or later),tf.nn.depthwise_conv2d backprop to filter (and therefore also tf.keras.layers.DepthwiseConv2D) now operate deterministically (and tf.errors.UnimplementedError is no longer thrown) when op-determinism has been enabled via tf.config.experimental.enable_op_determinism. This clo...

Read more
Assets 2

TensorFlow 2.10.0-rc0

03 Aug 22:21
@tensorflow-jenkins tensorflow-jenkins
v2.10.0-rc0
371c061
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
TensorFlow 2.10.0-rc0 Pre-release
Pre-release

Release 2.10.0

Breaking Changes

  • Causal attention in keras.layers.Attention and
    keras.layers.AdditiveAttention is now specified in the call() method
    via the use_causal_mask argument (rather than in the constructor),
    for consistency with other layers.
  • Some files in tensorflow/python/training have been moved to
    tensorflow/python/tracking and tensorflow/python/checkpoint. Please
    update your imports accordingly, the old files will be removed in Release
    2.11.
  • tf.keras.optimizers.experimental.Optimizer will graduate in Release 2.11,
    which means tf.keras.optimizers.Optimizer will be an alias of
    tf.keras.optimizers.experimental.Optimizer. The current
    tf.keras.optimizers.Optimizer will continue to be supported as
    tf.keras.optimizers.legacy.Optimizer, e.g.,
    tf.keras.optimizers.legacy.Adam. Most users won't be affected by this
    change, but please check the API doc
    if any API used in your workflow is changed or deprecated, and
    make adaptions. If you decide to keep using the old optimizer, please
    explicitly change your optimizer to tf.keras.optimizers.legacy.Optimizer.
  • RNG behavior change for tf.keras.initializers. Keras initializers will now
    use stateless random ops to generate random numbers.
    • Both seeded and unseeded initializers will always generate the same
      values every time they are called (for a given variable shape).
      For unseeded initializers (seed=None), a
      random seed will be created and assigned at initializer creation
      (different initializer instances get different seeds).
    • An unseeded initializer will raise a warning if it is reused (called)
      multiple times. This is because it would produce the same values
      each time, which may not be intended.

Major Features and Improvements

  • tf.lite:

    • New operations supported:
      • tflite SelectV2 now supports 5D.
      • tf.einsum is supported with multiple unknown shapes.
      • tf.unsortedsegmentprod op is supported.
      • tf.unsortedsegmentmax op is supported.
      • tf.unsortedsegmentsum op is supported.
    • Updates to existing operations:
      • tfl.scatter_nd now supports I1 for update arg.
    • Upgrade Flatbuffers v2.0.5 from v1.12.0

  • tf.keras:

    • EinsumDense layer moved from experimental to core. Its import path
      moved from tf.keras.layers.experimental.EinsumDense to
      tf.keras.layers.EinsumDense.
    • Added tf.keras.utils.audio_dataset_from_directory utility to easily
      generate audio classification datasets from directories of .wav files.
    • Added subset="both" support in
      tf.keras.utils.image_dataset_from_directory,
      tf.keras.utils.text_dataset_from_directory, and
      audio_dataset_from_directory, to be used with the validation_split
      argument, for returning both dataset splits at once, as a tuple.
    • Added tf.keras.utils.split_dataset utility to split a Dataset object
      or a list/tuple of arrays into two Dataset objects (e.g. train/test).
    • Added step granularity to BackupAndRestore callback for handling
      distributed training failures & restarts. The training state can now be
      restored at the exact epoch and step at which it was previously saved
      before failing.
    • Added tf.keras.dtensor.experimental.optimizers.AdamW.
      This optimizer is similar as the existing
      keras.optimizers.experimental.AdamW, and
      works in the DTensor training use case.
    • Improved masking support for tf.keras.layers.MultiHeadAttention.
      • Implicit masks for query, key and value inputs will
        automatically be used to compute a correct attention mask for the
        layer. These padding masks will be combined with any
        attention_mask passed in directly when calling the layer. This
        can be used with
        tf.keras.layers.Embedding
        with mask_zero=True to automatically infer a correct padding mask.
      • Added a use_causal_mask call time arugment to the layer. Passing
        use_causal_mask=True will compute a causal attention mask, and
        optionally combine it with any attention_mask passed in directly
        when calling the layer.
    • Added ignore_class argument in the loss
      SparseCategoricalCrossentropy and metrics IoU and MeanIoU,
      to specify a class index to be ignored
      during loss/metric computation (e.g. a background/void class).
    • Added tf.keras.models.experimental.SharpnessAwareMinimization.
      This class implements the sharpness-aware minimization technique, which
      boosts model performance on various tasks, e.g., ResNet on image
      classification.

  • tf.data:

    • Added support for cross-trainer data caching in tf.data service. This
      saves computation resources when concurrent training jobs train from the
      same dataset. See
      https://www.tensorflow.org/api_docs/python/tf/data/experimental/service#sharing_tfdata_service_with_concurrent_trainers
      for more details.
    • Added dataset_id to tf.data.experimental.service.register_dataset.
      If provided, tf.data service will use the provided ID for the dataset.
      If the dataset ID already exists, no new dataset will be registered.
      This is useful if multiple training jobs need to use the same dataset
      for training. In this case, users should call register_dataset with
      the same dataset_id.
    • Added a new field, inject_prefetch, to
      tf.data.experimental.OptimizationOptions. If it is set to True,
      tf.data will now automatically add a prefetch transformation to
      datasets that end in synchronous transformations. This enables data
      generation to be overlapped with data consumption. This may cause a
      small increase in memory usage due to buffering. To enable this
      behavior, set inject_prefetch=True in
      tf.data.experimental.OptimizationOptions.
    • Added a new value to tf.data.Options.autotune.autotune_algorithm:
      STAGE_BASED. If the autotune algorithm is set to STAGE_BASED, then it
      runs a new algorithm that can get the same performance with lower
      CPU/memory usage.
    • Added tf.data.experimental.from_list, a new API for creating
      Datasets from lists of elements.

  • tf.distribute:

    • Added tf.distribute.experimental.PreemptionCheckpointHandler
      to handle worker preemption/maintenance and cluster-wise consistent
      error reporting for tf.distribute.MultiWorkerMirroredStrategy.
      Specifically, for the type of interruption with advance notice, it
      automatically saves a checkpoint, exits the program without raising an
      unrecoverable error, and restores the progress when training restarts.

  • tf.math:

    • Added tf.math.approx_max_k and tf.math.approx_min_k which are the
      optimized alternatives to tf.math.top_k on TPU. The performance
      difference range from 8 to 100 times depending on the size of k. When
      running on CPU and GPU, a non-optimized XLA kernel is used.

  • tf.train:

    • Added tf.train.TrackableView which allows users to inspect the
      TensorFlow Trackable object (e.g. tf.Module, Keras Layers and models).

  • tf.vectorized_map:

    • Added an optional parameter: warn. This parameter controls whether or
      not warnings will be printed when operations in the provided fn fall
      back to a while loop.

  • XLA:

    • MWMS is now compilable with XLA.

Bug Fixes and Other Changes

  • New argument experimental_device_ordinal in LogicalDeviceConfiguration
    to control the order of logical devices. (GPU only)

  • tf.keras:

    • Changed the TensorBoard tag names produced by the
      tf.keras.callbacks.TensorBoard callback, so that summaries logged
      automatically for model weights now include either a /histogram or
      /image suffix in their tag names, in order to prevent tag name
      collisions across summary types.

  • When running on GPU (with cuDNN version 7.6.3 or later),
    tf.nn.depthwise_conv2d backprop to filter (and therefore also
    tf.keras.layers.DepthwiseConv2D) now operate deterministically (and
    tf.errors.UnimplementedError is no longer thrown) when op-determinism has
    been enabled via tf.config.experimental.enable_op_determinism. This closes
    issue 47174.

  • tf.random

    • Added tf.random.experimental.stateless_shuffle, a stateless version of
      tf.random.shuffle.

Deprecations

  • The C++ tensorflow::Code and tensorflow::Status will become aliases of
    respectively absl::StatusCode and absl::Status in some future release.
    • Use tensorflow::OkStatus() instead of `tensorflow::Status::OK(...
Read more
Assets 2

TensorFlow 2.9.1

23 May 17:00
@tensorflow-jenkins tensorflow-jenkins
v2.9.1
d8ce9f9
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
TensorFlow 2.9.1

Release 2.9.1

Add an upper bound for protobuf in setup.py since protobuf after version 3.20 is currently incompatible with TensorFlow. See #53234, protocolbuffers/protobuf#9954 and #56077.

Assets 2

TensorFlow 2.8.2

23 May 17:00
@tensorflow-jenkins tensorflow-jenkins
v2.8.2
2ea19cb
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
TensorFlow 2.8.2

Release 2.8.2

Add an upper bound for protobuf in setup.py since protobuf after version 3.20 is currently incompatible with TensorFlow. See #53234, protocolbuffers/protobuf#9954 and #56077.

Assets 2

TensorFlow 2.6.5

23 May 16:59
@tensorflow-jenkins tensorflow-jenkins
v2.6.5
6b54e9f
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
TensorFlow 2.6.5

Release 2.6.5

Add an upper bound for protobuf in setup.py since protobuf after version 3.20 is currently incompatible with TensorFlow. See #53234, protocolbuffers/protobuf#9954 and #56077.

This is the final release in the 2.6.x series.

Assets 2

TensorFlow 2.7.3

23 May 16:59
@tensorflow-jenkins tensorflow-jenkins
v2.7.3
fa3dcb4
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
TensorFlow 2.7.3

Release 2.7.3

Add an upper bound for protobuf in setup.py since protobuf after version 3.20 is currently incompatible with TensorFlow. See #53234, protocolbuffers/protobuf#9954 and #56077.

Assets 2

TensorFlow 2.9.0

16 May 21:12
@tensorflow-jenkins tensorflow-jenkins
v2.9.0
8a20d54
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
TensorFlow 2.9.0

Release 2.9.0

Breaking Changes

  • Due to security issues in TF 2.8, all boosted trees code has now been removed (after being deprecated in TF 2.8). Users should switch to TensorFlow Decision Forests.
  • Build, Compilation and Packaging
    • TensorFlow is now compiled with _GLIBCXX_USE_CXX11_ABI=1. Downstream projects that encounter std::__cxx11 or [abi:cxx11] linker errors will need to adopt this compiler option. See the GNU C++ Library docs on Dual ABI.
    • TensorFlow Python wheels now specifically conform to manylinux2014, an upgrade from manylinux2010. The minimum Pip version supporting manylinux2014 is Pip 19.3 (see pypa/manylinux. This change may affect you if you have been using TensorFlow on a very old platform equivalent to CentOS 6, as manylinux2014 targets CentOS 7 as a compatibility base. Note that TensorFlow does not officially support either platform.
    • Discussion for these changes can be found on SIG Build's TensorFlow Community Forum thread
  • The tf.keras.mixed_precision.experimental API has been removed. The non-experimental symbols under tf.keras.mixed_precision have been available since TensorFlow 2.4 and should be used instead.
    • The non-experimental API has some minor differences from the experimental API. In most cases, you only need to make three minor changes:
      • Remove the word "experimental" from tf.keras.mixed_precision symbols. E.g., replace tf.keras.mixed_precision.experimental.global_policy with tf.keras.mixed_precision.global_policy.
      • Replace tf.keras.mixed_precision.experimental.set_policy with tf.keras.mixed_precision.set_global_policy. The experimental symbol set_policy was renamed to set_global_policy in the non-experimental API.
      • Replace LossScaleOptimizer(opt, "dynamic") with LossScaleOptimizer(opt). If you pass anything other than "dynamic" to the second argument, see (1) of the next section.
    • In the following rare cases, you need to make more changes when switching to the non-experimental API:
      • If you passed anything other than "dynamic" to the loss_scale argument (the second argument) of LossScaleOptimizer:
      • If you passed a value to the loss_scale argument (the second argument) of Policy:
        • The experimental version of Policy optionally took in a tf.compat.v1.mixed_precision.LossScale in the constructor, which defaulted to a dynamic loss scale for the "mixed_float16" policy and no loss scale for other policies. In Model.compile, if the model's policy had a loss scale, the optimizer would be wrapped with a LossScaleOptimizer. With the non-experimental Policy, there is no loss scale associated with the Policy, and Model.compile wraps the optimizer with a LossScaleOptimizer if and only if the policy is a "mixed_float16" policy. If you previously passed a LossScale to the experimental Policy, consider just removing it, as the default loss scaling behavior is usually what you want. If you really want to customize the loss scaling behavior, you can wrap your optimizer with a LossScaleOptimizer before passing it to Model.compile.
      • If you use the very rarely-used function tf.keras.mixed_precision.experimental.get_layer_policy:
        • Replace tf.keras.mixed_precision.experimental.get_layer_policy(layer) with layer.dtype_policy.
  • tf.mixed_precision.experimental.LossScale and its subclasses have been removed from the TF2 namespace. This symbols were very rarely used and were only useful in TF2 for use in the now-removed tf.keras.mixed_precision.experimental API. The symbols are still available under tf.compat.v1.mixed_precision.
  • The experimental_relax_shapes heuristic for tf.function has been deprecated and replaced with reduce_retracing which encompasses broader heuristics to reduce the number of retraces (see below)

Major Features and Improvements

  • tf.keras:

    • Added tf.keras.applications.resnet_rs models. This includes the ResNetRS50, ResNetRS101, ResNetRS152, ResNetRS200, ResNetRS270, ResNetRS350 and ResNetRS420 model architectures. The ResNetRS models are based on the architecture described in Revisiting ResNets: Improved Training and Scaling Strategies
    • Added tf.keras.optimizers.experimental.Optimizer. The reworked optimizer gives more control over different phases of optimizer calls, and is easier to customize. We provide Adam, SGD, Adadelta, AdaGrad and RMSprop optimizers based on tf.keras.optimizers.experimental.Optimizer. Generally the new optimizers work in the same way as the old ones, but support new constructor arguments. In the future, the symbols tf.keras.optimizers.Optimizer/Adam/etc will point to the new optimizers, and the previous generation of optimizers will be moved to tf.keras.optimizers.legacy.Optimizer/Adam/etc.
    • Added L2 unit normalization layer tf.keras.layers.UnitNormalization.
    • Added tf.keras.regularizers.OrthogonalRegularizer, a new regularizer that encourages orthogonality between the rows (or columns) or a weight matrix.
    • Added tf.keras.layers.RandomBrightness layer for image preprocessing.
    • Added APIs for switching between interactive logging and absl logging. By default, Keras always writes the logs to stdout. However, this is not optimal in a non-interactive environment, where you don't have access to stdout, but can only view the logs. You can use tf.keras.utils.disable_interactive_logging() to write the logs to ABSL logging. You can also use tf.keras.utils.enable_interactive_logging() to change it back to stdout, or tf.keras.utils.is_interactive_logging_enabled() to check if interactive logging is enabled.
    • Changed default value for the verbose argument of Model.evaluate() and Model.predict() to "auto", which defaults to verbose=1 for most cases and defaults to verbose=2 when used with ParameterServerStrategy or with interactive logging disabled.
    • Argument jit_compile in Model.compile() now applies to Model.evaluate() and Model.predict(). Setting jit_compile=True in compile() compiles the model's training, evaluation, and inference steps to XLA. Note that jit_compile=True may not necessarily work for all models.
    • Added DTensor-related Keras APIs under tf.keras.dtensor namespace. The APIs are still classified as experimental. You are welcome to try it out. Please check the tutoral and guide on https://www.tensorflow.org/ for more details about DTensor.

  • tf.lite:

    • Added TFLite builtin op support for the following TF ops:
      • tf.math.argmin/tf.math.argmax for input data type tf.bool on CPU.
      • tf.nn.gelu op for output data type tf.float32 and quantization on CPU.
    • Add nominal support for unsigned 16-bit integer tensor types. Note that very few TFLite kernels support this type natively, so its use in mobile ML authoring is generally discouraged.
    • Add support for unsigned 16-bit integer tensor types in cast op.
    • Experimental support for lowering list_ops.tensor_list_set_item with DynamicUpdateSlice.
    • Enabled a new MLIR-based dynamic range quantization backend by default
      • The new backend is used for post-training int8 dynamic range quantization and post-training float16 quantization.
      • Set experimental_new_dynamic_range_quantizer in tf.lite.TFLiteConverter to False to disable this change
    • Native TF Lite variables are now enabled during conversion by default on all v2 TfLiteConverter entry points. experimental_enable_resource_variables on tf.lite.TFLiteConverter is now True by default and will be removed in the future.

  • tf.function:

    • Custom classes used as arguments for tf.function can now specify rules regarding when retracing needs to occur by implementing the Tracing Protocol available through tf.types.experimental.SupportsTracingProtocol.
    • TypeSpec classes (as associated with ExtensionTypes) also implement the Tracing Protocol which can be overriden if necessary.
    • The newly introduced reduce_retracing option also uses the Tracing Protocol to proactively generate generalized traces similar to experimental_relax_shapes (which has now been deprecated).

  • Unified eager and tf.function execution:

    • Eager mode can now execute each op as a tf.function, allowing for more consistent feature support in future releases.
    • It is available for immediate use.
      • See the TF_RUN_EAGER_OP_AS_FUNCTION environment variable in eager context.
      • Eager performance should be similar with this feature enabled.
        • A roughly 5us per-op overhead may be observed when running many small functions.
        • Note a known issue with GPU performance.
      • The behavior of tf.function itself is unaffected.
    • Note: This feature will be enabled by default in an upcoming version of TensorFlow.

  • `tf....

Read more
Assets 2