Releases: keras-team/keras
Keras 2.2.2
This is a bugfix release, fixing a significant bug in multi_gpu_model
.
For changes since version 2.2.0, see release notes for Keras 2.2.1.
Keras 2.2.1
Areas of improvement
- Bugs fixes
- Performance improvements
- Documentation improvements
API changes
- Add
output_padding
argument inConv2DTranspose
(to override default padding behavior). - Enable automatic shape inference when using Lambda layers with the CNTK backend.
Breaking changes
No breaking changes recorded.
Credits
Thanks to our 33 contributors whose commits are featured in this release:
@Ajk4, @Anner-deJong, @Atcold, @Dref360, @EyeBool, @ageron, @briannemsick, @cclauss, @davidtvs, @dstine, @eTomate, @ebatuhankaynak, @eliberis, @farizrahman4u, @fchollet, @fuzzythecat, @gabrieldemarmiesse, @jlopezpena, @kamil-kaczmarek, @kbattocchi, @kmader, @kvechera, @maxpumperla, @mkaze, @pavithrasv, @rvinas, @sachinruk, @seriousmac, @soumyac1999, @taehoonlee, @yanboliang, @yongzx, @yuyang-huang
Keras 2.2.0
Areas of improvements
- New model definition API:
Model
subclassing. - New input mode: ability to call models on TensorFlow tensors directly (TensorFlow backend only).
- Improve feature coverage of Keras with the Theano and CNTK backends.
- Bug fixes and performance improvements.
- Large refactors improving code structure, code health, and reducing test time. In particular:
- The Keras engine now follows a much more modular structure.
- The
Sequential
model is now a plain subclass ofModel
. - The modules
applications
andpreprocessing
are now externalized to their own repositories (keras-applications and keras-preprocessing).
API changes
- Add
Model
subclassing API (details below). - Allow symbolic tensors to be fed to models, with TensorFlow backend (details below).
- Enable CNTK and Theano support for layers
SeparableConv1D
,SeparableConv2D
, as well as backend methodsseparable_conv1d
andseparable_conv2d
(previously only available for TensorFlow). - Enable CNTK and Theano support for applications
Xception
andMobileNet
(previously only available for TensorFlow). - Add
MobileNetV2
application (available for all backends). - Enable loading external (non built-in) backends by changing your
~/.keras.json
configuration file (e.g. PlaidML backend). - Add
sample_weight
inImageDataGenerator
. - Add
preprocessing.image.save_img
utility to write images to disk. - Default
Flatten
layer'sdata_format
argument toNone
(which defaults to global Keras config). Sequential
is now a plain subclass ofModel
. The attributesequential.model
is deprecated.- Add
baseline
argument inEarlyStopping
(stop training if a given baseline isn't reached). - Add
data_format
argument toConv1D
. - Make the model returned by
multi_gpu_model
serializable. - Support input masking in
TimeDistributed
layer. - Add an
advanced_activation
layerReLU
, making the ReLU activation easier to configure while retaining easy serialization capabilities. - Add
axis=-1
argument in backend crossentropy functions specifying the class prediction axis in the input tensor.
New model definition API : Model
subclassing
In addition to the Sequential
API and the functional Model
API, you may now define models by subclassing the Model
class and writing your own call
forward pass:
import keras
class SimpleMLP(keras.Model):
def __init__(self, use_bn=False, use_dp=False, num_classes=10):
super(SimpleMLP, self).__init__(name='mlp')
self.use_bn = use_bn
self.use_dp = use_dp
self.num_classes = num_classes
self.dense1 = keras.layers.Dense(32, activation='relu')
self.dense2 = keras.layers.Dense(num_classes, activation='softmax')
if self.use_dp:
self.dp = keras.layers.Dropout(0.5)
if self.use_bn:
self.bn = keras.layers.BatchNormalization(axis=-1)
def call(self, inputs):
x = self.dense1(inputs)
if self.use_dp:
x = self.dp(x)
if self.use_bn:
x = self.bn(x)
return self.dense2(x)
model = SimpleMLP()
model.compile(...)
model.fit(...)
Layers are defined in __init__(self, ...)
, and the forward pass is specified in call(self, inputs)
. In call
, you may specify custom losses by calling self.add_loss(loss_tensor)
(like you would in a custom layer).
New input mode: symbolic TensorFlow tensors
With Keras 2.2.0 and TensorFlow 1.8 or higher, you may fit
, evaluate
and predict
using symbolic TensorFlow tensors (that are expected to yield data indefinitely). The API is similar to the one in use in fit_generator
and other generator methods:
iterator = training_dataset.make_one_shot_iterator()
x, y = iterator.get_next()
model.fit(x, y, steps_per_epoch=100, epochs=10)
iterator = validation_dataset.make_one_shot_iterator()
x, y = iterator.get_next()
model.evaluate(x, y, steps=50)
This is achieved by dynamically rewiring the TensorFlow graph to feed the input tensors to the existing model placeholders. There is no performance loss compared to building your model on top of the input tensors in the first place.
Breaking changes
- Remove legacy
Merge
layers and associated functionality (remnant of Keras 0), which were deprecated in May 2016, with full removal initially scheduled for August 2017. Models from the Keras 0 API using these layers cannot be loaded with Keras 2.2.0 and above. - The
truncated_normal
base initializer now returns values that are scaled by ~0.9 (resulting in correct variance value after truncation). This has a small chance of affecting initial convergence behavior on some models.
Credits
Thanks to our 46 contributors whose commits are featured in this release:
@ASvyatkovskiy, @AmirAlavi, @Anirudh-Swaminathan, @davidariel, @Dref360, @JonathanCMitchell, @KuzMenachem, @PeterChe1990, @Saharkakavand, @StefanoCappellini, @ageron, @askskro, @bileschi, @bonlime, @bottydim, @brge17, @briannemsick, @bzamecnik, @christian-lanius, @clemens-tolboom, @dschwertfeger, @dynamicwebpaige, @farizrahman4u, @fchollet, @fuzzythecat, @ghostplant, @giuscri, @huyu398, @jnphilipp, @masstomato, @morenoh149, @mrTsjolder, @nittanycolonial, @r-kellerm, @reidjohnson, @roatienza, @Sbebo, @stevemurr, @taehoonlee, @tiferet, @tkoivisto, @tzerrell, @vkk800, @wangkechn, @wouterdobbels, @zwang36wang
Keras 2.1.6
Areas of improvement
- Bug fixes
- Documentation improvements
- Minor usability improvements
API changes
- In callback
ReduceLROnPlateau
, renameepsilon
argument tomin_delta
(backwards-compatible). - In callback
RemoteMonitor
, add argumentsend_as_json
. - In backend
softmax
function, add argumentaxis
. - In
Flatten
layer, add argumentdata_format
. - In
save_model
(Model.save
) andload_model
functions, allow thefilepath
argument to be ah5py.File
object. - In
Model.evaluate_generator
, addverbose
argument. - In
Bidirectional
wrapper layer, addconstants
argument. - In
multi_gpu_model
function, add argumentscpu_merge
andcpu_relocation
(controlling whether to force the template model's weights to be on CPU, and whether to operate merge operations on CPU or GPU). - In
ImageDataGenerator
, allow argumentwidth_shift_range
to beint
or 1D array-like.
Breaking changes
This release does not include any known breaking changes.
Credits
Thanks to our 37 contributors whose commits are featured in this release:
@Dref360, @FirefoxMetzger, @Naereen, @NiharG15, @StefanoCappellini, @WindQAQ, @dmadeka, @edrogers, @eltronix, @farizrahman4u, @fchollet, @gabrieldemarmiesse, @ghostplant, @jedrekfulara, @jlherren, @joeyearsley, @johanahlqvist, @johnyf, @jsaporta, @kalkun, @lucasdavid, @masstomato, @mrlzla, @myutwo150, @nisargjhaveri, @obi1kenobi, @olegantonyan, @ozabluda, @pasky, @Planck35, @sotlampr, @souptc, @srjoglekar246, @stamate, @taehoonlee, @vkk800, @xuhdev
Keras 2.1.5
Areas of improvement
- Bug fixes.
- New APIs: sequence generation API
TimeseriesGenerator
, and new layerDepthwiseConv2D
. - Unit tests / CI improvements.
- Documentation improvements.
API changes
- Add new sequence generation API
keras.preprocessing.sequence.TimeseriesGenerator
. - Add new convolutional layer
keras.layers.DepthwiseConv2D
. - Allow weights from
keras.layers.CuDNNLSTM
to be loaded into akeras.layers.LSTM
layer (e.g. for inference on CPU). - Add
brightness_range
data augmentation argument inkeras.preprocessing.image.ImageDataGenerator
. - Add
validation_split
API inkeras.preprocessing.image.ImageDataGenerator
. You can passvalidation_split
to the constructor (float), then select between training/validation subsets by passing the argumentsubset='validation'
orsubset='training'
to methodsflow
andflow_from_directory
.
Breaking changes
- As a side effect of a refactor of
ConvLSTM2D
to a modular implementation, recurrent dropout support in Theano has been dropped for this layer.
Credits
Thanks to our 28 contributors whose commits are featured in this release:
@DomHudson, @Dref360, @VitamintK, @abrad1212, @ahundt, @bojone, @brainnoise, @bzamecnik, @caisq, @cbensimon, @davinnovation, @farizrahman4u, @fchollet, @gabrieldemarmiesse, @khosravipasha, @ksindi, @lenjoy, @masstomato, @mewwts, @ozabluda, @paulpister, @sandpiturtle, @saralajew, @srjoglekar246, @stefangeneralao, @taehoonlee, @tiangolo, @treszkai
Keras 2.1.4
Areas of improvement
- Bug fixes
- Performance improvements
- Improvements to example scripts
API changes
- Allow for stateful metrics in
model.compile(..., metrics=[...])
. A stateful metric inherits fromLayer
, and implements__call__
andreset_states
. - Support
constants
argument inStackedRNNCells
. - Enable some TensorBoard features in the
TensorBoard
callback (loss and metrics plotting) with non-TensorFlow backends. - Add
reshape
argument inmodel.load_weights()
, to optionally reshape weights being loaded to the size of the target weights in the model considered. - Add
tif
to supported formats inImageDataGenerator
. - Allow auto-GPU selection in
multi_gpu_model()
(setgpus=None
). - In
LearningRateScheduler
callback, the scheduling function now takes an argument:lr
, the current learning rate.
Breaking changes
- In
ImageDataGenerator
, change default interpolation of image transforms from nearest to bilinear. This should probably not break any users, but it is a change of behavior.
Credits
Thanks to our 37 contributors whose commits are featured in this release:
@DalilaSal, @Dref360, @galaxydream, @GarrisonJ, @Max-Pol, @May4m, @MiliasV, @MrMYHuang, @N-McA, @Vijayabhaskar96, @abrad1212, @ahundt, @angeloskath, @bbabenko, @bojone, @brainnoise, @bzamecnik, @caisq, @cclauss, @dsadulla, @fchollet, @gabrieldemarmiesse, @ghostplant, @gorogoroyasu, @icyblade, @kapsl, @kevinbache, @mendesmiguel, @mikesol, @myutwo150, @ozabluda, @sadreamer, @simra, @taehoonlee, @veniversum, @yongtang, @zhangwj618
Keras 2.1.3
Areas of improvement
- Performance improvements (esp. convnets with TensorFlow backend).
- Usability improvements.
- Docs & docstrings improvements.
- New models in the
applications
module. - Bug fixes.
API changes
trainable
attribute inBatchNormalization
now disables the updates of the batch statistics (i.e. iftrainable == False
the layer will now run 100% in inference mode).- Add
amsgrad
argument inAdam
optimizer. - Add new applications:
NASNetMobile
,NASNetLarge
,DenseNet121
,DenseNet169
,DenseNet201
. - Add
Softmax
layer (removing need to use aLambda
layer in order to specify theaxis
argument). - Add
SeparableConv1D
layer. - In
preprocessing.image.ImageDataGenerator
, allowwidth_shift_range
andheight_shift_range
to take integer values (absolute number of pixels) - Support
return_state
inBidirectional
applied to RNNs (return_state
should be set on the child layer). - The string values
"crossentropy"
and"ce"
are now allowed in themetrics
argument (inmodel.compile()
), and are routed to eithercategorical_crossentropy
orbinary_crossentropy
as needed. - Allow
steps
argument inpredict_*
methods on theSequential
model. - Add
oov_token
argument inpreprocessing.text.Tokenizer
.
Breaking changes
- In
preprocessing.image.ImageDataGenerator
,shear_range
has been switched to use degrees rather than radians (for consistency). This should not actually break anything (neither training nor inference), but keep this change in mind in case you see any issues with regard to your image data augmentation process.
Credits
Thanks to our 45 contributors whose commits are featured in this release:
@Dref360, @OliPhilip, @TimZaman, @bbabenko, @bdwyer2, @berkatmaca, @caisq, @decrispell, @dmaniry, @fchollet, @fgaim, @gabrieldemarmiesse, @gklambauer, @hgaiser, @hlnull, @icyblade, @jgrnt, @kashif, @kouml, @lutzroeder, @m-mohsen, @mab4058, @manashty, @masstomato, @mihirparadkar, @myutwo150, @nickbabcock, @novotnj3, @obsproth, @ozabluda, @philferriere, @piperchester, @pstjohn, @roatienza, @souptc, @spiros, @srs70187, @sumitgouthaman, @taehoonlee, @tigerneil, @titu1994, @tobycheese, @vitaly-krumins, @yang-zhang, @ziky90
Keras 2.1.2
Areas of improvement
- Bug fixes and performance improvements.
- API improvements in Keras applications, generator methods.
API changes
- Make
preprocess_input
in all Keras applications compatible with both Numpy arrays and symbolic tensors (previously only supported Numpy arrays). - Allow the
weights
argument in all Keras applications to accept the path to a custom weights file to load (previously only supported the built-inimagenet
weights file). steps_per_epoch
behavior change in generator training/evaluation methods:- If specified, the specified value will be used (previously, in the case of generator of type
Sequence
, the specified value was overridden by theSequence
length) - If unspecified and if the generator passed is a
Sequence
, we set it to theSequence
length.
- If specified, the specified value will be used (previously, in the case of generator of type
- Allow
workers=0
in generator training/evaluation methods (will run the generator in the main process, in a blocking way). - Add
interpolation
argument inImageDataGenerator.flow_from_directory
, allowing a custom interpolation method for image resizing. - Allow
gpus
argument inmulti_gpu_model
to be a list of specific GPU ids.
Breaking changes
- The change in
steps_per_epoch
behavior (described above) may affect some users.
Credits
Thanks to our 26 contributors whose commits are featured in this release:
@Alex1729, @alsrgv, @apisarek, @asos-saul, @athundt, @cherryunix, @dansbecker, @datumbox, @de-vri-es, @drauh, @evhub, @fchollet, @heath730, @hgaiser, @icyblade, @jjallaire, @knaveofdiamonds, @lance6716, @luoch, @mjacquem1, @myutwo150, @ozabluda, @raviksharma, @rh314, @yang-zhang, @zach-nervana
Keras 2.1.1
This release amends release 2.1.0 to include a fix for an erroneous breaking change introduced in #8419.
Keras 2.1.0
This is a small release that fixes outstanding bugs that were reported since the previous release.
Areas of improvement
- Bug fixes (in particular, Keras no longer allocates devices at startup time with the TensorFlow backend. This was causing issues with Horovod.)
- Documentation and docstring improvements.
- Better CIFAR10 ResNet example script and improvements to example scripts code style.
API changes
- Add
go_backwards
to cuDNN RNNs (enablesBidirectional
wrapper on cuDNN RNNs). - Add ability to pass
fetches
toK.Function()
with the TensorFlow backend. - Add
steps_per_epoch
andvalidation_steps
arguments inSequential.fit()
(to sync it withModel.fit()
).
Breaking changes
None.
Credits
Thanks to our 14 contributors whose commits are featured in this release:
@Dref360, @LawnboyMax, @anj-s, @bzamecnik, @datumbox, @diogoff, @farizrahman4u, @fchollet, @frexvahi, @jjallaire, @nsuh, @ozabluda, @roatienza, @yakigac