Releases: keras-team/keras
Keras 2.0.9
Areas of improvement
- RNN improvements:
- Refactor RNN layers to rely on atomic RNN cells. This makes the creation of custom RNN very simple and user-friendly, via the
RNN
base class. - Add ability to create new RNN cells by stacking a list of cells, allowing for efficient stacked RNNs.
- Add
CuDNNLSTM
andCuDNNGRU
layers, backend by NVIDIA's cuDNN library for fast GPU training & inference. - Add RNN Sequence-to-sequence example script.
- Add
constants
argument inRNN
'scall
method, making RNN attention easier to implement.
- Refactor RNN layers to rely on atomic RNN cells. This makes the creation of custom RNN very simple and user-friendly, via the
- Easier multi-GPU data parallelism via
keras.utils.multi_gpu_model
. - Bug fixes & performance improvements (in particular, native support for NCHW data layout in TensorFlow).
- Documentation improvements and examples improvements.
API changes
- Add "fashion mnist" dataset as
keras.datasets.fashion_mnist.load_data()
- Add
Minimum
merge layer askeras.layers.Minimum
(class) andkeras.layers.minimum(inputs)
(function) - Add
InceptionResNetV2
tokeras.applications
. - Support
bool
variables in TensorFlow backend. - Add
dilation
toSeparableConv2D
. - Add support for dynamic
noise_shape
inDropout
- Add
keras.layers.RNN()
base class for batch-level RNNs (used to implement custom RNN layers from a cell class). - Add
keras.layers.StackedRNNCells()
layer wrapper, used to stack a list of RNN cells into a single cell. - Add
CuDNNLSTM
andCuDNNGRU
layers. - Deprecate
implementation=0
for RNN layers. - The Keras progbar now reports time taken for each past epoch, and average time per step.
- Add option to specific resampling method in
keras.preprocessing.image.load_img()
. - Add
keras.utils.multi_gpu_model
for easy multi-GPU data parallelism. - Add
constants
argument inRNN
'scall
method, used to pass a list of constant tensors to the underlying RNN cell.
Breaking changes
- Implementation change in
keras.losses.cosine_proximity
results in a different (correct) scaling behavior. - Implementation change for samplewise normalization in
ImageDataGenerator
results in a different normalization behavior.
Credits
Thanks to our 59 contributors whose commits are featured in this release!
@alok, @Danielhiversen, @Dref360, @HelgeS, @JakeBecker, @MPiecuch, @MartinXPN, @RitwikGupta, @TimZaman, @adammenges, @aeftimia, @ahojnnes, @akshaychawla, @alanyee, @aldenks, @andhus, @apbard, @aronj, @bangbangbear, @bchu, @bdwyer2, @bzamecnik, @cclauss, @colllin, @datumbox, @deltheil, @dhaval067, @durana, @ericwu09, @facaiy, @farizrahman4u, @fchollet, @flomlo, @fran6co, @grzesir, @hgaiser, @icyblade, @jsaporta, @julienr, @jussihuotari, @kashif, @lucashu1, @mangerlahn, @myutwo150, @nicolewhite, @noahstier, @nzw0301, @olalonde, @ozabluda, @patrikerdes, @podhrmic, @qin, @raelg, @roatienza, @shadiakiki1986, @smgt, @souptc, @taehoonlee, @y0z
Keras 2.0.8
The primary purpose of this release is to address an incompatibility between Keras 2.0.7 and the next version of TensorFlow (1.4). TensorFlow 1.4 isn't due until a while, but the sooner the PyPI release has the fix, the fewer people will be affected when upgrading to the next TensorFlow version when it gets released.
No API changes for this release. A few bug fixes.
Keras 2.0.7
Areas of improvement
- Bug fixes.
- Performance improvements.
- Documentation improvements.
- Better support for training models from data tensors in TensorFlow (e.g. Datasets, TFRecords). Add a related example script.
- Improve TensorBoard UX with better grouping of ops into name scopes.
- Improve test coverage.
API changes
- Add
clone_model
method, enabling to construct a new model, given an existing model to use as a template. Works even in a TensorFlow graph different from that of the original model. - Add
target_tensors
argument incompile
, enabling to use custom tensors or placeholders as model targets. - Add
steps_per_epoch
argument infit
, enabling to train a model from data tensors in a way that is consistent with training from Numpy arrays. - Similarly, add
steps
argument inpredict
andevaluate
. - Add
Subtract
merge layer, and associated layer functionsubtract
. - Add
weighted_metrics
argument incompile
to specify metric functions meant to take into accountsample_weight
orclass_weight
. - Make the
stop_gradients
backend function consistent across backends. - Allow dynamic shapes in
repeat_elements
backend function. - Enable stateful RNNs with CNTK.
Breaking changes
- The backend methods
categorical_crossentropy
,sparse_categorical_crossentropy
,binary_crossentropy
had the order of their positional arguments (y_true
,y_pred
) inverted. This change does not affect thelosses
API. This change was done to achieve API consistency between thelosses
API and the backend API. - Move constraint management to be based on variable attributes. Remove the now-unused
constraints
attribute on layers and models (not expected to affect any user).
Credits
Thanks to our 47 contributors whose commits are featured in this release!
@5ke, @alok, @Danielhiversen, @Dref360, @NeilRon, @abnerA, @acburigo, @airalcorn2, @angeloskath, @athundt, @brettkoonce, @cclauss, @denfromufa, @enkait, @erg, @ericwu09, @farizrahman4u, @fchollet, @georgwiese, @ghisvail, @gokceneraslan, @hgaiser, @inexxt, @joeyearsley, @jorgecarleitao, @kennyjacob, @keunwoochoi, @krizp, @lukedeo, @milani, @n17r4m, @nicolewhite, @nigeljyng, @nyghtowl, @nzw0301, @rapatel0, @souptc, @srinivasreddy, @staticfloat, @taehoonlee, @td2014, @titu1994, @tleeuwenburg, @udibr, @waleedka, @wassname, @yashk2810
Keras 2.0.6
Areas of improvement
- Improve generator methods (
predict_generator
,fit_generator
,evaluate_generator
) and add data enqueuing utilities. - Bug fixes and performance improvements.
- New features: new
Conv3DTranspose
layer, newMobileNet
application, self-normalizing networks.
API changes
- Self-normalizing networks: add
selu
activation function,AlphaDropout
layer,lecun_normal
initializer. - Data enqueuing: add
Sequence
,SequenceEnqueuer
,GeneratorEnqueuer
toutils
. - Generator methods: rename arguments
pickle_safe
(replaced withuse_multiprocessing
) andmax_q_size
(replaced withmax_queue_size
). - Add
MobileNet
to the applications module. - Add
Conv3DTranspose
layer. - Allow custom print functions for model's
summary
method (argumentprint_fn
).
Keras 2.0.5
- Add beta CNTK backend.
- TensorBoard improvements.
- Documentation improvements.
- Bug fixes and performance improvements.
- Improve style transfer example script.
API changes:
- Add
return_state
constructor argument to RNNs. - Add
skip_compile
option toload_model
. - Add
categorical_hinge
loss function. - Add
sparse_top_k_categorical_accuracy
metric. - Add new options to
TensorBoard
callback. - Add
TerminateOnNaN
callback. - Generalize the
Embedding
layer to N (>=2) input dimensions.
Keras 2.0.4
- Documentation improvements.
- Docstring improvements.
- Update some examples scripts (in particular, new deep dream example).
- Bug fixes and performance improvements.
API changes:
- Add
logsumexp
andidentity
to backend. - Add
logcosh
loss. - New signature for
add_weight
inLayer
. get_initial_states
inRecurrent
is nowget_initial_state
.
2.0.0
Keras 2 release notes
This document details changes, in particular API changes, occurring from Keras 1 to Keras 2.
Training
- The
nb_epoch
argument has been renamedepochs
everywhere. - The methods
fit_generator
,evaluate_generator
andpredict_generator
now work by drawing a number of batches from a generator (number of training steps), rather than a number of samples.samples_per_epoch
was renamedsteps_per_epoch
infit_generator
.nb_val_samples
was renamedvalidation_steps
infit_generator
.val_samples
was renamedsteps
inevaluate_generator
andpredict_generator
.
- It is now possible to manually add a loss to a model by calling
model.add_loss(loss_tensor)
. The loss is added to the other losses of the model and minimized during training. - It is also possible to not apply any loss to a specific model output. If you pass
None
as theloss
argument for an output (e.g. in compile,loss={'output_1': None, 'output_2': 'mse'}
, the model will expect no Numpy arrays to be fed for this output when usingfit
,train_on_batch
, orfit_generator
. The output values are still returned as usual when usingpredict
. - In TensorFlow, models can now be trained using
fit
if some of their inputs (or even all) are TensorFlow queues or variables, rather than placeholders. See this test for specific examples.
Losses & metrics
- The
objectives
module has been renamedlosses
. - Several legacy metric functions have been removed, namely
matthews_correlation
,precision
,recall
,fbeta_score
,fmeasure
. - Custom metric functions can no longer return a dict, they must return a single tensor.
Models
- Constructor arguments for
Model
have been renamed:input
->inputs
output
->outputs
- The
Sequential
model not longer supports theset_input
method. - For any model saved with Keras 2.0 or higher, weights trained with backend X will be converted to work with backend Y without any manual conversion step.
Layers
Removals
Deprecated layers MaxoutDense
, Highway
and TimedistributedDense
have been removed.
Call method
- All layers that use the learning phase now support a
training
argument incall
(Python boolean or symbolic tensor), allowing to specify the learning phase on a layer-by-layer basis. E.g. by calling aDropout
instance asdropout(inputs, training=True)
you obtain a layer that will always apply dropout, regardless of the current global learning phase. Thetraining
argument defaults to the global Keras learning phase everywhere. - The
call
method of layers can now take arbitrary keyword arguments, e.g. you can define a custom layer with a call signature likecall(inputs, alpha=0.5)
, and then pass aalpha
keyword argument when calling the layer (only with the functional API, naturally). __call__
now makes use of TensorFlowname_scope
, so that your TensorFlow graphs will look pretty and well-structured in TensorBoard.
All layers taking a legacy dim_ordering
argument
dim_ordering
has been renamed data_format
. It now takes two values: "channels_first"
(formerly "th"
) and "channels_last"
(formerly "tf"
).
Dense layer
Changed interface:
output_dim
->units
init
->kernel_initializer
- added
bias_initializer
argument W_regularizer
->kernel_regularizer
b_regularizer
->bias_regularizer
b_constraint
->bias_constraint
bias
->use_bias
Dropout, SpatialDropout*D, GaussianDropout
Changed interface:
p
->rate
Embedding
Convolutional layers
- The
AtrousConvolution1D
andAtrousConvolution2D
layer have been deprecated. Their functionality is instead supported via thedilation_rate
argument inConvolution1D
andConvolution2D
layers. Convolution*
layers are renamedConv*
.- The
Deconvolution2D
layer is renamedConv2DTranspose
. - The
Conv2DTranspose
layer no longer requires anoutput_shape
argument, making its use much easier.
Interface changes common to all convolutional layers:
nb_filter
->filters
- float kernel dimension arguments become a single tuple argument,
kernel
size. E.g. a legacy callConv2D(10, 3, 3)
becomesConv2D(10, (3, 3))
kernel_size
can be set to an integer instead of a tuple, e.g.Conv2D(10, 3)
is equivalent toConv2D(10, (3, 3))
.subsample
->strides
. Can also be set to an integer.border_mode
->padding
init
->kernel_initializer
- added
bias_initializer
argument W_regularizer
->kernel_regularizer
b_regularizer
->bias_regularizer
b_constraint
->bias_constraint
bias
->use_bias
dim_ordering
->data_format
- In the
SeparableConv2D
layers,init
is split intodepthwise_initializer
andpointwise_initializer
. - Added
dilation_rate
argument inConv2D
andConv1D
. - 1D convolution kernels are now saved as a 3D tensor (instead of 4D as before).
- 2D and 3D convolution kernels are now saved in format
spatial_dims + (input_depth, depth))
, even withdata_format="channels_first"
.
Pooling1D
pool_length
->pool_size
stride
->strides
border_mode
->padding
Pooling2D, 3D
border_mode
->padding
dim_ordering
->data_format
ZeroPadding layers
The padding
argument of the ZeroPadding2D
and ZeroPadding3D
layers must be a tuple of length 2 and 3 respectively. Each entry i
contains by how much to pad the spatial dimension i
. If it's an integer, symmetric padding is applied. If it's a tuple of integers, asymmetric padding is applied.
Upsampling1D
length
->size
BatchNormalization
The mode
argument of BatchNormalization
has been removed; BatchNorm now only supports mode 0 (use batch metrics for feature-wise normalization during training, and use moving metrics for feature-wise normalization during testing).
beta_init
->beta_initializer
gamma_init
->gamma_initializer
- added arguments
center
,scale
(booleans, whether to use abeta
andgamma
respectively) - added arguments
moving_mean_initializer
,moving_variance_initializer
- added arguments
beta_regularizer
,gamma_regularizer
- added arguments
beta_constraint
,gamma_constraint
- attribute
running_mean
is renamedmoving_mean
- attribute
running_std
is renamedmoving_variance
(it is in fact a variance with the current implementation).
ConvLSTM2D
Same changes as for convolutional layers and recurrent layers apply.
PReLU
init
->alpha_initializer
GaussianNoise
sigma
->stddev
Recurrent layers
output_dim
->units
init
->kernel_initializer
inner_init
->recurrent_initializer
- added argument
bias_initializer
W_regularizer
->kernel_regularizer
b_regularizer
->bias_regularizer
- added arguments
kernel_constraint
,recurrent_constraint
,bias_constraint
dropout_W
->dropout
dropout_U
->recurrent_dropout
consume_less
->implementation
. String values have been replaced with integers: implementation 0 (default), 1 or 2.- LSTM only: the argument
forget_bias_init
has been removed. Instead there is a boolean argumentunit_forget_bias
, defaulting toTrue
.
Lambda
The Lambda
layer now supports a mask
argument.
Utilities
Utilities should now be imported from keras.utils
rather than from specific submodules (e.g. no more keras.utils.np_utils...
).
Backend
random_normal and truncated_normal
std
->stddev
Misc
- In the backend,
set_image_ordering
andimage_ordering
are nowset_data_format
anddata_format
. - Any arguments (other than
nb_epoch
) prefixed withnb_
has been renamed to be prefixed withnum_
instead. This affects two datasets and one preprocessing utility.