5.2.5 Bugfix release

• Fixes XML serialization of segmentation results (#597)
• Removes regression in polygonization code introduced with performance enhancements (#605)
• extract_polygons() now raises an exception when processing baselines < 5px in length (#606)
• Various small improvements to contrib/segmentation_overlay.py
• ketos compile progress bar now displays elapsed/remaining time (#504)

Hotfix release

• Fixes a regression in container-based binary dataset building
• Fixes spurious updates of validation metrics after sanity checking

Hotfix for segmentation training

What's Changed

• Hotfix for segmentation training

Hotfix for no_segmentation mode recognition

Hotfix release fixing a regression in no_segmentation recognition.

5.2.1 hotfix release

This release contains two small fixes for a regression related to bumping lightning up to 2.2 and a crash in Segmentation instantiation occurring when the first region type does not contain a region/dict.

5.0 release with minor bugfixes

Kraken 5.x is a major release introducing trainable reading order, a cleaner API, and changes resulting in a ~50% performance improvement of recognition inference, in addition to a large number of smaller bug fixes and stability improvements.

What's Changed

• Trainable reading order based on an neural order relation operator adapted from this method (#492)
• Updates to the ALTO/PageXML templates and the serializer which correct serialization of region and line taxonomies, use UUIDs, and reuse identifiers from input XML files in output.
• Requirements are now mostly pinned to avoid pytorch/lightning accuracy and speed regressions that popped up semi-regularly with more free package versions.
• Threadpool limits are now set in all CLI drivers to prevent slowdown from unreasonably large numbers of threads in libraries like OpenCV. As a result the --threads option of all commands has been split into --workers and --threads.
• kraken.repo methods have been adapted to the new Zenodo API. They also correctly handle versioned records now.
• A small fix enabling recognition inference with AMP.
• Support for --fixed-splits in ketos test (@PonteIneptique)
• Performance increase for polygon extraction by @Evarin in #555
• Speed up legacy polygon extraction by @anutkk in #586
• New container classes in kraken.containers replace the previous dicts produced and expected by segment/rpred/serialize.
• kraken.serialize.serialize_segmentation() has been removed as part of the container class rework.
• train/rotrain/segtrain/pretrain cosine annealing scheduling now allows setting the final learning rate with --cos-min-lr.
• Lots of PEP8/whitespace/spelling mistake fixes from @stweil

New features

Reading order training

Reading order can now be learned with ketos rotrain and reading order models can be added to segmentation model files. The training process is documented here.

Upgrade guide

Command line

Polygon extractor

The polygon extractor is responsible for taking a page image, baselines, and their bounding polygons and dewarping + masking out the line. Here is an example:

The new polygon extractor reduces line extraction time 30x, roughly halving inference time and significantly speeding up training from XML files and compilation of datasets. It should be noted that polygon extraction does not concern data in the legacy bounding box format nor does it touch the segmentation process as it is only a preprocessing step in the recognizer on an already existing segmentation.

Not all improvements in the polygon extractor are backward compatible, causing models trained with data extracted with the old implementation to suffer from a slight reduction in accuracy (usually <0.25 percentage points). Therefore models now contain a flag in their metadata indicating which implementation has been used to train them. This flag can be overridden, e.g.:

$ kraken --no-legacy-polygons -i ... ... ocr ...

to enable all speedups for a slight increase in character error rate.

For training the new extractor is enabled per default, i.e. models trained with kraken 5.x will perform slightly worse on earlier kraken version but will still work. It is possible to force use of only backwards compatible speedups:

$ ketos compile --legacy-polygons ...
$ ketos train --legacy-polygons ....
$ ketos pretrain --legacy-polygons ...

Threads and Multiprocessing

The command line tools now handle multiprocessing and thread pools more completely and configurably. --workers has been split into --threads and --workers, the former option limiting the size of thread pools (as much as possible) for intra-op parallelization, the latter setting the number of worker processes, usually for the purpose of data loading in training and dataset compilation.

API changes

While 5.x preserves the general OCR functional blocks, the existing dictionary-based data structures have been replaced with container classes and the XML parser has been reworked.

Container classes

For straightforward processing little has changed. Most keys of the dictionaries have been converted into attributes of their respective classes.

The segmentation methods now return a Segmentation object containing Region and BaselineLine/BBoxLine objects:

>>> pageseg.segment(im)
{'text_direction': 'horizontal-lr',
 'boxes': [(x1, y1, x2, y2),...],
 'script_detection': False
}

>>> blla.segment(im)
{'text_direction': '$dir',
 'type': 'baseline',
 'lines': [{'baseline': [[x0, y0], [x1, y1], ..., [x_n, y_n]], 'boundary': [[x0, y0, x1, y1], ... [x_m, y_m]]}, ...
          {'baseline': [[x0, ...]], 'boundary': [[x0, ...]]}]
 'regions': [{'region': [[x0, y0], [x1, y1], ..., [x_n, y_n]], 'type': 'image'}, ...
             {'region': [[x0, ...]], 'type': 'text'}]
}

becomes:

>>> pageseg.segment(im)
Segmentation(type='bbox', 
             imagename=None,
             text_direction='horizontal-lr',
             script_detection=False,
             lines=[BBoxLine(id='f1d5b1e2-030c-41d5-b299-8a114eb0996e',
                             bbox=[34, 198, 279, 251],
                             text=None,
                             base_dir=None,
                             type='bbox',
                             imagename=None,
                             tags=None,
                             split=None,
                             regions=None,
                             text_direction='horizontal-lr'),
                    BBoxLine(...],
             line_orders=[])

>>> blla.segment(im)
Segmentation(type='baseline', 
             imagename=im,
             text_direction='horizontal-lr',
             script_detection=False,
             lines=[BaselineLine(id='50ab1a29-c3b6-4659-9713-ff246b21d2dc',
                                 baseline=[[183, 284], [272, 282]],
                                 boundary=[[183, 284], ... ,[183, 284]],
                                 text=None,
                                 base_dir=None,
                                 type='baselines',
                                 tags={'type': 'default'},
                                 split=None,
                                 regions=['e28ccb6b-2874-4be0-8e0d-38948f0fdf09']), ...],
             regions={'text': [Region(id='e28ccb6b-2874-4be0-8e0d-38948f0fdf09',
                                      boundary=[[123, 218], ..., [123, 218]],
                                      tags={'type': 'text'}), ...],
                               'foo': [Region(...), ...]},
             line_orders=[])

The recognizer now yields
BaselineOCRRecords/BBoxOCRRecords
which both inherit from the BaselineLine/BBoxLine classes:

>>> record = rpred(network=model,
                   im=im,
                   segmentation=baseline_seg)
>>> record = next(rpred.rpred(im))
>>> record
BaselineOCRRecord pred: 'predicted text' baseline: ...
>>> record.type
'baselines'
>>> record.line
BaselineLine(...)
>>> record.prediction
'predicted text'

One complication is the new serialization function which now accepts a
Segmentation object instead of a list of ocr_records and ancillary metadata:

>>> records = list(x for x in rpred(...))
>>> serialize(records,
              image_name=im.filename,
              image_size=im.size,
              writing_mode='horizontal-tb',
              scripts=['Latn', 'Hebr'],
              regions=[{...}],
              template='alto',
              template_source='native',
              processing_steps=proc_steps)

becomes:

>>> import dataclasses
>>> baseline_seg
Segmentation(...)
>>> records = list(x for x in rpred(..., segmentation=baseline_seg))
>>> results = dataclasses.replace(baseline_seg, lines=records)
>>> serialize(results,
              image_size=im.size,
              writing_mode='horizontal-tb',
              scripts=['Latn', 'Hebr'],
              template='alto',
              template_source='native',
              processing_steps=proc_steps)

This requires the construction of a new Segmentation object that contains the
records produced by the text predictor. The most straightforward way to create
this new Segmentation is through the dataclasses.replace function as our
container classes are immutable.

Lastly, serialize_segmentation has been removed. The serialize function now
accepts Segmentation objects which do not contain text predictions:

>>> serialize_segmentation(segresult={'text_direction': '$dir',
                                      'type': 'baseline',
                                      'lines': [{'baseline': [[x0, y0], [x1, y1], ..., [x_n, y_n]], 'boundary': [[x0, y0, x1, y1], ... [x_m, y_m]]}, ...
                                          {'baseline': [[x0, ...]], 'boundary': [[x0, ...]]}]
              ...

4.3.10

This is mostly a bugfix release but also includes a couple of minor improvements and changes.

Changes

• Deterministic mode is now set to 'warn' preventing crashes in deterministic recognition training (CTC loss does not have a deterministic implementation).
• contrib/extract_lines.py work with binary datasets
• 'Word' error rate has been added as a validation metric in recognition training
• The fine-tuning options (--resize) add/both have been renamed to union/new. (Thibault Clérice) #488
• Tensorboard logging now also logs a couple of training images

4.3.5

This is just another hotfix release.

Changes

• 799ee78: Propagation of the --raise-on-error for raising non-blocking errors in blla segmentation (Thibault Clérice) #444
• d81e898: adds pl_logger to default hyperparams dict (Benjamin Kiessling)

4.3.4

This is a hotfix release to 4.3.0 correcting a regression in the CLI, fixing pretrain validation losses, and the conda environment files.

Commits

• ac5fab6: Invalid type in click option definition for loggers (Benjamin Kiessling)
• 0cb9e0e: fix validation loss computation in pretrain (Benjamin Kiessling)
• 7d5069b: Remove former development raise in segmentation (Thibault Clérice) #441
• 0e3d10f: Install coremltools from pip for conda environments (Benjamin Kiessling)

4.3.0

What's Changed

• Pretraining has been reimplemented to be more faithful to the original publication for more stable memory consumption and easier hyperparameter selection
• Learning rate warmup and backbone freezing in recognition training with --warmup and --freeze-backbone (mostly to enable fine-tuning pretrained models)
• Enable ketos compile to create precompiled datasets with lines without a corresponding transcription with the --keep-empty-lines switch (mostly for pretraining models).
• --failed-sample-threshold in training modules, aborting training after a certain number of samples failed to load
• tensorboard logging with --logger/--log-dir options
• Change codec construction during training when training and validation dataset alphabets don't match. Prior code points that only exist in the validation set would be copied to the model codec. Now the model codec only contains trained code points.
• Replace ocr_record with new smart classes BaselineOCRRecord and BBoxOCRRecord. These keep track of reading/display order, compute bounding polygons from the whole line bounding polygon, and average confidences when slicing.
• ALTO parsing now deals with any reasonable PointsType (see altoxml/schema#49)
• The fallback line orientation heuristic now takes into account the principal text orientation defined with --text-direction instead of assuming horizontal lines (--text-direction horizontal-lr/-rl).
• Baseline segmentation now supports padding of input images with --pad.
• CLI now allows serialization with custom jinja2 templates through the --template option.
• Switch validation metrics computation to torchmetrics.
• Various bugfixes, mostly to deal with shapely shenanigans.

Thanks

• @sixtyfive, @anutkk, @stweil, @colibrisson, @PonteIneptique for their contributions to this release.

Full Changelog: 4.2.0...4.3.0