Skip to content

Releases: openzim/python-scraperlib

3.3.2

25 Mar 09:16
2923655
Compare
Choose a tag to compare

Added

  • Add support for disable_metadata_checks and ignore_duplicates arguments in make_zim_file function ("zimwritefs-mode")

Changed

  • Relaxed constraints on Python dependencies
  • Upgraded optional dependencies used for test and QA

3.3.1

27 Feb 14:21
e31f5ed
Compare
Choose a tag to compare

Added

  • Set a user-agent for handle_user_provided_file #103

Changed

  • Migrate to generic syntax in all std collections #140

Fixed

  • Do not modify the ffmpeg_args in reencode function #144

3.3.0

14 Feb 09:42
04181e9
Compare
Choose a tag to compare

Added

  • New disable_metadata_checks parameter in zimscraperlib.zim.creator.Creator initializer, allowing to disable metadata check at startup (assuming the user will validate them on its own) #119

Changed

  • Rework the VideoWebmLow preset for faster encoding and smaller file size #122
    • preset has been bumped to version 2
    • when using an S3 cache, all videos using this preset will be reencoded and uploaded to cache again (it will replace the same file encoded with preset version 1)
  • When reencoding a video, ffmpeg now uses only 1 CPU thread by default (new arg to reencode allows to override this default value)
  • Using openZIM Python bootstrap conventions (including hatch-openzim plugin) #120
  • Add support for Python 3.12, drop Python 3.7 support #118
  • Replace "iso-369" by "iso639-lang" library
  • Replace "file-magic" by "python-magic" library for Alpine Linux support and better maintenance

Fixed

  • Fixed type hints of zimscraperlib.zim.Item and subclasses, and zimscraperlib.image.optimization:convert_image

3.2.0

16 Dec 18:25
4dc3012
Compare
Choose a tag to compare

Added

  • Add utility function to compute/check ZIM descriptions #110

Changed

  • Using pylibzim 3.4.0

Removed

  • Support for Python 3.7 (EOL)

3.1.1

18 Jul 18:19
4f8c3cc
Compare
Choose a tag to compare

Changed

  • Fixed declared (hint) return type of download.stream_file #104
  • Fixed declared (hint) type of content param for Creator.add_item_for #107

3.1.0

05 May 10:17
Compare
Choose a tag to compare

Changed

  • Using pylibzim 3.1.0
  • ZIM metadata check now allows multiple values (comma-separated) for Language
  • Using yt_dlp instead of youtube_dl

Removed

  • Dropped support for Python 3.6

3.0.0

31 Mar 11:07
Compare
Choose a tag to compare

⚠️ Warning: this release introduce several API changes to zim.creator.Creator and zim.filesystem.make_zim_file

Added

  • zim.creator.Creator.config_metadata method (returning Self) exposing all mandatory Metdata, all standard ones and allowing extra text metdadata.
  • zim.creator.Creator.config_dev_metadata method setting stub metdata for all mandatory ones (allowing overrides)
  • zim.metadata module with a list of per-metadata validation functions
  • zim.creator.Creator.validate_metadata (called on start) to verify metadata respects the spec (and its recommendations)
  • zim.filesystem.make_zim_file accepts a new optional long_description param.
  • i18n.is_valid_iso_639_3 to check ISO-639-3 codes
  • image.probing.is_valid_image to check Image format and size

Changed

  • zim.creator.Creator main_path argument now mandatory
  • zim.creator.Creator.start now fails on missing required or invalid metadata
  • zim.creator.Creator.add_metadata nows enforces validation checks
  • zim.filesystem.make_zim_file renamed its favicon_path param to illustration_path
  • zim.creator.Creator.config_indexing language argument now optionnal when indexing=False
  • zim.creator.Creator.config_indexing now validates language is ISO- 639-3 when indexing=True

Removed

  • zim.creator.Creator.update_metadata. See .config_metadata() instead
  • zim.creator.Creator language argument. See .config_metadata() instead
  • zim.creator.Creator keyword arguments. See .config_metadata() instead
  • zim.creator.Creator.add_default_illustration. See .config_metadata() instead
  • zim.archibe.Archive.media_counter (deprecated in 2.0.0)

2.1.0

06 Mar 16:32
Compare
Choose a tag to compare

Added

  • zim.creator.Creator(language=) can be specified as List[str]. ["eng", "fra"], ["eng"], "eng,fra", "eng" are all valid values.

Changed

  • Fixed zim.providers.URLProvider returning incomplete streams under certain circumstances (from openzim/kolibri#40)
  • Fixed zim.creator.Creator not supporting multiple values in for Language metadata, as required by the spec

2.0.0

06 Dec 15:12
Compare
Choose a tag to compare
  • Using pylibzim v2.1.0 (using libzim 8.1.0)

Added

  • [libzim] Entry.get_redirect_entry()
  • [libzim] Item.get_indexdata() to implement custom IndexData per entry (writer)
  • [libzim] Archive.media_count

Changed

  • [libzim] Archive.article_count updated to match scraperlib's version
  • Archive.article_counter now deprecated. Now returns Archive.article_count
  • Archive.media_counter now deprecated. Now returns Archive.media_count

Removed

  • [libzim] lzma compression algorithm

1.8.0

03 Oct 11:18
Compare
Choose a tag to compare

Added

  • download.get_session() to build a new requests Session

Changed

  • download.stream_file() accepts a session param to use instead of creating one