Charabia v0.8.10

Changes

• Update bors.toml with missing tests (#286) @curquiza
• Add swedish recomposition normalizer and link it to a feature (#287) @ManyTheFish

Thanks again to @ManyTheFish, @curquiza, @meili-bors[bot] ! 🎉

Charabia v0.8.9

Changes

• Add \t as recognized separator (#280) @Gusted
• Update Lindera to 0.30.0 (#279) @mosuka
• Fix char boundary panic (#281) @ManyTheFish
• Make the pinyin-normalization optional (#282) @ManyTheFish
  • This can be reactivated by enabling the chinese-normalization-pinyin feature

Thanks again to @Gusted, @ManyTheFish, and @mosuka! 🎉

Charabia v0.8.8

Changes

• Update Lindera to 0.28.0 (#273) @mosuka
• Support markdown formatted codeblocks (#275) @6543

Thanks again to @6543, @ManyTheFish, @dependabot, @dependabot[bot], @meili-bors[bot], and @mosuka! 🎉

Charabia v0.8.7

Changes

• Fix compilation when vietnamese feature is disabled (#259) @timvisee
• Fix unused FstSegmenter warning when not using khmer compiler features (#261) @timvisee
• Update dependencies (#262) @agourlay
• Add vietnamese benchmarks (#267) @ManyTheFish
• Update README.md (#269) @ManyTheFish
• Vietnamese: Add laking tests and fix bug (#270) @ManyTheFish

Thanks again to @ManyTheFish, @agourlay, @curquiza, @dependabot, @dependabot[bot], @meili-bors[bot], and @timvisee! 🎉

Charabia v0.8.6

Changes

• Improve khmer segmenter performance by using fst segmenter (#251) @xshadowlegendx
• Fix update-kvariants CI (#256) @choznerol
• normalize Ð and Đ into d (#257) @ngdbao

Thanks again to @ManyTheFish, @choznerol, @dependabot, @dependabot[bot], @meili-bors[bot], @ngdbao and @xshadowlegendx! 🎉

Charabia v0.8.5

Changes

• Fuzz testing with quickcheck for normalizers, segmenters, tokenizer and classifier. (#240) @choznerol
• add khmer segmenter (#203) @xshadowlegendx

Thanks again to @ManyTheFish, @choznerol, @dependabot, @dependabot[bot], @meili-bors[bot], and @xshadowlegendx! 🎉

Charabia v0.8.4

Changes

• Update Lindera to v0.27.1 for changing the UniDic download URL (#237) @mosuka
• Implement the CharNormalizer trait on the LowercaseNormalizer struct (#241) @Bradshaw

Thanks again to @Bradshaw, @ManyTheFish, @dependabot, @dependabot[bot], @meili-bors[bot], and @mosuka! 🎉

Charabia v0.8.3

Changes

• Remap the char map when lowercasing strings (#234) @Kerollmops

Thanks again to @Kerollmops, @dependabot, @dependabot[bot], @meili-bors[bot] ! 🎉

Charabia v0.8.2

Changes

• Update Lindera to 0.27.0 (#227) @mosuka
• Fix pre-segmenter when a string start by an uncategorized character (#231) @ManyTheFish

Thanks again to @ManyTheFish, @dependabot, @dependabot[bot], @meili-bors[bot], and @mosuka! 🎉

Charabia v0.8.1

Changes

• Remove colon from hard separators (#223) @ManyTheFish

Thanks again to @ManyTheFish! 🎉