Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

comprehensive french tokenizer without exceptions list #13378

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

comprehensive french tokenizer without exceptions list #13378

wants to merge 12 commits into from

Commits on Mar 15, 2024

  1. refactored french tokenizer

    @thjbdvlt
    thjbdvlt committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    5a3928f View commit details
    Browse the repository at this point in the history

  2. works 

    modifié :         __init__.py
modifié :         punctuation.py
modifié :         tokenizer_exceptions.py
    @thjbdvlt
    thjbdvlt committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    ef4c655 View commit details
    Browse the repository at this point in the history

  3. end french tokenizer + add sentences (examples)

    @thjbdvlt
    thjbdvlt committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    8e8dd31 View commit details
    Browse the repository at this point in the history

  4. removed few new sentences

    @thjbdvlt
    thjbdvlt committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    73b68c4 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. formatted with black

    @thjbdvlt
    thjbdvlt committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    3962669 View commit details
    Browse the repository at this point in the history

  2. isort + flake8 fixed

    @thjbdvlt
    thjbdvlt committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    f3967e8 View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2024

  1. fixed variable types

    @thjbdvlt
    thjbdvlt committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    6578492 View commit details
    Browse the repository at this point in the history

  2. missing suffix: 'ce'

    @thjbdvlt
    thjbdvlt committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    43aec2d View commit details
    Browse the repository at this point in the history

  3. add months + days abbrev

    @thjbdvlt
    thjbdvlt committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    b0b66fb View commit details
    Browse the repository at this point in the history

  4. add dot after some abbrevs

    @thjbdvlt
    thjbdvlt committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    54fcc5b View commit details
    Browse the repository at this point in the history

  5. add abbrev juill. (same as juil.)

    @thjbdvlt
    thjbdvlt committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    0f1a82b View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. add elleux

    @thjbdvlt
    thjbdvlt committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    aee667a View commit details
    Browse the repository at this point in the history