Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add the PixIT task, the ToTaToNet model, and a pipeline for joint speaker diarization/speech separation inference #1676

Open
wants to merge 88 commits into
base: develop
Choose a base branch
from
Open

Add the PixIT task, the ToTaToNet model, and a pipeline for joint speaker diarization/speech separation inference #1676

wants to merge 88 commits into from
+2,386 −0

Commits on Mar 18, 2024

  1. Merge pull request #5 from pyannote/develop 

    git: rebase to latest develop branch
    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    8365802 View commit details
    Browse the repository at this point in the history

  2. add convnet layers to PyanNet

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    ff2c705 View commit details
    Browse the repository at this point in the history

  3. add stft and free encoders/decoder

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    28c9057 View commit details
    Browse the repository at this point in the history

  4. multitask learning first attempt

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    eca0a31 View commit details
    Browse the repository at this point in the history

  5. properly logging mixit loss in train/valid

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    1d2d1ab View commit details
    Browse the repository at this point in the history

  6. add a weight to mixit_loss

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    4e175b6 View commit details
    Browse the repository at this point in the history

  7. reformulate multitask loss

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    7894bf7 View commit details
    Browse the repository at this point in the history

  8. add dprnn

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    9eeb29c View commit details
    Browse the repository at this point in the history

  9. pair mixtures from same file with no overlapping speakers

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    2fb16e9 View commit details
    Browse the repository at this point in the history

  10. fix mixit loss for odd batch size in validation

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    1a83fec View commit details
    Browse the repository at this point in the history

  11. make the MoM part of the original batch

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    3ebc46a View commit details
    Browse the repository at this point in the history

  12. clean up

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    055744f View commit details
    Browse the repository at this point in the history

  13. check that BS is divisible by 3

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    4af6be1 View commit details
    Browse the repository at this point in the history

  14. don't use MoMs with more than 3 speakers

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    4aa83b3 View commit details
    Browse the repository at this point in the history

  15. include original mixtures in separation branch training

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    ef14f8f View commit details
    Browse the repository at this point in the history

  16. matching the order of dimensions of branch outputs

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    b218473 View commit details
    Browse the repository at this point in the history

  17. make n_sources an argument for model constructor

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    ebe3471 View commit details
    Browse the repository at this point in the history

  18. changing LSTM default num_layers to 4

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    df6794b View commit details
    Browse the repository at this point in the history

  19. create separate tasks and models

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    1fe6106 View commit details
    Browse the repository at this point in the history

  20. Changing n_sources to 3

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    b3a7821 View commit details
    Browse the repository at this point in the history

  21. forcing alignment between separation and diarization

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    0c790d5 View commit details
    Browse the repository at this point in the history

  22. fixing edge case of 4 speakers in a second chunk

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    70adb7e View commit details
    Browse the repository at this point in the history

  23. adding a VAD-like forced alignment loss

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    fcde9b8 View commit details
    Browse the repository at this point in the history

  24. refactor: remove vad_loss and warm_up, assume powerset everywhere

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    699a2fb View commit details
    Browse the repository at this point in the history

  25. remove double check of num_speakers

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    065cde0 View commit details
    Browse the repository at this point in the history

  26. refactor: moved mom constrcution

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    3ff0ba5 View commit details
    Browse the repository at this point in the history

  27. remove unused mixit wrapper

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    8bb63d2 View commit details
    Browse the repository at this point in the history

  28. format with black

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    5bd913d View commit details
    Browse the repository at this point in the history

  29. fix for last batch in validation having size 1

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    40c11db View commit details
    Browse the repository at this point in the history

  30. adding documentation

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    3ee1484 View commit details
    Browse the repository at this point in the history

  31. diarization on sources separately and back to multilabel

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    71541c2 View commit details
    Browse the repository at this point in the history

  32. make lstm use optional

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    8c66d74 View commit details
    Browse the repository at this point in the history

  33. make alignment forcing optional

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    dcab13d View commit details
    Browse the repository at this point in the history

  34. bug fix

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    954e0f8 View commit details
    Browse the repository at this point in the history

  35. rename mixit_loss to separation_loss for clarity

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    7dee18c View commit details
    Browse the repository at this point in the history

  36. add 2 sources for noise and alignement accuracy measure

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    f792135 View commit details
    Browse the repository at this point in the history

  37. bug regarding specifications being a tuple

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    9f2cd5b View commit details
    Browse the repository at this point in the history

  38. clean up

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    ded4b5e View commit details
    Browse the repository at this point in the history

  39. add avg pooling to diarization branch for smaller kernel sizes

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    90c9b3a View commit details
    Browse the repository at this point in the history

  40. fix validation loss

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    f97d440 View commit details
    Browse the repository at this point in the history

  41. changing to pit_loss

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    2654528 View commit details
    Browse the repository at this point in the history

  42. changing validation dataloader

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    1ed87e9 View commit details
    Browse the repository at this point in the history

  43. make the additional 2 noise sources optional

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    b97dd16 View commit details
    Browse the repository at this point in the history

  44. make aligned training the only supported behavior

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    b2baf1d View commit details
    Browse the repository at this point in the history

  45. clean up

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    f84c683 View commit details
    Browse the repository at this point in the history

  46. change default model parameters

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    23f8b7b View commit details
    Browse the repository at this point in the history

  47. 3 source mixit

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    7ccbee1 View commit details
    Browse the repository at this point in the history

  48. first commit

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    6917aa0 View commit details
    Browse the repository at this point in the history

  49. diar branch from masked tf rep instead

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    407e1fb View commit details
    Browse the repository at this point in the history

  50. add lstm

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    66146fb View commit details
    Browse the repository at this point in the history

  51. add lstm back in

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    adcc594 View commit details
    Browse the repository at this point in the history

  52. fix forward

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    053324e View commit details
    Browse the repository at this point in the history

  53. adding training on single speaker sources

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    c1519f4 View commit details
    Browse the repository at this point in the history

  54. check edge case

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    680e181 View commit details
    Browse the repository at this point in the history

  55. clean up and format

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    031cb9d View commit details
    Browse the repository at this point in the history

  56. first commit

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    4d20d4a View commit details
    Browse the repository at this point in the history

  57. enable finetuning wavlm with separate lr (pytorch-lightning 2.1)

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    21d7f40 View commit details
    Browse the repository at this point in the history

  58. add gradient clipping

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    8081bef View commit details
    Browse the repository at this point in the history

  59. clean up

    @joonaskalda
    joonaskalda committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    5104521 View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. rename SepDiarNet and include receptive field

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    a8c8caa View commit details
    Browse the repository at this point in the history

  2. fix rebase mistake in PyanNet

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    0a9597b View commit details
    Browse the repository at this point in the history

  3. fix rebase mistake in segmentation mixins

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    ec36794 View commit details
    Browse the repository at this point in the history

  4. fix joint task setup

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    7cfdc75 View commit details
    Browse the repository at this point in the history

  5. fix joint task init

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    4b37532 View commit details
    Browse the repository at this point in the history

  6. fix data iteration and add docstrings

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    c963934 View commit details
    Browse the repository at this point in the history

  7. fix docstrings

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    658308a View commit details
    Browse the repository at this point in the history

  8. remove functionality for additional noise sources

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    b3ba2d4 View commit details
    Browse the repository at this point in the history

  9. remove functionality for using original sources for separation (when … 

    …one speaker speaking)
    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    e451a2b View commit details
    Browse the repository at this point in the history

  10. fixc rebase mistake in speaker diarization task

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    386d750 View commit details
    Browse the repository at this point in the history

  11. rename joint task to PixIT

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    60210d9 View commit details
    Browse the repository at this point in the history

  12. make wavlm finetuning optional

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    047f741 View commit details
    Browse the repository at this point in the history

  13. clean up ToTaToNet

    @joonaskalda
    joonaskalda committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    110355e View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. add joint diarization separation pipeline

    @joonaskalda
    joonaskalda committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    e2cdae4 View commit details
    Browse the repository at this point in the history

  2. fix docstrings and imports

    @joonaskalda
    joonaskalda committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    4df4950 View commit details
    Browse the repository at this point in the history

  3. fix ToTaToNet behavior when WavLM not used

    @joonaskalda
    joonaskalda committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    c848143 View commit details
    Browse the repository at this point in the history

  4. update requirements.txt

    @joonaskalda
    joonaskalda committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    ffe583a View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. Merge branch 'develop' into pixit

    @hbredin
    hbredin committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    9d79069 View commit details
    Browse the repository at this point in the history

  2. doc: update changelog

    @hbredin
    hbredin committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    75c3c8e View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. Merge branch 'develop' into pixit

    @hbredin
    hbredin committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    4112ef8 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. Merge branch 'develop' into pixit

    @hbredin
    hbredin committed May 24, 2024
    Configuration menu
    Copy the full SHA
    776349b View commit details
    Browse the repository at this point in the history

  2. chore: reorganize things a bit

    @hbredin
    hbredin committed May 24, 2024
    Configuration menu
    Copy the full SHA
    e133515 View commit details
    Browse the repository at this point in the history

  3. fix: fix import

    @hbredin
    hbredin committed May 24, 2024
    Configuration menu
    Copy the full SHA
    6e56060 View commit details
    Browse the repository at this point in the history

  4. fix: fix docstring

    @hbredin
    hbredin committed May 24, 2024
    Configuration menu
    Copy the full SHA
    6adcd2d View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. fix: fix docstrings and default values

    @joonaskalda
    joonaskalda committed May 27, 2024
    Configuration menu
    Copy the full SHA
    8be568c View commit details
    Browse the repository at this point in the history

  2. fix: rename speaker separation to speech separation

    @joonaskalda
    joonaskalda committed May 27, 2024
    Configuration menu
    Copy the full SHA
    e8a4db0 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. fix: renaming and reorganizing separation pipeline hyperparameters

    @joonaskalda
    joonaskalda committed May 28, 2024
    Configuration menu
    Copy the full SHA
    8f97c6a View commit details
    Browse the repository at this point in the history

  2. doc: add hyper-parameter documntation

    @hbredin
    hbredin committed May 28, 2024
    Configuration menu
    Copy the full SHA
    284fb83 View commit details
    Browse the repository at this point in the history

  3. feat(setup): make separation dependencies optional

    @hbredin
    hbredin committed May 28, 2024
    Configuration menu
    Copy the full SHA
    089504f View commit details
    Browse the repository at this point in the history