Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PIT Loss for multichannel audio for speech separation #691

Open
SutirthaChakraborty opened this issue Feb 25, 2024 · 1 comment
Open

PIT Loss for multichannel audio for speech separation #691

SutirthaChakraborty opened this issue Feb 25, 2024 · 1 comment
Labels
question Further information is requested

Comments

@SutirthaChakraborty
Copy link

SutirthaChakraborty commented Feb 25, 2024

I have a 4 channel audio generated by my model (left,right,side,mid).
I can I apply PIT loss into it
The shape of the tensors are
Speaker one : [batch,channel,time]
Speaker two: [batch,channel,time]

If I need to apply PIT, how should I apply : [batch,channel,speaker,time] ?

if I convert it to mono, or take the mean, the model is unable to learn 4 channels properly.

@SutirthaChakraborty SutirthaChakraborty added the question Further information is requested label Feb 25, 2024
@mpariente
Copy link
Collaborator

I think the channel should be first, in order to build the permutation matrix of dimension (batch, speaker, speaker) with broadcasting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants