You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
maybe change to
negative_score = neg_embeds.bmm(center_embeds.transpose(1, 2))
loss = self.logsigmoid(positive_score) + torch.sum(self.logsigmoid(negative_score), 1)
since based on the equation, the negative_socre first goes through a logsigmoid operation, and then sums up.
The text was updated successfully, but these errors were encountered:
I have learned a lot from this elegant project. Thanks a lots!
Based on the equation in the Skip-gram-Negative-Sampling algorithm below,
I think the negative example loss calculated by
negative_score = torch.sum(neg_embeds.bmm(center_embeds.transpose(1, 2)).squeeze(2), 1).view(negs.size(0), -1) # BxK -> Bx1
loss = self.logsigmoid(positive_score) + self.logsigmoid(negative_score)
maybe change to
negative_score = neg_embeds.bmm(center_embeds.transpose(1, 2))
loss = self.logsigmoid(positive_score) + torch.sum(self.logsigmoid(negative_score), 1)
since based on the equation, the negative_socre first goes through a logsigmoid operation, and then sums up.
The text was updated successfully, but these errors were encountered: