why use a alpha in NCALoss ?

[Andrewymd]

Hi, i have a question about the alpha which is a Hyperparameter

base = torch.mean(dist_mat[i]).data[0]

计算logit, base的作用是防止超过计算机浮点数

pos_logit = torch.sum(torch.exp(self.alpha*(base - pos_neig)))
neg_logit = torch.sum(torch.exp(self.alpha*(base - neg_neig)))
loss_ = -torch.log(pos_logit/(pos_logit + neg_logit))

In your implementation, I found that you first used K-nearest neighbors to select negative samples and then calculated the mean of the distances.
Maybe because the result is too small, so multiply by a parameter alpha?

[bnu-wangxun]

It is very important hyper-parameter. alpha is in [20, 60]. The porpose of this hyper-parameter is to make the loss focus on harder negative samples.

Small alpha is not a problem.

[Andrewymd]

Thanks