BNM v1 Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations (CVPR2020 oral)
BNM v2 Fast Batch Nuclear-norm Maximization and Minimization for Robust Domain Adaptation (TPAMI under review)
Clean code of BNM v1 can be found in old version
BNM v1: we prove in the paper that Batch Nuclear-norm Maximization (BNM) can ensure the prediction discriminability and diversity, which is an effective method under label insufficient situations.
BNM v2: we further devise Batch Nuclear-norm Minimization (BNMin) and Fast BNM (FBNM) for multiple domain adaptation scenarios.
Assume X is the prediction matrix. We can calculate BNM loss in both Pytorch and Tensorflow, as follows:
-Pytorch
- Direct calculation (Since there remains direct approach for nuclear-norm in Pytorch)
L_BNM = -torch.norm(X,'nuc')
- Calculation with SVD (For S, V and D, only S is useful for calculation of BNM)
L_BNM = -torch.sum(torch.svd(X, compute_uv=False)[1])
-Tensorflow
L_BNM = -tf.reduce_sum(tf.svd(X, compute_uv=False))
Assume X is the prediction matrix. Then FBNM can be calculated as:
list_svd,_ = torch.sort(torch.sqrt(torch.sum(torch.pow(X,2),dim=0)), descending=True)
nums = min(X.shape[0],X.shape[1])
L_FBNM = - torch.sum(list_svd[:nums])
-
- FBNM.(By approximation)
-
- BNMin.(On the other hand on source domain)
-
- Multiple BNM.(Multiple Batch Optimization)
-
- Balance domainnet.(New dataset)
-
- Semi-supervised DA.(New task)
We apply BNM to unsupervised domain adaptation (UDA) in DA, unsupervised open domain recognition (UODR) in UODR and semi-supervised learning (SSL) in SSL.
Training instructions for UDA, UODR and SSL are in the README.md in DA, UODR and SSL respectively.
If you use this code for your research, please consider citing:
@InProceedings{Cui_2020_CVPR,
author = {Cui, Shuhao and Wang, Shuhui and Zhuo, Junbao and Li, Liang and Huang, Qingming and Tian, Qi},
title = {Towards Discriminability and Diversity: Batch Nuclear-Norm Maximization Under Label Insufficient Situations},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}
@article{cui2021fast,
title={Fast Batch Nuclear-norm Maximization and Minimization for Robust Domain Adaptation},
author={Cui, Shuhao and Wang, Shuhui and Zhuo, Junbao and Li, Liang and Huang, Qingming and Tian, Qi},
journal={arXiv preprint arXiv:2107.06154},
year={2021}
}
Supplementary of BNM can be found in Google driver and baidu cloud (z7yt).
Supplementary of BNM2 can be found in Google driver and baidu cloud (hbqc).
If you have any problem about our code, feel free to contact
or describe your problem in Issues.