Highlight

https://xinshaoamoswang.github.io/paperlists/2020-02-16-arXiv/#foundation-of-deep-learning

• Instance Cross Entropy for Deep Metric Learning and its application in SimCLR-A Simple Framework for Contrastive Learning of Visual Representations

  • I am very glad to highlight that: our proposed ICE is simple and effective, which has also been demonstrated in recent work SimCLR, in the context of self-supervised learning: A Simple Framework for Contrastive Learning of Visual Representations

  • Its loss expression NT-Xent (the normalized temperature-scaled cross entropy loss) is a fantastic application of our recently proposed Instance Cross Entropy for Deep Metric Learning, in the context of self-supervised learnining. I am very excited about this.

    • #InstanceCrossEntropy #TemperatureScaling #RepresentationLearning

  • Research Gate

  • Open Review

  • Reddit

New update

Robustness From CVPR 2019: https://xinshaoamoswang.github.io/paperlists/2019-12-29-CVPR/#robustness

DML From CVPR 2019: https://xinshaoamoswang.github.io/paperlists/2019-12-29-CVPR/#deep-metric-learning

Label Noise & Importance Weighting From ICML 2019: https://xinshaoamoswang.github.io/paperlists/2019-12-29-ICML/

My Recent Work

• Derivative Manipulation for General Example Weighting

• IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude’s Variance Matters

• Paper Reading

Sampling and Weighting

Emphasis Regularisation by Gradient Rescaling for Training Deep Neural Networks with Noisy Labels (arXiv 2019)

Rethinking data fitting and generalisation: MAE has weak training data fitting ability. Please consider how simple our solution is, which is backed up by our fundamental analysis

• Paper: https://arxiv.org/pdf/1905.11233.pdf
• Comments, sharing, discussion: https://www.researchgate.net/publication/333418661_Emphasis_Regularisation_by_Gradient_Rescaling_for_Training_Deep_Neural_Networks_with_Noisy_Labels/comments

Improving MAE against CCE under Label Noise (arXiv 2019)

Rethinking data fitting and generalisation: MAE has weak training data fitting ability. Please consider how simple our solution is, which is backed up by our fundamental analysis

• Paper: https://arxiv.org/pdf/1903.12141.pdf
• Comments, sharing, discussion: https://www.researchgate.net/publication/332070641_Improving_MAE_against_CCE_under_Label_Noise

Ranked List Loss for Deep Metric Learning (CVPR 2019)

• Paper: http://arxiv.org/abs/1903.03238
• Slide: https://drive.google.com/file/d/1nSXCe-7t_EkNwjFuXTnmzzoFr-6jFKVW/view
• Poster: https://drive.google.com/file/d/1vSp3mDRJKdQFNUH12ehuDDyqQfjXFnWM/view

Deep Metric Learning by Online Soft Mining and Class-Aware Attention (AAAI 2019 Oral)

• Paper: https://arxiv.org/abs/1811.01459
• Slide: https://drive.google.com/file/d/1Z44yvdrnrjIeH8x2A4e9-r275y25piKo/view?usp=sharing
• Poster: https://drive.google.com/file/d/1PpCpD9HLtYJQK2tGtsgIhlr1HZ3IF8zF/view?usp=sharing

Sampling Matters in Deep Embedding Learning (ICCV 2017)

• Paper: http://openaccess.thecvf.com/content_ICCV_2017/papers/Wu_Sampling_Matters_in_ICCV_2017_paper.pdf

• Code (MXNet + Python): https://github.com/apache/incubator-mxnet/tree/master/example/gluon/embedding_learning

• Pipeline: net.features->Dense 128->L2 Norm -> Distance Weighted Sampling -> Margin Loss

• Automatic Learning Beta : does not help

• The margin in Margin Loss is sensitive

No Fuss Distance Metric Learning using Proxies (ICCV 2017)

A Unified View of Deep Metric Learning via Gradient Analysis (ICLR 2019 Submission)

Smart Mining for Deep Metric Learning (ICCV 2017)

Heated-up Softmax Embedding (ICLR 2019 Submission)

Deep Metric Learning via Lifted Structured Feature Embedding (CVPR 2016)

Hard-Aware Deeply Cascaded Embedding (ICCV 2017)

Mining on Manifolds: Metric Learning without Labels (CVPR 2018)

Ensemble-based Methods

Attention-based Ensemble for Deep Metric Learning (ECCV 2018)

Deep Metric Learning with BIER: Boosting Independent Embeddings Robustly (TPAMI SUBMISSSION)

BIER - Boosting Independent Embeddings Robustly (ICCV 2017)

Deep Randomized Ensembles for Metric Learning (ECCV 2018)

Deep Metric Learning with Hierarchical Triplet Loss (ECCV 2018)

Hard-Aware Deeply Cascaded Embedding (ICCV 2017)

Clustering Loss

Deep Metric Learning via Facility Location (CVPR 2017)

Deep Spectral Clustering Learning (ICML 2017)

Generative Methods

Deep Adversial Metric Learning (CVPR 2018)

Deep Variational Metric Learning (ECCV 2018)