Multi-Granularity Regularized Re-Balancing for Class Incremental Learning

Paper: https://ieeexplore.ieee.org/document/9815145

Brief Introduction

Data imbalance between old and new classes is a key issue that leads to performance degradation of the model in incremental learning. In this study, we propose an assumption-agnostic method, Multi-Granularity Regularized re-Balancing (MGRB), to address this problem. Re-balancing methods are used to alleviate the influence of data imbalance; however, we empirically discover that they would under-fit new classes. To this end, we further design a novel multi-granularity regularization term that enables the model to consider the correlations of classes in addition to re-balancing the data. A class hierarchy is first constructed by ontology, grouping semantically or visually similar classes. The multi-granularity regularization then transforms the one-hot label vector into a continuous label distribution, which reflects the relations between the target class and other classes based on the constructed class hierarchy. Thus, the model can learn the inter-class relational information, which helps enhance the learning of both old and new classes. Experimental results on both public datasets and a real-world fault diagnosis dataset verify the effectiveness of the proposed method.

The figure on the left shows the structure of the baseline method, and the figure on the right is the overview of our MGRB method. The proposed model has two significant parts compared with the baseline: (1) re-balancing modeling. We use the re-balancing strategies during training to alleviate the influence of data imbalance; and (2) multi-granularity regularization. A multi-granularity regularization term is designed to make the model consider class correlations. Through end-to-end learning, both old and new classes can be better learned.

Environment

• Python 3.7
• Pytorch 1.8.1
• CUDA 11.2

Requirement

• See requirements.txt for environment.
• The pre-trained word vector library can be found here. You can download it or download the library used in this paper here.

Results

The experimental results on the CIFAR100 are as follows：

Methods	CIFAR100-10/10		CIFAR100-20/20		CIFAR100-50/5		CIFAR100-50/10
	last	Avg acc	last	Avg acc	last	Avg acc	last	Avg acc
LwF	28.18	49.33	39.49	57.49	33.75	46.30	39.28	52.19
iCaRL	45.72	59.96	50.33	63.23	43.97	52.93	46.71	57.06
LUCIR	41.86	56.57	50.15	62.64	50.12	60.37	50.55	61.95
Mnemonics	42.42	58.53	48.95	63.18	50.79	60.43	53.58	63.12
BiC	47.30	57.56	40.90	48.33	41.63	54.11	50.88	58.35
Ours-CNN(ont)	45.81	61.02	56.03	66.45	52.59	61.50	58.20	65.45
Ours-CNN(vis)	47.72	62.38	57.00	68.27	52.69	61.51	58.63	65.69
PODNet	39.50	54.14	50.10	62.94	53.70	62.35	55.30	63.75
Ours-PODNet	40.70	54.61	50.70	63.44	53.90	62.85	55.10	64.32
AANets-iCaRL	45.32	61.40	51.75	65.40	47.91	59.79	50.24	61.71
Ours-AANets	45.63	63.02	49.42	65.72	47.33	59.93	50.26	62.19
DER	57.25	65.31	62.66	70.51	66.63	72.81	66.62	73.07
Ours-DER	58.03	65.42	62.94	70.74	66.61	73.06	66.82	73.27

File organization

The demo folder provides a demo version of the proposed multi-granularity regular term, which can be easily added to PODNet, AANets, and DER.

The code folder provides the complete code for MGRB.

├── data                    # dataset and nodepathinfo
├── demo
    ├── save                # data preprocessing file
    ├── parser.py           # add parameters
    ├── loss.py             # return loss
    ├── cal_MG.py           # calculate the multi-granularity regularization term
    ├── util.py             
    └── ...
├── code
    ├── save                # data preprocessing file
    ├── divide___.py        # data split
    ├── Load___.py          # data loader
    ├── etrain.py           # train and test
    ├── model.py            # network
    ├── cal_MG.py           # calculate the multi-granularity regularization term
    ├── cal_RB.py           # calculate the weight for class re-balancing loss
    └── util.py

Citation

If you use this paper/code in your research, please consider citing us:

H. Chen, Y. Wang and Q. Hu, "Multi-Granularity Regularized Re-Balancing for Class Incremental Learning," in IEEE Transactions on Knowledge and Data Engineering, 2022, doi: 10.1109/TKDE.2022.3188335.

@ARTICLE{9815145,
  author={Chen, Huitong and Wang, Yu and Hu, Qinghua},
  journal={IEEE Transactions on Knowledge and Data Engineering}, 
  title={Multi-Granularity Regularized Re-Balancing for Class Incremental Learning}, 
  year={2022},
  volume={},
  number={},
  pages={1-15},
  doi={10.1109/TKDE.2022.3188335}}