Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modification of weights for unbalanced dataset #27

Open
Eavis opened this issue Feb 25, 2019 · 1 comment
Open

Modification of weights for unbalanced dataset #27

Eavis opened this issue Feb 25, 2019 · 1 comment

Comments

@Eavis
Copy link

Eavis commented Feb 25, 2019

No description provided.

@Eavis
Copy link
Author

Eavis commented Feb 25, 2019

I have modified the ADE20K scene parsing benchmark produced by MIT. The original dataset contains 150 categoried(151 include background) and set background=0 to be ignored.
I have chosen 17 categories and set the previous background(label as 0) to 255, and set the other categories as 0(background). Thus ignore_class = 255

I count the pixels in the 20210 images after the modification. <category idx, number of pixel counts>
0,1637454744
1,737138625
2,503599787
3,414602893
4,290185267
5,211497624
6,93447406
7,84733791
8,78265150
9,74475972
10,70227430
11,55492152
12,28967419
13,28361240
14,24627334
15,9994821
16,7816482
17,3719357
255,411637498

The background is 0.6 of all the other categories in total(1-17)
However, the background is 440 times the 17th category. (1637454744 / 3719357= 440.3)
(16th: 1637454744 / 7816482 = 209.5)
Does anyone have any suggestion on how to set the weight for these categories?
Should I adjust the weight for only a few categories or all of the 17 categories?

weights = tf.to_float(tf.equal(scaled_labels, 0)) * label0_weight + tf.to_float(tf.equal(scaled_labels, 1)) * label1_weight + tf.to_float(tf.equal(scaled_labels, ignore_label)) * 0.0
    where you need to tune the label0_weight and label1_weight (e.g., set label0_weight=1 and increase label1_weight).


Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant