baseline model

New baseline model

• train-validation-test splits
• flip vertically in augmentation
• wandb: https://wandb.ai/cstorm125/choco-raisin

Dataloader

dblock = DataBlock(
    blocks=(ImageBlock, CategoryBlock), #x - image; y - single class
    get_items=get_image_files, #get image
    splitter=GrandparentSplitter(), #use parent folder as train-valid split
    get_y=parent_label, #use parent folder as label
    #two choices for resizing and rationale
    #squishing to prevent cropping places without chips/raisins
    item_tfms=Resize(512, method=ResizeMethod.Squish), 
    # #cropping to preserve image quality; tried and doesn't work - peaked at 0.85 val acc
    # item_tfms=RandomResizedCrop(512),
    batch_tfms=aug_transforms(size=512, flip_vert=True), #standard fastai augmentation at size 512
    )
dls = dblock.dataloaders(path, bs=64) #batch size = 64

Training

learn.fine_tune(epochs=5,
          base_lr=1e-3,
          freeze_epochs=1, 
          lr_mult=100, 
          pct_start=0.2, 
          div=5.0, 
          cbs=[WandbCallback(), #track to wandb
               SaveModelCallback(monitor='f1_score')] #monitor f1 score and save best model
          )

#three layer groups for finetuning resnet in fastai; lr_mult=100
1. `lr_0` - frozen max lr = `base_lr/10` (not trained) - unfrozen max lr = `base_lr/2/lr_mult` = `base_lr/2/100` 
2. `lr_1` - frozen max lr = `base_lr/10` (not trained) - unfrozen max lr = `base_lr/2/10` (slice with step 10)
3. `lr_2` - frozen max lr = `base_lr` - unfrozen max lr = `base_lr/2`

Validation results:

                precision    recall  f1-score   support

chocolate chip       0.94      0.93      0.94        72
        raisin       0.93      0.94      0.94        70

      accuracy                           0.94       142
     macro avg       0.94      0.94      0.94       142
  weighted avg       0.94      0.94      0.94       142

Test results:

                precision    recall  f1-score   support

chocolate chip       0.89      0.76      0.82        72
        raisin       0.79      0.90      0.84        70

      accuracy                           0.83       142
     macro avg       0.84      0.83      0.83       142
  weighted avg       0.84      0.83      0.83       142

baseline model

Baseline chocolate chip vs raisin cookies classification model with resnet34 trained with fp16. Validation accuracy at 0.901408.

learn.fine_tune(epochs=5,
          base_lr=1e-3, #max lr; when unfrozen base_lr/2
          freeze_epochs=1, #how many epochs to train frozen
          lr_mult=100, #train feature extractor with max lr at base_lr/lr_mult
          pct_start=0.2, #start decreasing lr at 
          div=5.0, #start at base_lr (max lr) / div
          cbs=[WandbCallback(), #track to wandb
               SaveModelCallback(monitor='f1_score')] #monitor f1 score and save best model
          )

#three layer groups for finetuning resnet in fastai
1. `lr_0` - frozen max lr = `base_lr/10` (not trained) - unfrozen max lr = `base_lr/2/lr_mult`
2. `lr_1` - frozen max lr = `base_lr/10` (not trained) - unfrozen max lr = `base_lr/2/(lr_mult*10)`
3. `lr_2` - frozen max lr = `base_lr` - unfrozen max lr = `base_lr/2`