SVM training is broken with certain inputs

[subdavis]

SVM training is broken

There are several different problems with SVM training that cause it to fail.

These examples show the fish one, but I also tried generic config.

category_models directory is used without first being created

This bug which can be hacked in DIVE but should be fixed in smqtk_trainer.py(137)

Video input groundtruth failure

source /opt/noaa/viame/setup_viame.sh && \
  /opt/noaa/viame/bin/viame_train_detector \
  -il /tmp/tmp8f377cwf/input_folder_list.txt \
  -it /tmp/tmp8f377cwf/input_truth_list.txt \
  -c /tmp/addons/extracted/configs/pipelines/train_svm_over_fish_detections.viame_csv.conf --no-query

With input:

worker@3ac8e49fcba5:/tmp/tmp8f377cwf$ cat input_truth_list.txt 
/tmp/tmp8f377cwf/tmp8ui_jrs9/groundtruth.csv

worker@3ac8e49fcba5:/tmp/tmp8f377cwf$ cat input_folder_list.txt 
/tmp/tmp8f377cwf/tmp8ui_jrs9/tmpywj7ove7.mp4

Produces this error:

Beginning Training Process

Initializing database... Success

Processing 1 video

Processing: tmpywj7ove7.mp4 on GPU... 
ERROR: Directory tmpywj7ove7 contains no GT files

exit code 0 means we can't detect this failure

Image silent failure

source /opt/noaa/viame/setup_viame.sh \
  && /opt/noaa/viame/bin/viame_train_detector \
  -il /tmp/tmpppele35q/input_folder_list.txt \
  -it /tmp/tmpppele35q/input_truth_list.txt \
  -c /tmp/addons/extracted/configs/pipelines/train_svm_over_fish_detections.viame_csv.conf --no-query

Produces error:

Opening groundtruth file /tmp/tmpppele35q/tmpk7ejvdyy/groundtruth.csv
......
Beginning Training Process

Initializing database... Success

Processing 1 video

Processing: tmpk7ejvdyy on GPU... Failure

ERROR: Processing failed, terminating.

exit code 0 means we can't detect this failure

[mattdawkins]

Given the comment about not finding groundtruth, svm training probably doesn't support -it /tmp/tmp8f377cwf/input_truth_list.txt option yet, will see how hard it is to add