Reproduce annotation reliability procedures

[lucasgautheron]

Tsi_reliability

• bash scripts/1_eaf2rttm_all.sh fails python: can't open file 'adjust_timestamps.py': [Errno 2] No such file or directory
• bash scripts/2_evaluate.sh

No 'all.rttm' file inAnnotation_supervad/2017/rttms/mapped_four_class/
Extraction starts for Annotation_supervad/2017/rttms/mapped_four_class/
VTC rttm files created
Traceback (most recent call last):
  File "scripts/splitting_vtc_rttm.py", line 121, in <module>
    main()
  File "scripts/splitting_vtc_rttm.py", line 116, in main
    extraction(rttm, gold, outer)
  File "scripts/splitting_vtc_rttm.py", line 31, in extraction
    with open(rttm, 'r') as init_rttm:
IsADirectoryError: [Errno 21] Is a directory: 'Annotation_supervad/2017/rttms/mapped_four_class/'
mkdir: cannot create directory ‘evaluations’: File exists

Done computing metrics between Annotation_Mustapha/2017/rttms/mapped_gold_no_ele_no_ovl and Annotation_supervad/2017/rttms/mapped_four_class.
Done computing metrics between Annotation_Mustapha/2017/rttms/mapped_gold_no_ele_no_ovl and Annotation_Natalia/2017/rttms/mapped_gold_no_ele_no_ovl.
Done framarizing Annotation_Mustapha/2017/rttms/mapped_gold_no_ele_no_ovl/
Done framarizing Annotation_supervad/2017/rttms/mapped_four_class
[1] "Gold labels found :"
 [1] "FEM"         "MAL"         "OCH"         "SIL"         "CHI"        
 [6] "FEM/MAL"     "FEM/OCH"     "CHI/FEM"     "CHI/MAL"     "CHI/OCH"    
[11] "MAL/OCH"     "CHI/FEM/OCH"
[1] "Lena labels found :"
 [1] "CHI"     "FEM"     "OCH"     "SIL"     "MAL"     "CHI/FEM" "CHI/OCH"
 [8] "FEM/OCH" "MAL/OCH" "FEM/MAL" "CHI/MAL"
[1] "Done computing confusion matrix."
mv: cannot stat ‘Annotation_Mustapha/2017/rttms/framed_gold/*.txt’: No such file or directory

• bash scripts/3_reliability.sh

$ bash scripts/3_reliability.sh 
python: can't open file 'labels_mapper.py': [Errno 2] No such file or directory
python: can't open file 'labels_mapper.py': [Errno 2] No such file or directory
python3: can't open file 'compute_metrics.py': [Errno 2] No such file or directory
mv: cannot stat ‘../Annotation_Mustapha/2017/rttms/mapped_gold_no_ele_no_ovl/gold_no_ele_no_ovl_gold_no_ele_no_ovl/’: No such file or directory
python3: can't open file 'frame_cutter.py': [Errno 2] No such file or directory
python3: can't open file 'frame_cutter.py': [Errno 2] No such file or directory
Fatal error: cannot open file 'conf_mat.r': No such file or directory
mkdir: cannot create directory ‘../reliability/conf_mat/’: No such file or directory
mv: cannot stat ‘../Annotation_Mustapha/2017/rttms/framed/*cm*’: No such file or directory
Fatal error: cannot open file 'extract_stat_reliability.r': No such file or directory

Paul

Following instructions from https://www.dropbox.com/sh/3c9f2p6631ghilv/AAD-MtY0dKG_0XsKLIlmX6ega?dl=0&preview=README.txt

• 0) Prerequistes

R -e "Sys.setenv(TAR='/bin/tar'); devtools::install_github('HomeBankCode/rlena', dependencies = TRUE)"

failed, had to to R

library("devtools") then the rest of the commands

• 1) Preprocessing
• 2) LENA evaluation
• 3) Plotting results (the script is missing, so TODO: write a script to produce confusion matrix plots)
• 4) Run iner_rather reliability

$ bash scripts/3_evaluate_IR.sh
Directory ./rttm_annotations_mm/mapped_gold  created
Directory ./rttm_annotations_ak/mapped_gold  created
Traceback (most recent call last):
  File "scripts/labels_mapper.py", line 337, in <module>
    main()
  File "scripts/labels_mapper.py", line 333, in main
    map_rttm(rttm, overlap, dict, same, output_folder)
  File "scripts/labels_mapper.py", line 213, in map_rttm
    onset, duration, sentence, sentence_type, speaker = float(splitted[3]), float(splitted[4]), splitted[5], splitted[6], splitted[7]
ValueError: could not convert string to float:
Done computing metrics between ./rttm_annotations_pa/mapped_gold and ./rttm_annotations_mm/mapped_gold.
Done computing metrics between ./rttm_annotations_pa/mapped_gold and ./rttm_annotations_ak/mapped_gold.
Done framarizing ./rttm_annotations_pa/mapped_gold/
Done framarizing ./rttm_annotations_mm/mapped_gold/
scripts/3_evaluate_IR.sh: line 34: pythin: command not found
[1] "Gold labels found :"
 [1] "CHI"         "OCH"         "SIL"         "FEM"         "MAL"
 [6] "CHI/OCH"     "CHI/FEM"     "FEM/OCH"     "FEM/MAL"     "CHI/MAL"
[11] "MAL/OCH"     "FEM/MAL/OCH" "CHI/FEM/OCH" "CHI/FEM/MAL" "CHI/MAL/OCH"
[1] "Lena labels found :"
 [1] "CHI"     "FEM"     "MAL"     "OCH"     "SIL"     "CHI/FEM" "FEM/MAL"
 [8] "FEM/OCH" "MAL/OCH" "CHI/MAL" "CHI/OCH"
[1] "Done computing confusion matrix."
[1] "Gold labels found :"
 [1] "CHI"         "OCH"         "SIL"         "FEM"         "MAL"
 [6] "CHI/OCH"     "CHI/FEM"     "FEM/OCH"     "FEM/MAL"     "CHI/MAL"
[11] "MAL/OCH"     "FEM/MAL/OCH" "CHI/FEM/OCH" "CHI/FEM/MAL" "CHI/MAL/OCH"
[1] "Lena labels found :"
NULL
[1] "Done computing confusion matrix."

Attaching package: ‘tidyr’

The following object is masked from ‘package:magrittr’:

    extract

Error in commadnArgs(TRUE) : could not find function "commadnArgs"
Execution halted

Attaching package: ‘tidyr’

The following object is masked from ‘package:magrittr’:

    extract

Error in commadnArgs(TRUE) : could not find function "commadnArgs"
Execution halted

_nam_stab_chi