Separate packing and uploading stages

[cromz22]

What?

This pull request includes changes to TEMPLATE for each task separating packing and uploading stages.
Specifically, this includes the following changes:

• Change egs2/TEMPLATE/<task>/<task>.sh for the above objective
• Update doc/espnet2_tutorial.md accordingly
• Update ci/test_integration_espnet2.sh accordingly

Notes on details of implementation

• For st1, mt1, and s2st1, the packing and uploading to Zenodo stages were tied and would have produced errors when executing the script.
• For diar1, enh1, enh_diar1, slu1, svs1, tts1, and uasr1, skip_upload implicitly meant uploading to Zenodo. It will be less confusing for programmers owing to the change of variable names.
• For asr1, asr2, lm1, spk1, and s2t1, which stages to skip were determined by additional if statement, therefore they would have worked without this pr. The changes made are for consistency.
• For enh_asr1, enh_st1, spk1, and ssl1, there are no stages of uploading to Zenodo, so this would also have worked without this pr. The changes made are for consistency.
• For asvspoof, there were no implementation of packing and uploading, so I kept it that way.
• I did not change st1 in this pr to avoid confict when merging with Bug fixes for egs2/TEMPLATE/st1 #5748.

Why?

Uploading models to Zenodo is deprecated and uploading them to huggingface is encouraged (cf. CONTRIBUTING.md). In some tasks, however, the packing stage and uploading to Zenodo stage are tied together with skip_upload, making it difficult to only upload to huggingface. The changes made here makes it possible to do that by specifying --skip_packing false --skip_upload_hf false.

See also

Issue #5748

[cromz22]

After discussion with @sw005320 and @ftshijt, we are dropping support for Zenodo upload.
I've deleted the stages from scripts and updated the documents.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 52.05%. Comparing base (2cf7402) to head (b3a33f1).
Report is 2 commits behind head on master.

❗ Current head b3a33f1 differs from pull request most recent head fd6738b

Please upload reports for the commit fd6738b to get more accurate results.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5752      +/-   ##
==========================================
- Coverage   54.27%   52.05%   -2.22%     
==========================================
  Files         767      767              
  Lines       70338    70092     -246     
==========================================
- Hits        38173    36484    -1689     
- Misses      32165    33608    +1443     

Flag	Coverage Δ
test_python_espnet2	52.05% <ø> (-0.59%)	⬇️
test_utils	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sw005320]

This looks good to me.

• There is a CI issue. I'll rerun it. If it still exists, it might be due to this PR. Let's check it.
• @ftshijt, can you also review this PR (again)?

[sw005320]

Hmm, we seem to have some mysterious doc error.
https://github.com/espnet/espnet/actions/runs/9191766239/job/25278895509?pr=5752
@cromz22, do you think this is specific to this PR or a general error?

[cromz22]

I don't think so. It seems that the problem is related to notebook, but I don't remember making any changes to .ipynb file.

I think it's related to https://github.com/espnet/notebook. There is no .ipynb file directly under the notebook directory, possibly causing the find command to fail.

espnet/ci/doc.sh

Line 29 in a83f82b

./doc/notebook2rst.sh > ./doc/_gen/notebooks.rst

espnet/doc/notebook2rst.sh

Line 17 in a83f82b

find ./notebook/*.ipynb -exec echo " {}" \;

[sw005320]

I don't think so. It seems that the problem is related to notebook, but I don't remember making any changes to .ipynb file.

I think it's related to https://github.com/espnet/notebook. There is no .ipynb file directly under the notebook directory, possibly causing the find command to fail.

espnet/ci/doc.sh

Line 29 in a83f82b

./doc/notebook2rst.sh > ./doc/_gen/notebooks.rst

espnet/doc/notebook2rst.sh

Line 17 in a83f82b

find ./notebook/*.ipynb -exec echo " {}" \;

Thanks for your investigation. You're right.

[cromz22]

The notebook error is fixed.

I don't know why the following error occurs: lmplz: command not found
https://github.com/espnet/espnet/actions/runs/9198419134/job/25301114078?pr=5752#step:16:3921

I did make changes to asr2.sh, but I don't think this is related to my changes.

[sw005320]

Thanks for the report.
I think Kenlm's installation has failed, or we may need to skip it.

@simpleoier, can you check this error?

[simpleoier]

Hi @cromz22, thanks for your changes!
The error in asr2 seems to be caused by these lines related to skip_stages. You did not add whitespace at the beginning. As a result, the skip_stage=1117, instead of 11 17. The N-gram stage is usually not checked by default.

[cromz22]

Thank you! I see, sorry I overlooked that.
It seems that asr2.sh is the only script that puts the spaces before the numbers and not after (asr1, lm1, spk1, and s2t1 puts them after).
I unified them to put the spaces after.

[cromz22]

Looks like CI was successful (except for MacOS errors).

[sw005320]

Thanks a lot, @cromz22!