Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improving CM script automation and CM scripts #21

Open
14 of 25 tasks
gfursin opened this issue Apr 24, 2024 · 0 comments
Open
14 of 25 tasks

Improving CM script automation and CM scripts #21

gfursin opened this issue Apr 24, 2024 · 0 comments
Assignees
@gfursin
Labels
enhancement New feature or request

Comments

@gfursin
Copy link
Contributor

gfursin commented Apr 24, 2024 •
edited

Aggregating tasks to improve CM script automation and CM scripts based on user feedback (this ticket is being gradually updated and tasks resolved based on our bandwidth and engineering resources):

Automation

  • Add --silent / -s mode to avoiding printing CM workflow execution
  • Add cm_min_version check for automation and scripts
  • Check current dumping of all versions of all dependencies Commit
  • Check readme generation with all deps Commit
    • Add --version when available Commit
  • Prepare tmp_run_final_script.sh and stop to let user run it manually without CM if needed
  • Generate sample Docker container during --repro
  • Add cfg to load default env, state and keys for all CM scripts
    • cm set cfg default --key.cm-script.silent --key.cm-script.env.CM_SUDO="no" ...
    • Add silent mode as default
    • Support add/update cfg keys ...
  • Add cm status to show version, paths to current repositories and if the version is up-to-date (useful for virtual environments)
  • Add to meta where to report errors if not default repository (for MLPerf inference -> report https://github.com/mlcommons/inference)
  • Add proper version detection for cuDNN

Documentation

  • Check individual documentation and input description for all main CM scripts
  • Add individual tests for all main CM scripts

Tutorials

Need to explain how to extend CM scripts, add new ones, unify inputs, etc)

  • Prepare tutorial about CM basics and CM scripts
  • Prepare tutorial about CM automation for basic inference
  • Prepare tutorial about CM automation for MLPerf loadgen
  • Prepare tutorial about CM automation for MLPerf inference
  • Prepare tutorial about CM automation for ABTF
  • Prepare tutorial about CM automation for SCC'24

Longer term

  • Use logging instead of print
  • Refactor and simplify CM script automation - it was heavily prototyped and it is stable but the implementation can be considerably improved and simplified
  • Refactor docker container generation and execution - the prototype implementation is very complex and can be dramatically simplified
@gfursin gfursin self-assigned this Apr 24, 2024
@gfursin gfursin added the enhancement New feature or request label Apr 24, 2024
@gfursin gfursin changed the title Improving CM scripts Improving CM script automation and CM scripts Apr 24, 2024
@gfursin gfursin mentioned this issue Apr 29, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gfursin gfursin

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@gfursin