Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ipex extra in pyproject.toml to use restricted transformers version #127

Open
jiafuzha opened this issue Feb 29, 2024 · 1 comment
Open

Comments

@jiafuzha
Copy link
Contributor

jiafuzha commented Feb 29, 2024

IPEX has restriction on transformers version, but llm-on-ray doesn't have. To verify IPEX and other llm-on-ray functions in parallel in CI, we can add a new ipex extra in pyproject.toml with right transformers version. Then, add corresponding nightly CI in github workflow.

The brief steps are,

  1. add "ipex" extra
    cpu = [
    "transformers>=4.35.0", # some models need higher version of transformers
    "intel_extension_for_pytorch==2.1.0+cpu",
    "torch==2.1.0+cpu",
    "oneccl_bind_pt==2.1.0+cpu"
    ]
    **+ipex = [
  • "transformers==4.31.0", # to make fully functional ipex. choose right version based on following ipex version.
  • "intel_extension_for_pytorch==2.1.0+cpu",
  • "torch==2.1.0+cpu",
  • "oneccl_bind_pt==2.1.0+cpu"
    +]**
  1. add a separate dockerfile so that it can be cached properly in CI build.
    copy one of Dockerfiles under dev/docker and rename it to Dockerfile.ipex. After that, replace 'pip install ...' with 'pip install .[ipex]

  2. add nightly CI
    copy the workflow-inference.yaml and rename to workflow-inference-ipex.yaml and call workflow-inference-ipex.yaml to workflow_orders_nightly.yaml.
    call-inference:
    uses: ./.github/workflows/workflow_inference.yml
    with:
    ci_type: nightly

  • call-inference-ipex:
  • uses: ./.github/workflows/workflow_inference-ipex.yml

Inside workflow-inference-ipex.yaml, add inference tests for ipex supported models.

@KepingYan
Copy link
Contributor

As a reminder, since mpt-7b CI failed, set ipex configuration to False in #125 first after verification.

cheehook pushed a commit to JoshuaL3000/llm-on-ray that referenced this issue Mar 8, 2024
* refactoring

Signed-off-by: Jiafu Zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* refactor argument passing and model configs by using yaml

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

* debugging gpt-j-6b

Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>

---------

Signed-off-by: Jiafu Zhang <jiafu.zhang@intel.com>
Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants