feat: create bash library to handle install/run/test of python backends

[cryptk]

Description

This creates a library of helper functions to help with installation, run, test of the python backends. The goal being that less time can be spent on these TOIL tasks and more time can be spent where it really matters, writing the code to wrap a backend in an GRPC layer.

For future backends, it will likely be enough to copy the backend/python/common/template folder to backend/python/some_backend_name add code to backend.py and add unit tests to test.py, and then specify dependencies in requirements.txt, requirements-$BUILD_TYPE.txt etc. After that, the backend should be ready for consumption by the LocalAI golang codebase.

The library provides the following functionality:

• initialization creates several useful variables

  • BACKEND_NAME: will have a value like "autogptq" or "diffusers" based on the folder name
  • MY_DIR: will always be the path to the folder that install/run/test.sh are located in, no matter what your current directory is when you execute those scripts
  • BUILD_PROFILE: on top of the existing hipblas and cublas BUILD_TYPE values, this variable could also be cublas11, cublas12, intel and cpu.

• Allow controlling what builds a backend will function with

  • setting LIMIT_TARGETS="cublas" will only allow the backend to operate in cublas builds. You can space separate multiple targets to allow the backend to install/run on several. For example `LIMIT_TARGETS="cublas11 intel" will allow the backend to install/run on cublas builds with CUDA_MAJOR_VERSION=11 and on builds for Intel ARC, but not on cublas version 12, nor on hipblas, nor on cpu (non-accelerated) builds.

• ensureVenv: creates a venv if it does not exist, activates the venv if it is not activated. This function is idempotent so you can run it even if the venv already exists, or is already activated. The end result is that no matter what, after running this, you should end with the correct venv for the backend activated.

• installRequirements: runs the installation for the following requirements files, if they exist, in order:

  • requirements-install.txt
  • requirements.txt
  • requirements-$BUILD_TYPE.txt (ex: requirements-hipblas.txt)
  • requirements-$BUILD_PROFILE.txt (ex: requirements-cuda11.txt only if it is different than the previous file)

• startBackend: attempts to find and start the python backend. It handles ensuring that the virtualenv is properly activated. You can either manually specify a path to a python file by setting BACKEND_FILE="/path/to/something.py" before calling this function or you can have it discover the backend code file based on the following filenames:

  • server.py
  • backend.py
  • ${BACKEND_NAME}.py (ex: autogptq.py)

• runUnittests: handles activating the virtualenv and running the python unittests located in the backend folder within a file named test.py. If you need to run tests from a custom file location (perhaps a file inside a checked out source directory), you can set TEST_FILE=/some/path/to/testfile.py before calling runUnittests.

Notes for Reviewers

All existing backends have been migrated to use the new library functions, so there should be plenty of examples. Most backends only need a few lines to fully handle their installation now, and that comes with the bonus that all installations are handled the same way, so supporting/troubleshooting backends should become easier as the functionality is highly similar between all of them.

Signed commits

• Yes, I signed my commits.

[netlify]

✅ Deploy Preview for localai canceled.

Name	Link
🔨 Latest commit	f266e7e
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/663ee8e21962b500081fbdf5

[cryptk]

I also ran the bark, vllm and petals tests locally with passing results (they all require too much ram to run on the CI infra)

[mudler]

really liking it!