Merged
Conversation
…ied management script
- Moved `flash-attn` to optional `[advanced]` dependencies in pyproject.toml. - Updated installation scripts to first install core dependencies, then server dependencies, and finally attempt advanced dependencies without failing the installation. - Enhanced README to clarify the installation steps and dependency management.
- Added "flash-attn" to the main dependencies in pyproject.toml. - Revised README to clarify the installation order of dependencies. - Simplified the installation script to first install PyTorch, followed by the Magma package with all dependencies.
Contributor
Author
@microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new API server for the Magma multimodal model, enabling vision and language processing via REST endpoints. The changes include server implementation, deployment options, and updates to dependencies and documentation.
API Server Implementation:
server/main.py) to provide endpoints for health checks, predictions using base64-encoded images, and file uploads. It integrates the Magma model for vision and language tasks.server/magma-server.sh) to simplify running the server directly, deploying with Docker, or setting up as a system service.server/manage_magma_service.sh) for installing, starting, stopping, and managing the API server as a systemd service.Deployment Options:
server/docker/Dockerfile) for containerized deployment of the API server, including CUDA support for GPU acceleration.docker-compose.ymlfile (server/docker/docker-compose.yml) for managing the Docker-based deployment with GPU support.Dependency and Documentation Updates:
pyproject.tomlto include server-specific dependencies likefastapi,uvicorn, andpython-multipart.README.mdwith a new "API Server" section, detailing the server's features, deployment options, and usage instructions.server/README.mdwith comprehensive documentation on the API server's functionality, setup, and deployment methods.