-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for using Anthropic Claude models through Amazon Bedrock #2038
base: release
Are you sure you want to change the base?
Conversation
@@ -36,7 +36,9 @@ | |||
"ws": "^8.13.0", | |||
"yaml": "^2.3.4", | |||
"yargs": "^17.7.1", | |||
"yauzl": "^2.10.0" | |||
"yauzl": "^2.10.0", | |||
"@aws-sdk/client-bedrock-runtime": "^3.515.0", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this really necessary? We don't use official backend client packages but do direct HTTP calls instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it isn't. bedrock has a rest api
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, Bedrock do have a REST API, but I'd like to recommand using Bedrock SDK, cause:
- Bedrock has its own complicate authN mechanism(i.e. through SigV4),
- Customized wrapper over streaming data instead of SSE
There might be much more coding effort to do with REST API, thus offical SDK is a better choice.
Amazon Bedrock is a cloud service provided by AWS, it supports many Fundation Models, through this PR, user can use Anthropic's Claude models through Amazon Bedrock, in both streaming and non-streaming mode.