Anthropic Bedrock

SDKs for Anthropic's models on AWS Bedrock

By

Mustafa Aljadery

&

Siddharth Sharma

Building using AWS Bedrock is an exercise left to the reader. It's very difficult to get AWS Bedrock to work. Our goal with this SDK is to change this.

This SDK handles all of the AWS auth for you (the difficult part), and provides a Completion and ChatCompletion functions to interact with Claude. There are a couple other helper functions as well for a better LLM developer experience.

There are two SDKs, one for Python and the other for NodeJS. The goal is to make it extremely easy to build using Anthropic's models. Anthropic's models have amazing features not available in other models, an example is the 100k context length. You can upload an entire book to Claude and chat with it.

PythonTypescript

Features

This SDK supports all of these features out the box:

  • Text completion
  • Chat with context history
  • Count tokens & estimate price
  • Streaming
  • Async functions

Python Overview

In this example we use the Python SDK to

import os
from bedrock_anthropic import AnthropicBedrock
 
anthropic = AnthropicBedrock(
    access_key=os.getenv("AWS_ACCESS_KEY"),
    secret_key=os.getenv("AWS_SECRET_KEY"),
    region=os.getenv("AWS_REGION")
)
 
completion = anthropic.Completion.create(
    model="anthropic.claude-v2",
    prompt="In one sentence, what is good about the color blue?",
)
 
print(completion["completion"])

Typescript Overview

In this example we use the Typescript SDK to get started with Claude 2 on AWS Bedrock.

import AnthropicBedrock from "anthropic-bedrock";
 
const anthropic = new AnthropicBedrock({
  access_key: process.env["AWS_ACCESS_KEY"],
  secret_key: process.env["AWS_SECRET_KEY"],
  region: process.env["AWS_REGION"]
});
 
async function main() {
    const completion = await anthropic.Completion.create(
        model: "anthropic.claude-v2",
        prompt: "In one sentence, what is good about the color blue?"
    )
 
    console.log(completion["completion"])
}
 
main();

Credits

Check out the following for updates: