Python
Chat

Chat

The ChatCompletion endpoint allows you to have a chat conversation with a model.

chat = anthropic.ChatCompletion.create(
    model="anthropic.claude-v2",
    messages=[
        {"role": "user", "content": "Hello!"},
        # { role: "system", content: "Hi how are you doing today." },
    ],
    max_tokens_to_sample=300
)

Usage

import os
from bedrock_anthropic import AnthropicBedrock
 
anthropic = AnthropicBedrock(
    access_key=os.getenv("AWS_ACCESS_KEY"),
    secret_key=os.getenv("AWS_SECRET_KEY"),
    region=os.getenv("AWS_REGION")
)
 
chat = anthropic.ChatCompletion.create(
    model="anthropic.claude-v2",
    max_tokens_to_sample=300,
    messages=[
        {"role": "user", "content": "Hello!"},
        # { role: "system", content: "Hi how are you doing today." },
    ]
)
 
print(chat["messages"])

Configuration

model

The model that will complete your prompt. Refer to the models page.

anthropic.ChatCompletion.create(
    model="anthropic.claude-v2",
    max_tokens_to_sample=300,
    messages=[
        {"role": "user", "content": "Hello!"},
        # { role: "system", content: "Hi how are you doing today." },
    ]
)

messages

The messages to chat with the model.

anthropic.ChatCompletion.create(
    model="anthropic.claude-v2",
    max_tokens_to_sample=300,
    messages=[
        {"role": "user", "content": "Hello!"},
        # { role: "system", content: "Hi how are you doing today." },
    ]
)

max_tokens_to_sample (optional)

The maximum number of tokens to generate before stopping.

  • Default: 256
  • Range depends on the model, refer to the models page.
 
anthropic.ChatCompletion.create(
    model="anthropic.claude-v2",
    max_tokens_to_sample=300,
    messages=[
        {"role": "user", "content": "Hello!"},
        # { role: "system", content: "Hi how are you doing today." },
    ]
)

stop_sequences (optional)

Sequences that will cause the model to stop generating completion text.

  • Default: []
anthropic.ChatCompletion.create(
    model="anthropic.claude-v2",
    max_tokens_to_sample=300,
    messages=[
        {"role": "user", "content": "Hello!"},
        # { role: "system", content: "Hi how are you doing today." },
    ],
    stop_sequences=[
        "sequence"
    ]
)

temperature (optional)

Amount of randomness injected in the response.

  • Default: 1
  • Range: 0-1
anthropic.ChatCompletion.create(
    model="anthropic.claude-v2",
    max_tokens_to_sample=300,
    messages=[
        {"role": "user", "content": "Hello!"},
        # { role: "system", content: "Hi how are you doing today." },
    ],
    temperature=0.7
)

top_p (optional)

Use nucleus sampling.

  • Default: 1
  • Range: 0-1
anthropic.ChatCompletion.create(
    model="anthropic.claude-v2",
    max_tokens_to_sample=300,
    messages=[
        {"role": "user", "content": "Hello!"},
        # { role: "system", content: "Hi how are you doing today." },
    ],
    top_p=0.7
)

top_k (optional)

Only sample from the top K options for each subsequent token.

  • Default: 250
  • Range: 0-500
anthropic.ChatCompletion.create(
    model="anthropic.claude-v2",
    max_tokens_to_sample=300,
    messages=[
        {"role": "user", "content": "Hello!"},
        # { role: "system", content: "Hi how are you doing today." },
    ],
    top_k=250
)
CompletionCount Tokens