Skip to content

Your first chat call

By the end of this you'll have a command-line program that takes a question, sends it to Claude, and prints the answer — first in one go, then streaming token by token. Then you'll write a test for it that needs no API key and no network.

Allow about 30 minutes. The first cargo build pulls in a fair few crates and takes a couple of minutes on a cold cache.

Before you start

You'll need:

  • Rust 1.82 or newer. rustc --version will tell you.
  • An Anthropic API key, in ANTHROPIC_API_KEY. Steps 1 to 5 send real requests and cost real (small) money. Step 6 doesn't — if you haven't got a key, read through to step 6 and start there.

Create the project

cargo new first-chat
cd first-chat
cargo add rtb-chat tokio --features tokio/full
cargo add secrecy futures-util

That gives you the client, an async runtime to run it on, the secret wrapper the API key travels in, and the stream extension trait you'll need in step 5.

Build a client

AiClient::new takes a Config. Most of it has sensible defaults, so you set the provider, the model and the key and spread the rest.

Replace src/main.rs with:

use rtb_chat::{AiClient, ChatRequest, Config, Message, Provider};
use secrecy::SecretString;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let question = std::env::args().nth(1).unwrap_or_else(|| {
        "Explain what a merge request is, in two sentences.".to_string()
    });

    let client = AiClient::new(Config {
        provider: Provider::Anthropic,
        model: "claude-opus-4-7".into(),
        api_key: SecretString::from(std::env::var("ANTHROPIC_API_KEY")?),
        ..Config::default()
    })?;

    println!("client ready");
    Ok(())
}

Run it:

cargo run

client ready

Nothing has gone over the network yet. AiClient::new only checks the configuration — a missing key, an empty model, a base URL it doesn't like — so a mistake there fails immediately rather than on your first request.

Try it. Blank the variable and run again:

ANTHROPIC_API_KEY= cargo run

Error: InvalidConfig("api_key must not be empty")

That is the error you'll hit most often, and it always means the key never made it into the config.

main returning Result prints errors with Debug, which is why you get the variant name. Print it yourself with {e} and you get the friendlier form: invalid AI client config: api_key must not be empty.

Ask a question

A ChatRequest holds the conversation. Message::user builds a single user turn, and ..ChatRequest::default() fills in everything you haven't set.

Add this before Ok(()):

let response = client
    .chat(ChatRequest {
        messages: vec![Message::user(&question)],
        max_tokens: Some(512),
        ..ChatRequest::default()
    })
    .await?;

let answer = response.message.content[0].as_text().unwrap_or_default();
println!("{answer}");
println!(
    "\n[{} tokens in, {} out]",
    response.usage.input_tokens, response.usage.output_tokens
);

cargo run -- "What is a fast-forward merge?"

A fast-forward merge happens when the target branch has no commits that the
source branch doesn't already contain, so Git can simply move the branch
pointer forward instead of creating a merge commit. The result is a linear
history with no merge commit at all.

[19 tokens in, 61 out]

Set max_tokens even on your first call. Leave it out and the cap is 1024, which is generous for a sentence and much too small for anything long — and a reply that hits the cap just stops mid-word.

Give the model a role

The model does better work when it knows what it's for. That goes in system, not in the user message:

let response = client
    .chat(ChatRequest {
        system: Some("You answer questions about Git and GitLab. Be concise and concrete.".into()),
        messages: vec![Message::user(&question)],
        max_tokens: Some(512),
        ..ChatRequest::default()
    })
    .await?;

Keep the system prompt in that field rather than pushing a Message with Role::System into messages. On this backend a system-role message inside messages is sent as ordinary user input, which is not what you meant.

Stream the answer instead

A long answer arriving all at once feels slow. chat_stream gives you the tokens as the model produces them.

Swap the chat call for this:

use futures_util::StreamExt;
use rtb_chat::ChatStreamEvent;
use std::io::Write;

let mut stream = client
    .chat_stream(ChatRequest {
        system: Some("You answer questions about Git and GitLab. Be concise and concrete.".into()),
        messages: vec![Message::user(&question)],
        max_tokens: Some(512),
        ..ChatRequest::default()
    })
    .await?;

while let Some(event) = stream.next().await {
    match event {
        ChatStreamEvent::Token(t) => {
            print!("{t}");
            std::io::stdout().flush()?;
        }
        ChatStreamEvent::Done(_) => break,
        ChatStreamEvent::Error(e) => return Err(e.into()),
        _ => {}
    }
}
println!();

cargo run -- "What is a fast-forward merge?"

The answer appears a few characters at a time.

Three things there are easy to get wrong:

  • break on Done. The stream doesn't end when the answer does — it keeps yielding until the connection closes. Without the break, your program sits there looking finished but still running.
  • Flush after each token. Tokens are fragments, not lines, so a line-buffered stdout shows nothing until a newline turns up.
  • Keep the _ arm. ChatStreamEvent can gain variants in a minor release, so the compiler insists on it.

The Usage in Done is not worth reading. On the streaming path it comes back as zeros — a known defect, described in Streaming token counts are always zero. Use the non-streaming chat call when you need the numbers.

Test it without an API key

You don't want your test suite spending money, and you can't assert on what a model says anyway. What you can pin is everything either side of it: the request you build, and what your code does with the reply.

Point the client at a local mock server. Two fields make that work — base_url, and allow_insecure_base_url, which lets an http:// endpoint through the HTTPS check.

cargo add --dev wiremock serde_json url

Add tests/chat.rs:

use std::time::Duration;

use rtb_chat::{AiClient, ChatRequest, Config, Message, Provider};
use secrecy::SecretString;
use url::Url;
use wiremock::matchers::{header, method, path};
use wiremock::{Mock, MockServer, ResponseTemplate};

#[tokio::test]
async fn reads_the_reply_and_the_token_counts() {
    let server = MockServer::start().await;

    Mock::given(method("POST"))
        .and(path("/v1/messages"))
        .and(header("anthropic-version", "2023-06-01"))
        .and(header("x-api-key", "test-key"))
        .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({
            "id": "msg_01",
            "type": "message",
            "role": "assistant",
            "content": [{ "type": "text", "text": "hello, friend" }],
            "usage": { "input_tokens": 7, "output_tokens": 3 }
        })))
        .mount(&server)
        .await;

    let client = AiClient::new(Config {
        provider: Provider::Anthropic,
        model: "claude-opus-4-7".into(),
        base_url: Some(Url::parse(&server.uri()).unwrap()),
        api_key: SecretString::from("test-key".to_string()),
        timeout: Duration::from_secs(5),
        allow_insecure_base_url: true,
    })
    .expect("client builds");

    let response = client
        .chat(ChatRequest {
            messages: vec![Message::user("hi")],
            ..ChatRequest::default()
        })
        .await
        .expect("chat succeeds");

    assert_eq!(response.message.content[0].as_text(), Some("hello, friend"));
    assert_eq!(response.usage.input_tokens, 7);
    assert_eq!(response.usage.output_tokens, 3);
}

cargo test

running 1 test
test reads_the_reply_and_the_token_counts ... ok

No key, no network, and the header matchers prove your client really did send x-api-key and anthropic-version — which is the part you'd otherwise only find out about in production.

Leave allow_insecure_base_url in tests and nowhere else. It also switches off https_only on the HTTP client underneath, so a production client carrying it will send your key in plaintext to any host that asks.

Where to go next