Get structured output¶

Ask for a type instead of prose¶

Define the type you want, derive Deserialize and JsonSchema, and call chat_structured::<T>():

use rtb_chat::{AiClient, ChatRequest, Message};
use schemars::JsonSchema;
use serde::Deserialize;

#[derive(Debug, Deserialize, JsonSchema)]
struct Analysis {
    severity: String,
    fix: String,
}

async fn run(client: &AiClient, log_text: &str) -> Result<(), rtb_chat::AiError> {
    let analysis: Analysis = client
        .chat_structured(ChatRequest {
            messages: vec![Message::user(format!("Analyse this log:\n{log_text}"))],
            max_tokens: Some(1024),
            ..ChatRequest::default()
        })
        .await?;

    println!("{} — {}", analysis.severity, analysis.fix);
    Ok(())
}

rtb-chat generates the JSON Schema from Analysis, appends it to the system prompt as a hard instruction, validates the reply against that schema, and only then deserialises. A reply that does not match the schema never becomes a partial Analysis — it becomes an error.

Add schemars and serde to your own Cargo.toml; they are rtb-chat dependencies but not re-exported.

Keep your own system prompt¶

Your system string is kept as the prefix, with the schema instruction appended after it:

use rtb_chat::{ChatRequest, Message};
let req = ChatRequest {
    system: Some("You are a terse SRE. Prefer the smallest safe fix.".into()),
    messages: vec![Message::user("…")],
    ..ChatRequest::default()
};

So the model reads your instructions first, then the schema requirement. Anything you need to override — tone, language, how to handle a missing field — goes in system and will still be read.

Raise max_tokens before you need it¶

On the Anthropic-direct path an unset max_tokens means 1024. A structured type with more than a few fields, or one with a free-text field in it, will run into that, and a truncated reply is not valid JSON.

Truncation does not announce itself. It arrives as:

response was not valid JSON for the requested type: EOF while parsing a string at line 1 column 1024

Set max_tokens to something comfortably larger than the JSON you expect.

Read the two failure modes¶

They mean different things and want different fixes:

AiError::Deserialize — the reply was not JSON at all, or serde rejected it. In practice: a truncated answer, or a model that wrapped its output in a ```json fence despite being told not to. Nothing strips fences for you. Raise max_tokens, or restate the requirement in your own system prompt.

AiError::SchemaValidation — the reply was valid JSON but did not match the schema. A field is missing, or has the wrong type. The message names the offending path. Usually the type is stricter than the question: an age: u32 against a model that answered "forty".

Prefer simple types¶

The schema goes into the prompt, so the type is not free — a deeply nested type spends tokens on every call and gives the model more ways to get the shape wrong.

Flat structs of strings, numbers, booleans and small enums work best. If you need something richer, ask for the flat version and build the rich one yourself.

Do not combine it with extended thinking¶

chat_structured calls the non-streaming path, which keeps only the text blocks of a response. Thinking blocks are dropped, so a thinking budget buys you nothing here except tokens and latency — and if the budget exceeds max_tokens, the request is rejected outright.