Choose a provider¶

Talk to Anthropic Cloud¶

The default. Config::default() already selects it:

use rtb_chat::{AiClient, Config, Provider};
use secrecy::SecretString;

let client = AiClient::new(Config {
    provider: Provider::Anthropic,
    model: "claude-opus-4-7".into(),
    api_key: SecretString::from(std::env::var("ANTHROPIC_API_KEY")?),
    ..Config::default()
})?;

This is the backend that supports prompt caching, extended thinking, citations and typed auth and rate-limit errors. Prefer it when you have the choice.

Talk to an Anthropic-compatible gateway¶

Use Provider::AnthropicLocal and set base_url. Without the override it sends to api.anthropic.com, exactly like Provider::Anthropic:

use url::Url;

let client = AiClient::new(Config {
    provider: Provider::AnthropicLocal,
    model: "claude-opus-4-7".into(),
    base_url: Some(Url::parse("https://gateway.internal/anthropic")?),
    api_key: SecretString::from(std::env::var("GATEWAY_TOKEN")?),
    ..Config::default()
})?;

/v1/messages is appended to whatever you supply, with any trailing slash trimmed first.

Talk to OpenAI or Gemini¶

let client = AiClient::new(Config {
    provider: Provider::OpenAi,
    model: "gpt-4.1".into(),
    api_key: SecretString::from(std::env::var("OPENAI_API_KEY")?),
    ..Config::default()
})?;

One thing to know before you ship this: AiClient::new writes the key into the process environment as OPENAI_API_KEY (or GEMINI_API_KEY for Gemini). Build one client per process; a second client for the same provider overwrites the first one's key. See What rtb-chat does not do.

Talk to a local Ollama¶

let client = AiClient::new(Config {
    provider: Provider::Ollama,
    model: "llama3".into(),
    // Required to be non-empty, never sent anywhere.
    api_key: SecretString::from("unused".to_string()),
    ..Config::default()
})?;

The endpoint is genai's default, http://localhost:11434/. To reach Ollama on another host, set base_url, and allow_insecure_base_url: true if it serves plain http://.

Point at an OpenAI-compatible endpoint¶

Use Provider::OpenAiCompatible and set base_url to the endpoint's base, the part before chat/completions:

let client = AiClient::new(Config {
    provider: Provider::OpenAiCompatible,
    model: "meta-llama/Llama-3-70b-chat-hf".into(),
    base_url: Some(Url::parse("https://api.together.xyz/v1")?),
    api_key: SecretString::from(std::env::var("TOGETHER_API_KEY")?),
    ..Config::default()
})?;

The request goes to https://api.together.xyz/v1/chat/completions through genai's OpenAI adapter, whatever the model is called. The key travels the same way as for Provider::OpenAi: through OPENAI_API_KEY in the process environment.

Without base_url there is no endpoint to send to, and genai falls back to routing on the model name. The next section shows how to spot that.

Check the request went where you meant¶

On the genai path with no base_url, routing is quiet: no error, just an answer from somewhere else, or a connection refused on localhost.

Two ways to confirm:

Read the INFO log. AiClient::new emits one line with the provider and the host it will use:

rtb-chat: AiClient ready (genai) provider=OpenAiCompatible host=openai-compatible

The literal openai-compatible is the tell that there is no real endpoint behind this client.

Read the error. genai names the adapter it chose when a call fails:

provider error: Web call failed for model 'llama-3.1-70b-instruct (adapter: Ollama)'.
Cause: Reqwest error: error sending request for url (http://localhost:11434/api/chat)

adapter: Ollama on a provider you did not set to Ollama means the model name matched nothing and fell through to the default.