Choose a provider¶

Talk to Anthropic Cloud¶

The default. Config::default() already selects it:

use rtb_chat::{AiClient, Config, Provider};
use secrecy::SecretString;

let client = AiClient::new(Config {
    provider: Provider::Anthropic,
    model: "claude-opus-4-7".into(),
    api_key: SecretString::from(std::env::var("ANTHROPIC_API_KEY")?),
    ..Config::default()
})?;

This is the backend that supports prompt caching, extended thinking, citations, temperature, max_tokens, timeout and typed auth and rate-limit errors. Prefer it when you have the choice.

Talk to an Anthropic-compatible gateway¶

Use Provider::AnthropicLocal and set base_url. Without the override it sends to api.anthropic.com, exactly like Provider::Anthropic:

use url::Url;

let client = AiClient::new(Config {
    provider: Provider::AnthropicLocal,
    model: "claude-opus-4-7".into(),
    base_url: Some(Url::parse("https://gateway.internal/anthropic")?),
    api_key: SecretString::from(std::env::var("GATEWAY_TOKEN")?),
    ..Config::default()
})?;

/v1/messages is appended to whatever you supply, with any trailing slash trimmed first.

This is also the only way to reach a self-hosted endpoint at all — see Point at an OpenAI-compatible endpoint.

Talk to OpenAI or Gemini¶

let client = AiClient::new(Config {
    provider: Provider::OpenAi,
    model: "gpt-4.1".into(),
    api_key: SecretString::from(std::env::var("OPENAI_API_KEY")?),
    ..Config::default()
})?;

Two things to know before you ship this:

• AiClient::new writes the key into the process environment as OPENAI_API_KEY (or GEMINI_API_KEY for Gemini). Build one client per process; a second client for the same provider overwrites the first one's key.
• temperature, max_tokens, timeout and base_url are all ignored on this path. Whatever you set, the provider's defaults apply.

Both are covered in What rtb-chat does not do.

Talk to a local Ollama¶

let client = AiClient::new(Config {
    provider: Provider::Ollama,
    model: "llama3".into(),
    // Required to be non-empty, never sent anywhere.
    api_key: SecretString::from("unused".to_string()),
    ..Config::default()
})?;

The endpoint is genai's default, http://localhost:11434/. base_url will not move it.

Point at an OpenAI-compatible endpoint¶

You cannot, in v0.7.2. Provider::OpenAiCompatible validates base_url and then ignores it, so the request goes wherever genai routes it from the model name — and an unrecognised name routes to Ollama on localhost. The full explanation is in A custom base_url is ignored on every genai-backed provider.

Your options today:

• If the endpoint speaks the Anthropic Messages format, use Provider::AnthropicLocal with base_url. That path honours the override.
• If it speaks the OpenAI format, use the vendor's own genai adapter by namespacing the model name — together::meta-llama/Llama-3-70b, groq::llama-3.1-70b. Config::api_key will not reach it: Provider::OpenAiCompatible writes the key to OPENAI_API_KEY, and those adapters read TOGETHER_API_KEY and GROQ_API_KEY, so you have to export the vendor's variable yourself. That reaches the vendor's public endpoint, not a private one.
• If it is a private OpenAI-format host, rtb-chat cannot reach it yet.

Check the request went where you meant¶

The routing surprise on the genai path is quiet: no error, just an answer from somewhere else, or a connection refused on localhost.

Two ways to confirm:

Read the INFO log. AiClient::new emits one line with the provider and the host it will use:

rtb-chat: AiClient ready (genai) provider=OpenAiCompatible host=openai-compatible

The literal openai-compatible is the tell that there is no real endpoint behind this client.

Read the error. genai names the adapter it chose when a call fails:

provider error: Web call failed for model 'llama-3.1-70b-instruct (adapter: Ollama)'.
Cause: Reqwest error: error sending request for url (http://localhost:11434/api/chat)

adapter: Ollama on a provider you did not set to Ollama means the model name matched nothing and fell through to the default.