What rtb-chat does not do¶

Documentation that only describes what software does leaves you to infer the rest from silence. This page states the gaps.

Every gap here is labelled deliberate: an omission that is not going to change soon, with the reason. A gap where the crate does not do what its own API implies is a defect, and it belongs in the issue tracker as well as on this page. There are none listed at the moment. Each section describes what the crate actually does, not what it is meant to do.

Only text content is supported¶

Deliberate, for now. ContentBlock has one variant, Text. Images, audio, tool use and function calling have no representation in the request, and any non-text block in a response is dropped during parsing rather than surfaced.

ContentBlock and Provider are both #[non_exhaustive] so that adding those variants later is not a breaking change.

No tool use, no function calling, no agent loop¶

Deliberate. rtb-chat sends a request and returns a reply. It does not execute tools, does not loop, and does not manage conversation state. The messages vector you pass is the whole history, and keeping it is your job.

genai's tool-call stream chunks are received and thrown away, which is what makes this a hard boundary rather than a partial implementation.

No retries, no backoff, no circuit breaker¶

Deliberate. A 429 becomes AiError::RateLimited carrying retry_after when the provider sent a parseable Retry-After header, and that is the extent of it. Nothing sleeps and nothing retries.

Retry policy belongs to the caller because the right policy depends on whether a human is waiting. Note that AiError::RateLimited never occurs on a genai provider (see below), so a retry loop keyed on that variant covers only the two Anthropic providers.

genai providers report every failure as a provider error¶

Deliberate consequence of the two-backend split, worth knowing about. The status-code mapping that produces AiError::Auth and AiError::RateLimited lives in rtb-chat's own Anthropic path. genai surfaces failures through its own error type, and all of them are wrapped as AiError::Provider.

A bad OpenAI key does not give you AiError::Auth. It gives you AiError::Provider with genai's description inside.

The genai backend puts your key in the process environment¶

Deliberate, and the crate's one unsafe block. genai resolves provider credentials from environment variables. To hand it the key from Config, AiClient::new calls std::env::set_var: OPENAI_API_KEY for Provider::OpenAi and Provider::OpenAiCompatible, GEMINI_API_KEY for Provider::Gemini.

Three consequences follow, none of them obvious from the type signature:

• The key outlives the client. It stays in the environment for the life of the process and is readable by anything in it, including code that dumps the environment into a crash report.
• The last client wins. Build a second AiClient for the same provider with a different key and the first client starts using the second key, because both read the same variable at call time.
• Constructing clients concurrently is a data race. std::env::set_var is unsafe on modern Rust editions for exactly this reason. The crate assumes the common pattern of one client built once per process; building clients from several threads while other threads read the environment is undefined behaviour.

The Anthropic-direct path does not do this. It sends the key as a header and touches no environment variable. Neither does Provider::Ollama, which sets nothing.

An API key is required even when the provider does not need one¶

Deliberate, and awkward. AiClient::new rejects an empty api_key for every provider, Provider::Ollama included. A local Ollama needs no credential, so you have to invent one:

api_key: SecretString::from("unused".to_string()),

The check is one rule applied uniformly rather than a per-provider table. It costs a placeholder string; leaving it out would let a genuine missing-key mistake reach the provider as a confusing 401.

Config cannot be deserialised¶

Deliberate. Config derives Debug and Clone and nothing else. There is no Deserialize, so a config file cannot be mapped onto it directly. The tool layer reads its own settings type and constructs a Config in code.

That is what keeps allow_insecure_base_url out of reach of anything but Rust source, which is the property that makes the HTTPS enforcement worth having. See Why the endpoint is validated.

One model per client, and no fallback¶

Deliberate. Config::model is fixed at construction. There is no per-request override, no fallback chain, and no "try the cheap model first" behaviour. Build a second AiClient for a second model, bearing in mind the environment-variable caveat above if both are genai-backed.

anthropic-version cannot be changed¶

Deliberate. The Anthropic-direct path sends anthropic-version: 2023-06-01 from a compile-time constant. A Messages API feature gated behind a newer version string needs a crate release, not a config change.

Structured output is one attempt, with no repair¶

Deliberate. chat_structured asks for bare JSON, parses what comes back, validates it against the schema and deserialises it. If any of those fails, you get the error. There is no retry with a corrective prompt, no fence-stripping, no partial extraction.

The most common failure is a reply truncated by the 1024-token default max_tokens, which arrives as AiError::Deserialize rather than anything that mentions length.

Where defects are recorded¶

This page used to list four defects on the genai path: a custom base_url and Config::timeout were ignored, temperature and max_tokens were dropped, and streamed token counts came back as zero on both backends. They were fixed under rust/chat#7. A new defect gets its own issue and a section here, marked as such, until it is fixed.