Two independent selectors: which backend generates text (llm_provider), and
which model turns text into vectors (embedding_provider). Mixing them is
normal — remote generation with local embeddings is the cheap default, because
retrieval then costs nothing.
Everything on this page is about llm_provider. A remote chat model is not a
reason to change embedding_provider; the one case that is, and what it costs,
are in
configuring a remote embedding provider by hand.
This page says which settings each backend needs. What each setting means, and its default, is in configuration.
llm_provider |
Speaks | Use for | Extra |
|---|---|---|---|
vllm |
OpenAI wire, local server | A model you serve yourself on an AMD GPU | chat-cli |
openai |
OpenAI wire | OpenAI, OpenRouter, Groq, Together, Fireworks, DeepSeek, Mistral, xAI, and any gateway speaking the same protocol | chat-cli |
litellm |
LiteLLM’s routing | Anthropic, Gemini and Bedrock, whose wire protocols are not OpenAI-compatible; also Azure OpenAI Service | chat-all |
chat-all is chat-ui plus LiteLLM and nothing else; see
installation for what each extra costs.
Adding a backend is a new module under aorta/chat/inference/providers/
exposing name, get_chat_model(), preflight(), probe(),
unreachable_hint() and describe(), plus one entry in that package’s
factory. No node, graph or tool code changes.
preflight() and probe() are separate because they answer to different
callers. A session’s preflight may be permissive – the local vLLM backend waits
up to five minutes for a server that is still loading weights and then starts
anyway, rather than refuse a warm-up the user can see progressing. probe() is
the raising counterpart, on a short budget, and is what aorta chat doctor
calls: a diagnostic that inherits the permissive behaviour reports a healthy
backend when there is none. unreachable_hint() supplies the text printed
instead of a traceback when a query fails to connect.
Only vllm reaches the network to answer probe(). The two remote backends
validate configuration and stop there, because a reachability check against a
metered endpoint means billing the operator for running doctor – so a remote
provider whose key is valid and whose endpoint is down still reports ok, and
the failure surfaces on the first query as unreachable_hint().
llm_provider = "vllm"
vllm_base_url = "http://localhost:8000/v1"
vllm_model = "deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct"
Preflight polls the server’s /health and warns rather than failing, so a
server that is still loading does not abort the session.
aorta agent mitigate --llm-backend vllm reads this same profile, so one
served model backs both the chatbot and the probe agent.
Start the engine with its Qwen3 reasoning parser, so the thinking goes to
reasoning_content and content carries only the answer. On TokenSpeed, also
pick a sampling backend that honours temperature: on AMD GPUs its default,
greedy, ignores temperature, top_p and seed without an error.
vllm serve Qwen/Qwen3-8B --reasoning-parser qwen3
tokenspeed serve Qwen/Qwen3-8B --reasoning-parser qwen3 --sampling-backend triton
# add --grammar-backend xgrammar if requests use JSON mode
<think>…</think> block
(Qwen3.8’s chat template opens it in the prompt, so only </think> appears).
The probe agent drops a terminated block before reading the JSON, so replies
parse either way. A reply cut off before its answer is recorded as an
unparseable response and ends the search; Qwen3-8B thinks for about
1,000–1,250 tokens a step and Qwen3.8-27B for about 4,000–4,700, so a
server-side output cap below that ends it early.enable_thinking: false in
chat_template_kwargs, or a no-think template) makes every reply parse, but on
the agent’s prompt Qwen3-8B’s first proposal named a mitigation that fixes the
failure 19–21% of the time, against 40–43% with thinking on; many replies gave
up at unknown and proposed nothing.response_format JSON schema, which --llm-backend litellm
sends on an [agent]-only install) needs the parser. Without it the grammar
applies from the first token while the template has thinking on. Under the
earlier bare {"type": "json_object"}, 12–14% of Qwen3-8B’s first replies
listed 15 or more mitigation names
(#510); the schema now caps
next_mitigations at five, so one reply can no longer take the whole menu,
but the grammar still applies before any thinking. With the parser the model
thinks before the grammar applies, and none showed the tail (0 of 448 first
replies from a Qwen3-8B fine-tune that showed it without the parser). The
server has to accept json_schema response formats — vLLM, TokenSpeed with a
grammar backend, and OpenAI’s gpt-4o-mini and later do. On
TokenSpeed, JSON mode combined with enable_thinking: false and the parser
parsed only 3–4% of replies (vLLM was unaffected); aorta’s clients never send
that combination.llm_provider = "openai"
remote_llm_model = "gpt-4o-mini"
remote_llm_api_key = "sk-..."
# Only for a non-OpenAI endpoint:
# remote_llm_base_url = "https://openrouter.ai/api/v1"
Azure OpenAI is not OpenAI-wire-compatible: it rewrites the URL path to
/openai/deployments/<deployment>/chat/completions and requires an
api-version query parameter, neither of which the openai backend can
express. Route it through LiteLLM, which builds that path only for a model
prefixed azure/:
aorta chat config init --profile azure-openai
or by hand:
llm_provider = "litellm"
remote_llm_model = "azure/<your-deployment-name>"
remote_llm_base_url = "https://<resource>.openai.azure.com"
remote_llm_api_version = "<the api-version your endpoint documents>"
remote_llm_api_key = "<your key>"
The same settings cover a corporate gateway that exposes only the Azure deployment path. Three details decide whether the request lands:
azure/ in front, not the
underlying model name. Without the prefix, LiteLLM treats the base URL as an
OpenAI-compatible endpoint and posts to <base>/chat/completions, which such
a gateway answers with 404 Resource not found
(#556)./openai. LiteLLM appends
/openai/deployments/... itself, so a base URL ending in /openai requests
/openai/openai/deployments/....remote_llm_api_version. Left empty, LiteLLM reads
AZURE_API_VERSION and otherwise falls back to its own default, which is a
preview version a gateway may not serve.The key goes out in Azure’s api-key header. A gateway that wants it in a
header of its own, such as API Management’s Ocp-Apim-Subscription-Key, takes
remote_llm_auth_header as below. With remote_llm_api_key and
remote_llm_base_url empty, LiteLLM falls back to AZURE_API_KEY and
AZURE_API_BASE.
aorta chat doctor warns, and the startup log says the same, when
llm_provider = "litellm" has a base URL and a bare model LiteLLM will send
to /chat/completions (an OpenAI model name such as gpt-4o-mini), or a
bare name LiteLLM does not recognise, or an azure/ model whose base URL
ends in /openai. A bare name LiteLLM already routes natively, such as
claude-sonnet-4-5, is not warned about: that request uses the provider’s
own path (/v1/messages for Anthropic), not /chat/completions. Neither
check makes a request. A failure inside that check is recorded at debug and
ignored: it is advice, and it must not turn doctor into a failure or stop
chat from starting.
Launch, Watch, and Autopsy send this azure/<deployment> model,
remote_llm_api_version, and the same gateway headers as chat
(remote_llm_auth_header and remote_llm_extra_headers). They otherwise
pin their own Qwen model, which has no provider prefix and would be posted
to /chat/completions. The same applies to any other LiteLLM provider
prefix on a base URL (anthropic/..., and the rest): that configured name
is the route. An openai/ model still selects the OpenAI-compatible route,
and the agents keep Qwen there.
llm_provider = "litellm"
remote_llm_model = "claude-sonnet-4-5"
Leave remote_llm_api_key empty and LiteLLM reads ANTHROPIC_API_KEY,
GEMINI_API_KEY and friends itself. Setting it is what makes the gateway flow
below work on this backend.
Current Claude Opus builds accept only temperature=1 and LiteLLM raises rather
than negotiating. The graph asks for 0.0 and 0.1, so this backend enables
LiteLLM’s drop_params — which means routing and criticism are less
deterministic on those models than on one that honours temperature.
remote_llm_api_key on its own assumes the key travels as
Authorization: Bearer <key>, which is what public providers expect. Corporate
gateways often disagree. Five families, and only the first three are “put this
string somewhere static”, which is why one header mechanism covers them:
| Family | How the key travels | Seen on | Supported by |
|---|---|---|---|
| Bearer token | Authorization: Bearer <key> |
OpenAI, OpenRouter, Groq, Together, Fireworks, DeepSeek, Mistral, xAI, vLLM | openai, out of the box |
| Custom static header | One named header | Azure API Management (Ocp-Apim-Subscription-Key), Azure OpenAI (api-key), Anthropic direct (x-api-key), Gemini (x-goog-api-key) |
openai + remote_llm_auth_header |
| Query parameter | ?key=<key> |
Google AI Studio, legacy endpoints | openai, by putting it in remote_llm_base_url |
| OAuth2 / token exchange | Short-lived bearer, refreshed | Entra ID for Azure OpenAI, Vertex AI service accounts, IBM watsonx | litellm |
| Request signing | Signature over the request | AWS Bedrock (SigV4) | litellm |
The last two need refresh or signing logic, which is deliberately not
reimplemented here — that is what the litellm backend is for.
llm_provider = "openai"
remote_llm_base_url = "https://gateway.example.com/openai"
remote_llm_model = "gpt-oss-20b"
remote_llm_api_key = "<your subscription key>"
remote_llm_auth_header = "Ocp-Apim-Subscription-Key"
remote_llm_extra_headers = { user = "your-username" }
llm_tool_mode = "native"
Two details that surprise people:
user header here is attribution and quota, not authentication. That is
why extra headers are a separate setting from the auth one — a JSON blob
holding a live credential is far easier to leak into a log line, a pasted
config or a PR diff than a lone remote_llm_api_key. Keeping them apart is
also what lets the startup line name the auth header while guaranteeing no
value is printed.remote_llm_api_key. When remote_llm_auth_header
is set, the bearer slot receives the throwaway string unused, because the
OpenAI client rejects an empty api_key even when the gateway ignores it.Preflight reports the header name and never the key:
Using remote OpenAI-compatible -- gpt-oss-20b at https://gateway.example.com/openai (auth: Ocp-Apim-Subscription-Key header, plus user)
A gateway may also expose a native Anthropic route on a different path of the
same host, speaking Anthropic’s protocol rather than OpenAI’s — so finding one
tells you nothing about the other. curl the path with /v1/messages: if it
answers, use llm_provider = "litellm" with an anthropic/-prefixed model
name, not openai.
The agent is agentic, not a single completion, so one question fans out:
| Path | Calls |
|---|---|
| Question (route → retrieve → answer) | 2 |
| Action, first pass (route → plan → retrieve → act → critic) | 3 + up to max_act_rounds, plus one synthesis call if the loop is exhausted |
| Each critic rejection | Replays act + critic; max_retry_iterations caps act passes, so the shipped 3 allows two replays |
A search-shaped action query can therefore reach 12 calls in one pass, and
all three passes reach 32 — or 34 when it is also the query that
escalates to native, which pays two text rounds once (see Automatic
escalation to native for the per-case
breakdown). Most action queries
land in the 4–6 range in practice, because the act loop stops as soon as the
model answers without a tool call and the critic usually accepts first time.
With embedding_provider = "remote", each retrieval and each search_code call
adds one embedding call;
configuring a remote embedding provider by hand
has the full cost.
Against a metered endpoint that is real money, so the remote backends log the
per-query call count at INFO, visible without --verbose:
aorta.chat.inference.callcount INFO Remote LLM calls for this query: 7
It is a process-wide total read as a before/after delta, so concurrent UI sessions inflate each other’s numbers — a spend indicator, not an accounting record. The local backend does not attach the counter.
Knobs that lower the bill, roughly in order of effect:
| Setting | Effect |
|---|---|
max_act_rounds_search / max_act_rounds |
Hard cap on the most expensive loop. Lowering the search budget to 3–4 is the single biggest saving. |
max_retry_iterations |
0 removes the critic’s multiplier on everything above. |
llm_max_tokens |
Caps output tokens per call. |
retriever_k / search_tool_k |
Fewer chunks means a smaller prompt, and prompt tokens dominate a long act loop. |
llm_max_retries |
Lower it on an unreliable endpoint, so failures do not silently triple. |
embedding_provider = "local" |
Keeps all retrieval free even when generation is remote. Already the case unless you set it by hand. |
remote_llm_model |
A smaller model in the same family is usually the cheapest change of all. |
The agent’s act loop needs the model to call tools. Two protocols, chosen by
llm_tool_mode:
text (default) |
native |
|
|---|---|---|
| How | The model writes ACTION: search_code(query="x") and AORTA parses it |
The provider’s function-calling API returns structured tool_calls |
| Endpoint requirement | None | Must accept the tools parameter |
| Local vLLM | Works as shipped | Needs --enable-auto-tool-choice and a --tool-call-parser |
| Reasoning models | Does not work | Works |
A reasoning model puts its working in a separate channel and returns empty
content when it wants to act. It never writes the ACTION: line, so the parse
finds nothing, the loop re-prompts, and the query ends with no answer having
spent the whole retry budget. Measured against gpt-oss-20b on one search
query: 0 parseable actions in 8 rounds, 11 billed calls, no answer. The same
query in native mode drove 8 real tool calls and returned a complete answer.
The symptom is distinctive — finish_reason is stop, output tokens are
non-zero, and content is empty — and the logs say so:
act_node ... produced no text despite N output tokens.
nativeAORTA now acts on that signature rather than only logging it. On seeing it,
the query is retried once through native; if that retry answers, the protocol
is kept for the rest of the process and one log line says so. The default stays
text, because flipping it globally would break a stock local vLLM (see the
endpoint row in the table above) — this is detection, not a new default.
Five things follow from that:
llm_tool_mode is never overridden, whether it comes from
the profile file or AORTA_CHAT_LLM_TOOL_MODE. If you set text
deliberately, set it; the escalation only ever moves the built-in default.The wasted spend is bounded; the retry itself is not capped to one call.
What is bounded is the cost of failure: the escalated retry tolerates a
single empty reply rather than the usual two, so a model that is silent on
native as well costs exactly one extra call. A retry that is working —
calling tools and getting results — is not truncated; it runs the ordinary act
loop under the ordinary round budget, because cutting a productive loop off at
one round would spend the call and throw away the answer it was about to
reach.
So the cases have different ceilings, and only the first is the escalation’s
own bill. Counted as whole turns — act always flows to critic, so a
figure that stops at the act loop is a subtotal, not a cost:
| Escalated turn | Calls | Made up of |
|---|---|---|
| Silent on both protocols | 6 | router, plan, 2 text rounds, 1 escalated native round, 1 retrieval-fallback answer — the critic short-circuits on the empty command_output and spends nothing |
| Native drives tools, critic accepts | 14 search, 11 otherwise | the same 4-call prefix, then MAX_ACT_ROUNDS_SEARCH (8) or MAX_ACT_ROUNDS (5) native rounds, 1 synthesis call, 1 critic validation |
| Native drives tools, critic rejects every pass | 34 search | three act passes — MAX_RETRY_ITERATIONS (3) counts passes, not hand-backs, so the critic returns to act twice — with the second and third starting on native and paying no text rounds |
The escalation does not raise the round budget; it reaches the ordinary one on a query that had already spent two rounds proving it needed to. And because the protocol moves once per process rather than once per query, those two text rounds are paid once and not on every later query — the same rejected-every- pass turn costs 32 once the protocol has already moved.
text for good. Only a failure with no tool call
behind it counts, which is the shape a refused tools payload actually has.
The same applies when the retry’s calls were all duplicates of ones the
text loop had already run: the duplicate guard below answers them from the
recorded results, so nothing new is executed and the retry can end with no
fresh results and no prose. It emitted structured tool_calls to get there,
which is the whole question being asked, so it moves the protocol and spends
no failure. The log for it says the calls were repeats rather than claiming
the query was answered, because on this path it was not.
The budget also counts probes, not requests in flight: two ui sessions
that both fail inside the same outage spend one failure between them, because
neither was a second probe — nothing was tried in between, so the pair says
nothing the first failure did not. Two failures with a completed attempt
between them still write native off, which is the case the budget is for.enable_shell_tool it is a
side effect that does not happen twice.aorta chat that is
the same thing, but aorta chat ui serves many browser sessions from one
server, and there the escalation is shared by all of them. That is deliberate:
what was detected is a property of the model — it emits reasoning instead of
an ACTION: line — so it is equally true for every session talking to that
endpoint, and sharing it means only the first query in the process pays the
wasted round. Nothing about a conversation is carried across; the state is a
flag and a counter about protocol support. Sessions that dead-end at the same
moment each get their own retry — the shared state records the outcome, it
does not ration the attempt.Three places name it. The LLM backend: line logged at startup names the
protocol alongside the provider. The escalation logs a line of its own when it
fires, and that one names the protocol itself rather than pointing at the
startup banner, so it stands on its own wherever it is read — including in a
server log where the startup line has scrolled away. And aorta chat doctor’s
tool-mode line carries a hint naming the configured protocol and what it costs.
For local vLLM in native mode, doctor also adds an llm native tools
check that sends a forced function call and validates its returned name and
arguments. Remote providers remain configuration-only so a diagnostic does
not spend a billable request.
aorta chat ui names it too, on each session’s welcome banner and in the same
LLM backend: ... (tool protocol: ...) line in the server log
(#468). Both are written when a
browser session opens rather than once when the server starts, because the
escalation is process-wide: a session that opens after it is told native, with
a note that the server switched from the configured text, instead of the
protocol the server started on.
When the act loop gives up, it makes one tool-free attempt to answer from the
context retrieve already gathered, and labels that answer as having used no
tools. This is independent of the escalation to native and worth stating
separately, because the two are often confused: it fires whenever the loop
abandons with no tool run, including under an explicitly configured
llm_tool_mode = "text", where the escalation is refused outright and no native
retry is attempted at all. So an action-routed question to a reasoning model
does not come back empty-handed even when the protocol never moves.
That label applies only when no tool ran at all. Once one has, “I could not use my tools” would be untrue, so the same single call is made with a different label, chosen by what the tools returned (#475):
| What ran before the loop gave up | What the one tool-free call answers from | Label |
|---|---|---|
| Nothing | Retrieved context | I could not use my tools for this question |
| Only tool calls that failed | Retrieved context | The tools I tried for this question failed |
| At least one tool that returned a result | Those results, appended to the retrieved context | I stopped before I finished looking into this — plus some of the tools I tried failed when any did |
The third row covers a loop that ran tools and then went quiet, and an escalated native retry whose backend fell over after its tool calls: the call that would have summarised the results is the one that failed, so this call takes its place. Its answer is told the investigation stopped early and must not present itself as complete. A failure is what the tool itself reported as one — an error, a refused or missing path, a command that exited non-zero — so a failed command’s output counts as a failure here even though it is in the trace.
What the first row covers is a model that can drive neither protocol, and an
endpoint that refuses the escalated one (a stock local vLLM without
--enable-auto-tool-choice and a matching --tool-call-parser does): that
refusal never moves the protocol — the switch is thrown only once native has
answered, so there is nothing to roll back — and it lands on this same fallback
rather than escaping as an error. It takes two such failures to write native
off for the rest of the process, so one transient timeout does not disable the
escalation; the warning names which attempt it was.
aorta chat doctor reports the resolved mode as its own check, and warns before
you spend a query on it when text is paired with a model whose name reads as a
reasoning one — a locally served one as much as a remote one, since the channel
is the model’s rather than the endpoint’s. A deployment can be served under any
name, so for a name the check does not recognise — or a provider with no model
name to read — it still names native, and what turning it on costs here, in
the hint under the line it prints. Under native the same line names what the
endpoint has to accept. A local vLLM first answers /health, then must pass the
llm native tools request; the remote backends are not called at all — their
probe() is a configuration preflight, deliberately, so that a diagnostic
cannot bill you for a round trip. The startup line names the resolved protocol
alongside the provider.
A query still ends with no answer when that one call also fails or returns
empty content — rarer than it sounds, since the reporter’s transcript shows the
same question answered correctly through the question route in 2 calls while
the action route returned nothing in 4, because the empty-content behaviour
belongs to the tool protocols and not to the model. Whatever tools ran are
still recorded on the turn’s trace.
Both protocols run the same tools, retrieval and critic, and both are guarded the same way: an empty reply is never used as the answer, unproductive rounds are capped at two, a repeated identical tool call is answered with “you already asked that” rather than re-run, an unknown or protocol-mangled tool name returns an error the model can read instead of aborting the request, and the final synthesis call runs with no tools bound (offered tools, a model that has not found what it wants keeps calling them and returns no prose).
| Symptom | Cause and fix |
|---|---|
LLM backend unavailable: unknown LLM provider: 'gpt4' |
llm_provider must be vllm, openai or litellm. |
remote_llm_api_key is not set, and llm_provider=openai requires it |
Set the key, or export AORTA_CHAT_REMOTE_LLM_API_KEY. |
llm_provider=litellm needs both litellm and langchain-litellm |
pip install 'amd-aorta[chat-all]'. The lazy import cannot tell which of the two is missing, so it names both. |
401 / 403 from a gateway whose key works in curl |
The gateway wants a named header, not a bearer token. Set remote_llm_auth_header. |
Access denied due to missing subscription key |
Azure API Management’s wording for the same thing: remote_llm_auth_header = "Ocp-Apim-Subscription-Key". |
Incorrect API key provided: unused from platform.openai.com |
remote_llm_auth_header is set but remote_llm_base_url is empty, so the request went to OpenAI. The preflight line says at the provider default endpoint when this is wrong. |
404 on an *.openai.azure.com endpoint |
Azure OpenAI needs the litellm backend, not openai. |
litellm.exceptions.NotFoundError: ... Resource not found on llm_provider = "litellm" |
The model has no azure/ prefix, so the request went to <base>/chat/completions and the gateway only serves Azure deployment paths. Set remote_llm_model = "azure/<deployment>" and remote_llm_api_version; see Azure OpenAI Service. aorta chat doctor flags this configuration as a warning. |
remote_llm_model is 'azure/', which names no Azure deployment |
The azure-openai profile’s model prompt was accepted unchanged. Put the deployment name after azure/. |
missing_keys: ['AZURE_API_VERSION', ...] |
Set remote_llm_api_version, or export the AZURE_* variables LiteLLM reads. |
I wasn't able to answer that -- check the warning logged... |
The act loop produced no usable text. Which attempts it made first depends on llm_tool_mode and on whether a tool had already run, so read the warning logged beside this message — it names the step that gave up. aorta chat doctor covers the configuration faults that reach this message by other routes. |
this process will use native from here |
Not an error. A round under text returned neither text nor a tool call — the line names the signature that was observed — and native then proved the protocol works, so chat switched for the rest of this process. It is a claim about that round, not about the query: earlier rounds may have run tools successfully, their results are on the turn’s trace, and the retry is seeded with them. The same line is logged whether native answered the query or only drove structured tool calls without answering it — the clause before the comma says which, and only the first means this query got a reply. Set llm_tool_mode yourself to pin it either way. |
The escalated native tool-calling request failed ... without making a tool call |
The retry was tried and the request did not come back, having called nothing. If it is a local vLLM, it needs both --enable-auto-tool-choice and a matching --tool-call-parser (see the endpoint row in the table above); otherwise the endpoint may simply have been unwell. The protocol does not move, and the line says which attempt it was — after the second, native is not tried again in this process. Set llm_tool_mode = "text" to skip the attempt entirely. |
The escalated native tool-calling request failed before it could call anything |
The same outcome, one step earlier: the backend could not even be built or the tool schemas could not be bound, so no request was made. Counts as an attempt in the same way. Read the exception named on the line — this is a backend or configuration fault, not a protocol one. |
... drove structured tool calls and then failed |
Structured tool calling works on this endpoint and the backend fell over afterwards. So the protocol does move to native, and this deliberately does not count against the two-failure budget — otherwise two transient errors after working tool calls would strand the process on text. What those calls gathered is recorded on the turn, and since the request that would have turned it into an answer is the one that failed, one tool-free call answers from it instead, labelled I stopped before I finished looking into this (#475). |
The escalated native tool-calling request returned no answer and no tool call either |
The retry reached the endpoint and the model was as silent on native as it was on text, so the protocol is not what it is failing on. text stays in force, and this counts as one of the two attempts above. Nothing here is a configuration fault; the model cannot drive either protocol for this query. |
An answer prefixed I could not use my tools for this question |
The act loop gave up without any tool having run, so the answer came from retrieved context alone and anything needing a live lookup is missing from it. The prefix does not identify why, and it is not always the row above: it is also reached when llm_tool_mode is explicitly text so no escalation is allowed, when the two-failure budget is already spent and native was never proved to work (once it has been, a spent budget no longer refuses the retry), and when the native retry failed before it could call anything. Read the warning logged beside the answer — that line, not this prefix, names the cause. |
An answer prefixed The tools I tried for this question failed |
The same fallback, after every tool the loop ran had reported a failure. The lookup was attempted, so look at the tool rather than the model: the failures are on the turn’s trace. |
An answer prefixed I stopped before I finished looking into this |
The act loop gave up after at least one tool had returned a result, and the answer was built from those results in one tool-free call. Treat it as partial. The warning logged beside it says why the loop stopped. |
Many Act round N: ... re-prompting lines and no answer |
Same cause. Set llm_tool_mode = "native". |
Waiting for vLLM at ... when you meant to go remote |
llm_provider is still vllm. Check the backend line printed at startup. |
| The call-count line never appears | Expected on llm_provider = "vllm"; only the remote backends attach the counter. |
extra header #N is missing '=' |
remote_llm_extra_headers takes name=value pairs or a JSON object. #N is the position in the comma-separated list, counted from 1 — the entry itself is not quoted back, because a header value may be a credential. |