DocsLLM traffic lens

Inspect

LLM traffic lens

Pro feature. The LLM traffic lens needs Akev Pro or its 14-day trial. Without a licence, LLM calls are still captured and shown like any other request.

Apps and agents talk to LLM APIs all day, and the raw traffic is hard to read: a request is a wall of JSON with the whole conversation in it, and a streamed reply is hundreds of data: {"delta": …} events. Akev recognises calls to LLM APIs and shows them as what they are — a conversation.

Open an LLM call in Traffic: the response section starts with an LLM tab (selected by default; the tab you pick is remembered for LLM calls separately). It shows:

  • Header — provider, API, model (the one that answered, and the one requested when they differ), whether the response streamed, why generation stopped (Finished, Tool call, Token limit, Filtered… with the provider's own wording in the tooltip), and tiles for input / output / cache read / cache write tokens, TTFT (time to first token), speed (output tokens per second) and total time. Every other request parameter (temperature, max_tokens, reasoning_effort, thinking, tool_choice, response_format…) is a chip; structured ones open in a popover.
  • System prompt (collapsible) and Tools — every tool definition with its description; click one to see its JSON Schema.
  • Conversation — the messages sent, as chat bubbles: user turns on the right, assistant turns on the left with their thinking (collapsed, dimmed), tool calls as cards with the arguments pretty-printed, and tool results as cards labelled with the tool they answer (JSON results are formatted, errors are red). Inline images show as thumbnails (click to enlarge); remote image URLs are never loaded. Long texts and old turns of long agent loops fold away.
  • Response — the model's reply, highlighted: text (rendered as Markdown — toggle with the ¶ button), thinking, refusals and tool calls. Streamed responses are reassembled into the final message, and fill in live while the stream is still running.
  • API errors (rate limits, overloads, invalid requests) as a banner with the provider's error type and message, and a warning when a body was malformed or a stream ended early.

The copy button copies the whole conversation as Markdown (a readable transcript) or as JSON (the normalised model below); each message has its own copy button.

In the traffic list, LLM calls show LLM in the Type column, the LLM type chip in the toolbar shows only them, and the optional Model column (right-click a column header) shows the provider and model.

What is recognised

API Endpoints Streaming
OpenAI Chat Completions …/chat/completions on OpenAI, Azure OpenAI, OpenRouter, Groq, Mistral, DeepSeek, xAI, Together, Fireworks, Perplexity, Cerebras, Hugging Face, Anthropic's and Gemini's OpenAI-compatible endpoints, and any OpenAI-compatible server (Ollama, vLLM, LM Studio, llama.cpp, LiteLLM…) SSE data: chunks with deltas, tool-call argument fragments, reasoning_content / reasoning, the final usage chunk, [DONE]
OpenAI legacy Completions …/completions SSE choices[].text chunks
OpenAI Responses …/v1/responses response.* events (output_text.delta, function_call_arguments.delta, reasoning_summary_text.delta, response.completed…)
Anthropic Messages api.anthropic.com/v1/messages, any …/v1/messages that looks like it, Claude on Vertex AI (…/publishers/anthropic/models/…:rawPredict) message_start, content_block_start, content_block_delta (text_delta, input_json_delta, thinking_delta), message_delta, message_stop, error
Google Gemini generativelanguage.googleapis.com and Vertex AI …/models/{model}:generateContent / :streamGenerateContent SSE (?alt=sse) and the streamed JSON array
Ollama native /api/chat, /api/generate on port 11434 (or an ollama host) NDJSON lines; tokens/s from Ollama's own eval_count / eval_duration
Amazon Bedrock Converse bedrock-runtime.<region>.amazonaws.com/model/{model}/converse and /converse-stream (provider aws-bedrock): system prompt, messages of content blocks (text, images, documents, tool use and results, reasoning), tools, inference config, usage AWS's binary event stream (application/vnd.amazon.eventstream), exceptions included
Anthropic Messages on Bedrock bedrock-runtime.<region>.amazonaws.com/model/anthropic.…/invoke and /invoke-with-response-stream (InvokeModel with Claude models; the model id comes from the path) The event stream's base64 chunk payloads, decoded into Anthropic's own events

Calls to well-known provider hosts are recognised from their URL. On any other host (a local server, a gateway) the path must match and the request body must look like that API's (for example model + messages for Chat Completions), so an app's own /v1/messages endpoint isn't mistaken for Anthropic. Local servers are named by their default port: 11434 is Ollama, 1234 LM Studio, anything else OpenAI-compatible.

Recognition happens when the request is captured, so is:llm works on live traffic, imported HAR files, sessions and Charles / mitmproxy imports alike.

Filtering

is:llm                         # every LLM call
is:llm status:>=400            # rate limits, overloads, bad requests
provider:anthropic             # one provider (whole id; provider:*openai* for all OpenAI flavours)
provider:aws-bedrock           # Amazon Bedrock
model:gpt-4.1 duration:>5s     # slow calls to one model

See the filter language for everything else.

Numbers

  • Input tokens count the whole prompt, cached tokens included. (Anthropic reports uncached input separately; Akev adds its cache reads and writes so providers compare.) The tile shows the share served from the cache.
  • Output tokens include reasoning tokens; the tile shows how many were reasoning.
  • TTFT is measured from the moment the request was sent upstream to the arrival of the first stream event that carried generated content (text, thinking or a tool call). The proxy records every server-sent event with its arrival time, so this is the real latency your client saw — not available for non-streamed responses, nor for Bedrock's binary event streams, whose events aren't timestamped.
  • Speed is output tokens per second after the first token (over the whole response when it wasn't streamed); for Ollama, its own measurement.
  • Total runs from the request being sent to the response being complete.

API and MCP

GET /api/exchanges/:id/llm returns the normalised conversation (404 not-llm for other exchanges) — see API.md for the LlmExchange model. AI agents get the same through the MCP tool akev_llm_conversation { id, maxTextChars? } (long texts cut, inline image data left out), and see llm: { provider, model } on akev_list_exchanges rows.

Limitations

  • On Amazon Bedrock, only the Converse API and InvokeModel with Anthropic models are recognised: InvokeModel calls to other model families (their bodies are each model's own format) aren't. Cohere's native API isn't recognised either.
  • WebSocket realtime APIs (OpenAI Realtime, Gemini Live) show in the Messages tab as usual, not as a conversation.
  • Embeddings, model listing, batch and file endpoints aren't conversations and aren't tagged.
  • Token counts come from the provider's usage; when a provider doesn't report it (or the stream was cut before it), the tiles are left out rather than estimated.

Type to search guides, the CLI and the API.