Inspect
LLM traffic lens
Pro feature. The LLM traffic lens needs Akev Pro or its 14-day trial. Without a licence, LLM calls are still captured and shown like any other request.
Apps and agents talk to LLM APIs all day, and the raw traffic is hard to read: a request is a
wall of JSON with the whole conversation in it, and a streamed reply is hundreds of
data: {"delta": …} events. Akev recognises calls to LLM APIs and shows them as what they
are — a conversation.
Open an LLM call in Traffic: the response section starts with an LLM tab (selected by default; the tab you pick is remembered for LLM calls separately). It shows:
- Header — provider, API, model (the one that answered, and the one requested when they
differ), whether the response streamed, why generation stopped (Finished, Tool call,
Token limit, Filtered… with the provider's own wording in the tooltip), and tiles for
input / output / cache read / cache write tokens, TTFT (time to first token),
speed (output tokens per second) and total time. Every other request parameter
(
temperature,max_tokens,reasoning_effort,thinking,tool_choice,response_format…) is a chip; structured ones open in a popover. - System prompt (collapsible) and Tools — every tool definition with its description; click one to see its JSON Schema.
- Conversation — the messages sent, as chat bubbles: user turns on the right, assistant turns on the left with their thinking (collapsed, dimmed), tool calls as cards with the arguments pretty-printed, and tool results as cards labelled with the tool they answer (JSON results are formatted, errors are red). Inline images show as thumbnails (click to enlarge); remote image URLs are never loaded. Long texts and old turns of long agent loops fold away.
- Response — the model's reply, highlighted: text (rendered as Markdown — toggle with the ¶ button), thinking, refusals and tool calls. Streamed responses are reassembled into the final message, and fill in live while the stream is still running.
- API errors (rate limits, overloads, invalid requests) as a banner with the provider's error type and message, and a warning when a body was malformed or a stream ended early.
The copy button copies the whole conversation as Markdown (a readable transcript) or as JSON (the normalised model below); each message has its own copy button.
In the traffic list, LLM calls show LLM in the Type column, the LLM type chip in the toolbar shows only them, and the optional Model column (right-click a column header) shows the provider and model.
What is recognised
| API | Endpoints | Streaming |
|---|---|---|
| OpenAI Chat Completions | …/chat/completions on OpenAI, Azure OpenAI, OpenRouter, Groq, Mistral, DeepSeek, xAI, Together, Fireworks, Perplexity, Cerebras, Hugging Face, Anthropic's and Gemini's OpenAI-compatible endpoints, and any OpenAI-compatible server (Ollama, vLLM, LM Studio, llama.cpp, LiteLLM…) |
SSE data: chunks with deltas, tool-call argument fragments, reasoning_content / reasoning, the final usage chunk, [DONE] |
| OpenAI legacy Completions | …/completions |
SSE choices[].text chunks |
| OpenAI Responses | …/v1/responses |
response.* events (output_text.delta, function_call_arguments.delta, reasoning_summary_text.delta, response.completed…) |
| Anthropic Messages | api.anthropic.com/v1/messages, any …/v1/messages that looks like it, Claude on Vertex AI (…/publishers/anthropic/models/…:rawPredict) |
message_start, content_block_start, content_block_delta (text_delta, input_json_delta, thinking_delta), message_delta, message_stop, error |
| Google Gemini | generativelanguage.googleapis.com and Vertex AI …/models/{model}:generateContent / :streamGenerateContent |
SSE (?alt=sse) and the streamed JSON array |
| Ollama native | /api/chat, /api/generate on port 11434 (or an ollama host) |
NDJSON lines; tokens/s from Ollama's own eval_count / eval_duration |
| Amazon Bedrock Converse | bedrock-runtime.<region>.amazonaws.com/model/{model}/converse and /converse-stream (provider aws-bedrock): system prompt, messages of content blocks (text, images, documents, tool use and results, reasoning), tools, inference config, usage |
AWS's binary event stream (application/vnd.amazon.eventstream), exceptions included |
| Anthropic Messages on Bedrock | bedrock-runtime.<region>.amazonaws.com/model/anthropic.…/invoke and /invoke-with-response-stream (InvokeModel with Claude models; the model id comes from the path) |
The event stream's base64 chunk payloads, decoded into Anthropic's own events |
Calls to well-known provider hosts are recognised from their URL. On any other host (a local
server, a gateway) the path must match and the request body must look like that API's
(for example model + messages for Chat Completions), so an app's own /v1/messages
endpoint isn't mistaken for Anthropic. Local servers are named by their default port: 11434 is
Ollama, 1234 LM Studio, anything else OpenAI-compatible.
Recognition happens when the request is captured, so is:llm works on live traffic,
imported HAR files, sessions and Charles / mitmproxy imports alike.
Filtering
is:llm # every LLM call
is:llm status:>=400 # rate limits, overloads, bad requests
provider:anthropic # one provider (whole id; provider:*openai* for all OpenAI flavours)
provider:aws-bedrock # Amazon Bedrock
model:gpt-4.1 duration:>5s # slow calls to one modelSee the filter language for everything else.
Numbers
- Input tokens count the whole prompt, cached tokens included. (Anthropic reports uncached input separately; Akev adds its cache reads and writes so providers compare.) The tile shows the share served from the cache.
- Output tokens include reasoning tokens; the tile shows how many were reasoning.
- TTFT is measured from the moment the request was sent upstream to the arrival of the first stream event that carried generated content (text, thinking or a tool call). The proxy records every server-sent event with its arrival time, so this is the real latency your client saw — not available for non-streamed responses, nor for Bedrock's binary event streams, whose events aren't timestamped.
- Speed is output tokens per second after the first token (over the whole response when it wasn't streamed); for Ollama, its own measurement.
- Total runs from the request being sent to the response being complete.
API and MCP
GET /api/exchanges/:id/llm returns the normalised conversation (404 not-llm for other
exchanges) — see API.md for the LlmExchange model. AI agents
get the same through the MCP tool akev_llm_conversation { id, maxTextChars? } (long texts cut,
inline image data left out), and see llm: { provider, model } on akev_list_exchanges rows.
Limitations
- On Amazon Bedrock, only the Converse API and InvokeModel with Anthropic models are recognised: InvokeModel calls to other model families (their bodies are each model's own format) aren't. Cohere's native API isn't recognised either.
- WebSocket realtime APIs (OpenAI Realtime, Gemini Live) show in the Messages tab as usual, not as a conversation.
- Embeddings, model listing, batch and file endpoints aren't conversations and aren't tagged.
- Token counts come from the provider's
usage; when a provider doesn't report it (or the stream was cut before it), the tiles are left out rather than estimated.