maple-agent-tracing-cloudflare-agents

Maple agent tracing: Cloudflare Agents SDK

Goal

One chat (one agent instance) = one Maple Agent Session, one turn per user message, with the transcript, every model call (model, tokens), every tool call (name, args, result, failures), and a lane per sub-agent.

Known gaps (tell the user, don't try to fix): cost shows as "unpriced" (the AI SDK emits no cost). These traces are separate from Cloudflare's native Workers traces (different trace ids); that's expected.

Step 0: Detect

agents in package.json; chat agents extend AIChatAgent from @cloudflare/ai-chat (or the older agents/ai-chat-agent), others extend Agent from agents. Find every class and every AI SDK call site in them: streamText(, generateText(, generateObject(, streamObject(, new ToolLoopAgent(, .generate(, .stream(.
ai version: >= 7.0.106 required. On 5.x/6.x ask the user to upgrade (npx @ai-sdk/codemod v7). ai@7 also forces agents >= 0.23, @cloudflare/ai-chat >= 0.11, @ai-sdk/react@^4, workers-ai-provider@^4 (and @ai-sdk/openai|anthropic@^4): upgrade them in the same install. On ERESOLVE, regenerate the lockfile. The upgrade can break unrelated agent code (e.g. chatRecovery = true now needs as const); typecheck and tell the user. Don't use experimental_telemetry / metadata (v6 API).
Models via workers-ai-provider (createWorkersAI({ binding: env.AI })), @ai-sdk/openai, @ai-sdk/anthropic, AI Gateway: fine at the ^4 majors above; the spans come from the AI SDK, not the provider.
Agents that call a model without the AI SDK (raw env.AI.run(...), fetch to a provider, @tanstack/ai): this skill doesn't cover them. Use the maple-agent-tracing-opentelemetry skill to write the spans by hand with the same tracer provider from Step 2.
wrangler.jsonc/wrangler.toml must have compatibility_flags: ["nodejs_compat"] (Agents SDK projects always do; the context manager needs AsyncLocalStorage).
Existing OpenTelemetry in the Worker: search for @microlabs/otel-cf-workers (instrument(, instrumentDO(), BasicTracerProvider, WebTracerProvider, registerTelemetry(, @sentry/cloudflare, @langfuse/otel, braintrust.
- registerTelemetry(...) already exists: extend that call; never call it twice.
- An existing tracer provider (including otel-cf-workers): add one Maple span processor/exporter to it and pass tracer: provider.getTracer("gen_ai") to OpenTelemetry; don't build a second provider. Still add the per-turn flush from Step 4 (untested whether otel-cf-workers' own flush covers AIChatAgent turns, which run over WebSocket messages and finish after the handler returns). Don't add otel-cf-workers to a project that doesn't have it: this setup doesn't need it (last release May 2025).
- Cloudflare's native observability.traces destinations (Workers Observability) can stay; they export runtime spans (fetch, bindings, DO calls, and the Agents SDK's own cloudflare.agents.* spans) but never AI SDK spans, because @ai-sdk/otel uses @opentelemetry/api, which the native tracer doesn't back.

Step 1: Key and region

US https://ingest.maple.dev/v1/traces, EU https://ingest.eu.maple.dev/v1/traces. The exporter takes the full URL (/v1/traces included). Header authorization: Bearer <key>.
Key from the user's prompt; no key: use the literal MAPLE_TEST (ingest accepts and discards it) and tell the user to replace it with their key from Settings → Ingestion.
Store it as a Worker secret: npx wrangler secret put MAPLE_INGEST_KEY, and MAPLE_INGEST_KEY=<key> in .dev.vars for wrangler dev (check .dev.vars is gitignored). Re-run the repo's own type generation afterwards if it uses generated Env types (its types script, e.g. npm run types, or npx wrangler types <file>; the output may be worker-configuration.d.ts, env.d.ts...); otherwise add MAPLE_INGEST_KEY: string to the Env interface.
An unset secret becomes Bearer undefined and every export 401s with no other hint: confirm the key is in .dev.vars (and npx wrangler secret list for deploys) before the verification run.
A 401 ingest_unauthorized / "Invalid ingest key" with a key you trust usually means the key belongs to the other region (keys are region-bound): try the other endpoint.
Follow the repo's existing secret naming if it has one.

Step 2: Install and create the tracer provider

npm install ai@^7.0.106 @ai-sdk/otel @opentelemetry/api @opentelemetry/sdk-trace-base @opentelemetry/exporter-trace-otlp-http @opentelemetry/resources @opentelemetry/context-async-hooks

Use the repo's package manager. telemetry.ts next to the agent classes:

import { OpenTelemetry } from "@ai-sdk/otel"
import { context, diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api"
import { AsyncLocalStorageContextManager } from "@opentelemetry/context-async-hooks"
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http"
import { resourceFromAttributes } from "@opentelemetry/resources"
import { BasicTracerProvider, BatchSpanProcessor } from "@opentelemetry/sdk-trace-base"
import { registerTelemetry } from "ai"
import { env } from "cloudflare:workers"

// surfaces export failures
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.ERROR)
context.setGlobalContextManager(new AsyncLocalStorageContextManager().enable())

// A missing key disables export; it never stops the Worker.
if (!env.MAPLE_INGEST_KEY) console.warn("MAPLE_INGEST_KEY is not set; Maple telemetry export is disabled")

export const tracerProvider = new BasicTracerProvider({
	resource: resourceFromAttributes({
		"service.name": "support-agent",
		"deployment.environment.name": "production",
	}),
	spanProcessors: env.MAPLE_INGEST_KEY
		? [
				new BatchSpanProcessor(
					new OTLPTraceExporter({
						url: "https://ingest.maple.dev/v1/traces",
						headers: { authorization: `Bearer ${env.MAPLE_INGEST_KEY}` },
					}),
				),
			]
		: [],
})

registerTelemetry(
	new OpenTelemetry({
		tracer: tracerProvider.getTracer("gen_ai"),
		usage: true,
		runtimeContext: true,
	}),
)

service.name: the Worker's name from the Wrangler config unless the user wants something else. deployment.environment.name: take it from an existing env var if the repo has environments; else production.
Import tracerProvider from ./telemetry in every module that defines an agent class (that import also runs registerTelemetry at module load). Registration only has to happen before the first AI SDK call, which is always at request time.
import { env } from "cloudflare:workers" at module scope works (secrets included). If the repo's compatibility date or style prevents it, build the provider lazily on first use from this.env instead, once per isolate (module-level let), never per request.
tracer: tracerProvider.getTracer("gen_ai"): the scope name must stay gen_ai (Maple's Vercel AI SDK detection keys on scope gen_ai/ai). Alternative: trace.setGlobalTracerProvider(tracerProvider) and omit tracer; both verified.
The context manager is required for sub-agents: without it, a generateText inside a tool's execute starts a new trace with no conversation id, which Maple shows as a separate trace:<id> session. With it, the sub-agent's invoke_agent nests under the parent's execute_tool (verified).
Keep usage: true (adds the reasoning-token breakdown, ai.usage.*) and runtimeContext: true (records the included runtime context keys; Maple reads the conversation id from them).
The exporter resolves to its browser build under wrangler/esbuild (workerd/browser conditions), which posts application/json via fetch with keepalive; Maple ingest accepts OTLP JSON. Don't switch to exporter-trace-otlp-proto (pulls Node transports) or gRPC (not supported by ingest).
Don't use @opentelemetry/sdk-node, NodeSDK, NodeTracerProvider or OTEL_EXPORTER_OTLP_* env vars: the Node SDK doesn't run on Workers and the browser exporter doesn't read env vars.
Don't set attribute length limits: they cut the message JSON.

Step 3: Conversation id

The agent instance name is the conversation id: this.name (the DO name from useAgent({ agent, name }), getAgentByName(ns, name), or the URL /agents/<class>/<name>).

Every AI SDK call in the agent must pass both:

runtimeContext: { conversationId: this.name },
telemetry: { functionId: "support_agent", includeRuntimeContext: { conversationId: true } },

AIChatAgent (the common case):

import { AIChatAgent } from "@cloudflare/ai-chat"
import { convertToModelMessages, streamText } from "ai"
import { tracerProvider } from "./telemetry"

export class ChatAgent extends AIChatAgent<Env> {
	async onChatMessage() {
		const result = streamText({
			model,
			messages: await convertToModelMessages(this.messages),
			tools,
			runtimeContext: { conversationId: this.name },
			telemetry: { functionId: "support_agent", includeRuntimeContext: { conversationId: true } },
		})
		return result.toUIMessageStreamResponse()
	}

	protected async onChatResponse() {
		await tracerProvider.forceFlush()
	}
}

Check the client: useAgent({ agent: "ChatAgent" }) without name connects every user to the default instance, so all chats share one DO, one message history and one session. That's an app bug beyond tracing; tell the user and pass a per-chat id as name if they agree.
If the app keeps several conversations inside one agent instance (for example one agent per user with a thread list), use the thread id instead of this.name (from options.body of onChatMessage(onFinish, options) if the client sends it, or from the app's own state). It must be stable for the whole conversation and differ between conversations.
If runtimeContext already exists, add conversationId to it and to includeRuntimeContext; keep other keys excluded (they may hold secrets).
ToolLoopAgent inside an agent: add callOptionsSchema: z.object({ conversationId: z.string() }) + prepareCall: ({ options, ...rest }) => ({ ...rest, runtimeContext: { conversationId: options.conversationId } }) + the telemetry above, and call it with options: { conversationId: this.name }.
Tool approvals / client-side tools: the continuation after the approval or client tool result is a new onChatMessage call (options.continuation === true) and a new trace. It uses the same this.name, so it lands in the same session as an extra turn. onChatResponse fires for it too.
Sub-agents called from a tool's execute: give each its own functionId; it doesn't need the conversation id (the context manager nests it). Agents SDK sub-agents that are separate agent instances (this.subAgent(...), getAgentByName from inside a tool) are reached over RPC; don't rely on trace context crossing that boundary (untested). Pass the parent's conversation id as an RPC argument, use it in the child's runtimeContext, and flush in the child. Expect the child's calls as separate turns of the same session rather than a nested lane.

Step 4: Flush

A DO with hibernatable WebSockets (every AIChatAgent) can be evicted between messages; the BatchSpanProcessor's 5 s timer is not guaranteed to fire. Flush at the end of every turn.

AIChatAgent: protected async onChatResponse() { await tracerProvider.forceFlush() } (the base declares it protected). It runs after the stream finished and the assistant message was persisted, so every span of the turn has ended (verified: all of a turn's spans have arrived by the end of the turn; turns over 5 s may span several POSTs). If the class already overrides onChatResponse, add the flush at its end.
Agent methods that call a model (onRequest, onMessage, @callable() methods, schedule callbacks, onEmail, queue/workflow steps): wrap in try { ... } finally { this.ctx.waitUntil(tracerProvider.forceFlush()) }, or await tracerProvider.forceFlush() in the finally if the method returns after the stream is fully read.
Streams returned to the client from an Agent.onRequest or a Worker fetch (toUIMessageStreamResponse() / toTextStreamResponse()): the spans end only after the stream has been read, after any finally ran. Use this.ctx.waitUntil(result.consumeStream().then(() => tracerProvider.forceFlush())) before returning the response (verified: one POST with every span, client still receives the full stream). Flushing from streamText({ onFinish }) or onEnd is too early: the root invoke_agent span ends after those callbacks and misses the flush.
Worker fetch handlers (outside a DO) that call a model without streaming the reply: ctx.waitUntil(tracerProvider.forceFlush()) after building the response.
Never shutdown() the provider per request; it's shared by the isolate.

Step 5: Content, tools, errors

Content (prompts, replies, tool args/results) is recorded by default. Only set recordInputs: false / recordOutputs: false in a call's telemetry if the user asks; the transcript is empty for those calls.
Tool failures must throw from execute to show as failed. If a tool returns { error } for real failures, change it to throw (ask first if the model relies on the payload).
Set a distinct telemetry.functionId on every call/agent (snake_case). It becomes gen_ai.agent.name. The class name isn't exported.
Don't pass telemetry.integrations on a call unless it includes the OpenTelemetry instance: it replaces the global integrations for that call.
Images/files in messages are recorded as base64 on every step; if exports fail with 413, suggest recordInputs: false on those calls.

Step 6: Verify

Local, with no Maple key needed: run a throwaway OTLP receiver (e.g. a Bun.serve on 127.0.0.1:<free port> that appends each POST /v1/traces JSON body to a file), point the exporter url at http://127.0.0.1:<port>/v1/traces temporarily, npx wrangler dev, and drive the agent:

AIChatAgent without a browser: open ws://127.0.0.1:8787/agents/<kebab-class-name>/<name> and send {"type":"cf_agent_use_chat_request","id":"<uuid>","init":{"method":"POST","body":"{\"messages\":[{\"id\":\"m1\",\"role\":\"user\",\"parts\":[{\"type\":\"text\",\"text\":\"...\"}]}]}"}}; the turn is done at the cf_agent_use_chat_response frame with that id and done: true. Turn 2+ must send the full history: the last cf_agent_chat_messages broadcast's messages plus the new user message, with "trigger":"submit-message" in the body. The server replaces its stored history with what it receives, so sending only the new message drops turn 1.
Put this in a small driver script: one conversation id (instance name), 2+ turns, at least one tool call.
Model without keys: MockLanguageModelV4 from ai/test with a doStream that returns a tool-call chunk when the last prompt message isn't a tool result, then text.

Check in the received spans: scope gen_ai; per turn one trace with invoke_agent (root) → step n → chat + execute_tool; ai.settings.context.conversationId = the instance name on invoke_agent, same across turns and different per chat; gen_ai.agent.name = the functionId on invoke_agent (only there); gen_ai.input.messages on turn 2 contains turn 1's history; gen_ai.tool.call.arguments / gen_ai.tool.call.result on execute_tool; gen_ai.usage.input_tokens / output_tokens on every chat; all of a turn's spans have arrived by the end of the turn (flush works; turns over 5 s may span several POSTs); a sub-agent's invoke_agent has the parent's execute_tool as parent. Restore the Maple URL afterwards.

Then against Maple: the wrangler dev output shows no export errors (the diag logger prints 401 / export response failure lines); silence without the local receiver check proves nothing (no spans also looks silent). With the Maple MCP, list_agent_sessions with search=<instance name> returns one row; get_agent_session (or the Agent Sessions page, ~30 s after a run) shows one session per chat named after the instance name, framework Vercel AI SDK, turns = user messages (plus approval continuations), transcript with prompts/replies/tool calls, tokens on every model call, cost unpriced (expected), no attribute containing the provider API key or Bearer .

Do not

Do not add a second AI SDK tracer that also exports to Maple (Langfuse/Braintrust/Sentry AI integrations): every model call gets recorded twice.