Capture MCP tool calls in PostHog - #128
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b1a9ccf. Configure here.
| "manage_api_keys", | ||
| "manage_credentials", | ||
| "manage_proxies", | ||
| ]); |
There was a problem hiding this comment.
Credential tool allowlist incomplete
High Severity
CREDENTIAL_TOOLS omits manage_auth_connections, whose fields map is the same arbitrary name-to-value pattern as manage_credentials values. Key-name redaction cannot cover those entries, so login/MFA submit parameters can still ship to PostHog in $mcp_parameters.
Reviewed by Cursor Bugbot for commit b1a9ccf. Configure here.


Summary
Instruments the MCP server with PostHog MCP analytics (
@posthog/mcp, pinned to0.10.1) so every inboundtools/call,tools/list, andinitializeis captured as a$mcp_*event — tool name, arguments, latency, error state, and client name/version.What that buys us: per-tool call volume, error rate and latency percentiles, which clients connect, and which advertised tools never get called. Today we ship tool changes blind — kernel API traffic tagged
X-Source: mcp-servershows endpoints, not tools.src/lib/mcp/analytics.ts— module-scopeposthog-nodeclient plusinstrument()wiring.identifyattributes events to the Clerk user id when the request carries a JWT; API-key requests stay session-scoped rather than getting a made-up identity.src/app/[transport]/route.ts— instrument inside thecreateMcpHandlercallback, and drain the queue withafter()so the flush runs once the response is already on the wire. Capture adds no latency to a tool call..env.example—POSTHOG_PROJECT_TOKENandPOSTHOG_HOST. With no token set the whole thing is a no-op, so deploys and local dev without PostHog credentials are unaffected (dev logs a one-time error so the miss isn't silent).POSTHOG_PROJECT_TOKENandPOSTHOG_HOSTneed to be set per environment before any events show up.No payloads leave the server
Every tool serializes its result through
jsonResponse, so a response reaches the SDK as a single JSON string and its key-name redaction can't see inside it — a newly created API key, a TOTP code, or a CDP URL would be captured verbatim.beforeSendtherefore:$mcp_responseon every event. Call metadata is what the analytics are for; response bodies aren't worth that risk on a public server.$mcp_parametersformanage_api_keys,manage_credentials, andmanage_proxies, whose arguments carry arbitrary field/value maps (values: { username, pin, ... }) that key-name matching can't cover either.Deliberately left off
instrument()can inject a requiredcontextargument into every tool schema to capture what the agent was trying to do ($mcp_intent). It's the most interesting signal, but it changes a public tool surface — a required field plus a 15-25 word instruction on all 17 toolsets — so this ships withcontext: false. Worth turning on as its own change once the basic numbers are landing.The SDK is pre-1.0 (
0.x), so event names and properties can change in minor releases — hence the exact-version pin.Verification
bunx tsc --noEmitpasses (what CI runs).$mcp_initialize,$mcp_tools_list, and both a successful and a failing$mcp_tool_callarrive with latency and error flag populated, and withcontext: falsethe tool schemas come back unmodified.valuesmap, and a CDP URL: no$mcp_responseon any event, no$mcp_parameterson the credential tools, parameters intact onmanage_browsers, and no canary string anywhere in the captured events.bun run buildfails locally on this branch and onmainalike (page data collection for/authorizewithout Clerk/Redis env), so that failure is unrelated.