feat(doctor): notice when Claude usage cannot be recorded at all - #129
Merged
Conversation
Every Claude figure TokenTracker reports is parsed out of the session transcripts under ~/.claude/projects. A CLI started with --no-session-persistence writes none, so those calls cost real tokens and contribute zero — and the only symptom is a source quietly reporting less than it should, which is indistinguishable from a quiet day. That is how this was found: a gateway-routed stream stopped being counted on 2026-07-28 and nothing said so. `doctor` stayed green the whole time. The detector is live rather than historical. It reads the process list once and answers "is something running right now that I cannot see", so it needs no baseline, no threshold, and no per-source history. Source-level silence detection was tried first and rejected. Every Claude model aggregates under source=claude, so the source read as active 0.4h ago while the affected model had been silent 37h — it would have missed the exact incident it was written for. At (source, model) granularity it does fire, but on the one machine it was tested against it produced three false positives for two true ones: a user retiring a model looks identical to a stream breaking. Parked rather than shipped; #128 records the replay. Two states are kept distinct on purpose. "Checked and found nothing" is [OK]; "could not read the process list" is a warning; and a platform that cannot answer at all — no /bin/ps — omits the check rather than printing [OK] for something that never ran. What leaves the detector is a count, model ids, and a coarse reason. No pid, argv, or environment value: the local API answers unauthenticated loopback GETs, and a command line can carry a user's file paths. The tests assert that absence by serialising the result and searching it, rather than by inspection. Model ids are included deliberately — they are already first-class tracked data and they are the one field that says *which* stream is unobservable — and are charset-clamped so no arbitrary command-line text can ride out through them. `degraded` joins `ok` on the doctor report. `ok` still governs the exit code and only `critical` moves it, so no existing caller changes behaviour; `degraded` is true whenever any check warns. A warning otherwise leaves an entirely green-looking report, which is what let this stay invisible. parseProcessLine moves to src/lib/process-list.js so both callers share one copy; usage-limits.js keeps its behaviour and its tests. Refs: #128
|
To use Codex here, create an environment for this repo. |
Owner
Author
QA gate — independent 7-gate passQA agent: Codex Read this first: merging publishes 0.40.0This PR carries
(b) because of the upheld Edge Case gap below: no test drives the real Gates
Verified as passing: version Two fixes before merge, both cheap
Follow-ups (not blocking)
On issue #128Directly related, but it does not resolve #128. The PR deliberately rejects #128's ranked proposal #1 Evidence gaps
|
`ps -ax` lists every user's processes. transcript-suppression serves counts and model ids derived from that output over `GET /functions/tokentracker- ingest-health`, which is unauthenticated, so on a multi-user host the endpoint answered questions about other people's Claude sessions. Verified on the machine that found this: `-ax` spans 40 distinct users including a second human account, `-x` returns 518 lines against 750 and still lists every Claude process this user is running. `-x` keeps processes with no controlling terminal, which is what the check needs, and loses nothing: a session TokenTracker could not have recorded is by definition one this user is running. Blast radius is contained to this module — `grep -rn PS_ARGS src/ test/` returns only process-list.js, and usage-limits.js:1354 keeps its own literal array (still `-ax`; filed as a follow-up, not touched here because nothing in this PR tests that path). Adds test/process-list.test.js, which the module did not have. Two of its cases were checked by breaking the code and watching them fail, not by watching them pass: - test/process-list.test.js:23 asserts the literal argv rather than observed output, because a real `ps` run on a single-user machine looks identical either way. Re-introducing `-ax`: 1 fail, then 5 pass on revert. - test/process-list.test.js:71 drives the default `cp.spawnSync` path with no injected runner, and :79 asserts `ok === true`, so a broken real invocation fails a suite in which every other case fakes `ps`. Pointing PS_BINARY at a non-existent path: 1 fail, then 5 pass on revert.
pitimon
force-pushed
the
feat/128-transcript-suppression-detector
branch
from
July 30, 2026 03:58
6e38d1c to
908b4d8
Compare
Owner
Author
Both pre-merge items from the QA gate are addressed, and the release bump is droppedForce-pushed (rebase). The branch is now three commits and no longer carries a version bump, so merging this
On the scope fix
Blast radius stayed inside the module: The two new tests were checked by breaking the code, not by watching them pass
Suite counts after the change: Still open from the review — none of it blocking
|
added 2 commits
July 30, 2026 11:36
The notice was `role="note"`, a static landmark. It is inserted after an async
fetch resolves — no user action, no focus move — so a screen reader announces
it only if the user later navigates onto it. The one class of user who cannot
see an amber box appear was also the one never told about it, on a notice
whose entire purpose is to say "your tokens are not being counted".
`role="status"` + `aria-live="polite"` queues the announcement behind whatever
is being read instead of interrupting.
The existing five `ByRole("note")` assertions move to `"status"`.
IngestHealthNotice.test.jsx:83 is new and guards the live-region attribute
separately from the role, because the role alone is what regressed.
Dashboard suite 309 passed / 41 files, run as part of `ci:local` — package.json:26
includes `npm --prefix dashboard run test` in the chain.
`dashboard/src/ui/components/DismissibleHint.jsx:55` also uses `role="note"` and
is left alone here; whether it is inserted asynchronously, and so has the same
problem, is unchecked.
The scoping decision from the previous commit lived only in a code comment and a commit body. openwiki/local-api.md is where a reader goes to learn what this endpoint discloses, and it described the payload shape without saying whose processes the payload describes — the question that matters on a shared host. Also names `src/lib/usage-limits.js` as a separate `ps` scan that still passes `-ax`, so a future reader does not read the new sentence as a repo-wide property. `docs:openwiki:check` reports 0 findings.
pitimon
force-pushed
the
feat/128-transcript-suppression-detector
branch
from
July 30, 2026 04:37
908b4d8 to
da080dc
Compare
Owner
Author
Q3 from the cross-verify pass closed, plus a correction to my own commit bodyForce-pushed again.
Also newly recorded in that commit body, since the cross-verify pass turned it up while tracing One finding upgraded from inferred to verifiedThe
Two things fell out of the same run: the detector fires on a real suppressed session right now, so UnchangedStill no human review on this PR. Everything above — the original code, the QA gate, the |
This was referenced Jul 30, 2026
Owner
Author
Review debt from this PR is now filed — nothing left blocking except a human readEvery follow-up raised in the QA gate and the cross-verify pass has an issue. None of them block
Also commented on #128 asking that it stay open on merge: #128 (comment) — StateFour commits, 18 files, no version bump, Still zero human review. The original code, the QA gate, the adjudication of that gate, the two |
9 tasks
pitimon
added a commit
that referenced
this pull request
Jul 30, 2026
* fix(doctor): count only non-advisory warns toward `degraded` (#130) `degraded` was `summary.warn > 0 || summary.fail > 0`, which made it useless on the machine it was written for. That box carries a standing `queue.row_invariant` warning about two malformed rows out of 6482, so `degraded` read true on a perfectly healthy day. An alert wired to it could never clear, and an alert that never clears fails the same way as one that never fires — which is the exact failure #128 was about. A check may now set `advisory: true` to say its warn is a standing condition the operator cannot act on at the moment they read the report. `queue.row_invariant` is the only one that does, and it earns it on reasoning already written above `queueCheck`: the rows are on disk and already being rendered, which is why that check warns rather than fails in the first place. `degraded` is now "at least one non-advisory warn or fail". `degraded_checks` lists the ids that put it there, sorted — without it, `degraded: true` is unactionable, since a consumer would have to re-walk `checks` to find out why and a human reading the JSON cannot tell a new problem from the standing one. An advisory check is unchanged everywhere else: still `warn` in `checks`, still counted in `summary.warn`. The report is not quieter; only the alert signal is narrower. Opting out is explicit, so a check that forgets the flag stays alert-worthy — a problem that reads as silence is the thing being prevented. Not a breaking change for anyone: `degraded` has never shipped. `git show v0.39.44:src/lib/doctor.js | grep -c degraded` is 0, and npm `latest` is still 0.39.44 — the field landed on main in #129 and that merge deliberately did not publish. Narrowing it now costs no consumer anything. Verified on the machine that reported #128, running the built CLI rather than a fixture: ok: True | degraded: True degraded_checks: ['ingest.transcript_suppressed'] summary: {'ok': 9, 'warn': 2, 'fail': 0, 'critical': 0} WARN queue.row_invariant | advisory: True WARN ingest.transcript_suppressed | advisory: False Two warns still reported, one reaches the alert signal, and `degraded_checks` names which. Before this change both would have been indistinguishable behind a single boolean. Both mutation points were checked by breaking them, not by watching tests pass: neutralising the `advisory` filter in `listDegradedChecks` fails 3 tests; restoring the old `summary.warn > 0` predicate fails 1. `ci:local` exit 0 with 1002 node tests and 41 dashboard test files. * fix(doctor): close three fail-open holes an independent review found A Fable review of the first cut returned correct-but-incomplete. All three of its substantive findings held up against the source, and the worst one was mine entirely. 1. `advisory` was stamped per check id, not per warn. `queueCheck` hard-coded `advisory: true` on everything it returned, and `checkQueueRows` uses it for two different warns: the row-invariant one (standing — the rows are already written and rendered) and `queue unreadable` (a non-ENOENT read error: permissions, disk). The second is new, actionable, and plausibly means ingestion has stopped — the #128 class this field exists to surface — and it was going out as `warn` in `checks` with `degraded: false` on the wire. I had written "a problem that reads as silence is the thing being prevented" in the same file and then opened that exact path. `advisory` is now an explicit per-call-site argument defaulting to false. 2. `listDegradedChecks` dropped checks with a missing or malformed `id`, and `degraded` derives from the length of that list, so an id typo removed a genuine warn from the alert signal altogether — `[WARN] unknown` to a human (`src/commands/doctor.js` tolerates a missing id) and nothing at all to automation. Such a check is now counted under `UNNAMED_CHECK_ID`. 3. The filter suppressed advisory `fail`s as well as advisory warns. The rationale for the flag is about standing warnings; nothing argues for muting a fail on the same id. Now scoped to warns. Each fix has a test that fails without it, checked one at a time by reverting just that fix: queueCheck hard-codes advisory again -> "an unreadable queue is actionable" fails id map drops non-strings again -> "a warn with a missing id still degrades" fails advisory suppresses fail too -> "advisory never suppresses a fail" fails .sort() removed -> "degraded_checks is sorted" fails all restored -> 20/20 That last row is also a review finding: the docs claimed sorted output and no test pinned it, because the only multi-element expectation was already in sorted input order. The unreadable-queue fixture uses a directory where a file is expected (EISDIR) rather than chmod, because CI runs as root and an unreadable-by-permission file is still readable there. The review's remaining finding is real and is NOT fixed here: `browser.opener` warns permanently on a headless host, so `degraded` still pins there. Reproduced rather than reasoned about: $ node bin/tracker.js doctor --json | jq -c '{degraded, degraded_checks}' {"degraded":true,"degraded_checks":["ingest.transcript_suppressed"]} $ CI=true node bin/tracker.js doctor --json | jq -c '{degraded, degraded_checks}' {"degraded":true,"degraded_checks":["browser.opener","ingest.transcript_suppressed"]} Classifying that check means a severity call on code I did not write, with no coverage for the headless path, so openwiki now states the limit instead of implying the alert-wiring advice is general. Tracked in #137. openwiki also had two sentences that described the intent rather than the mechanism: it said the flag is per check when it is per warn, and omitted the id rule. Both corrected. ci:local exit 0 — 1006 node tests, 41 dashboard test files, OpenWiki fact check 0 findings. --------- Co-authored-by: itarun.p <itarun.p@somapait.com>
22 tasks
pitimon
pushed a commit
that referenced
this pull request
Jul 30, 2026
Two gaps a review found in the previous commit. The `-x` scoping was only ever demonstrated on macOS, where `ps` is BSD and `-x` plainly means "own user, tty restriction lifted". Linux `ps` is procps and parses dash-prefixed options as UNIX-style, where `-x` is not an option — and `isProcessListSupported` returns true for every platform except win32, so Linux runs this argv. Had procps rejected it, `listProcessLines` would have returned `process_list_failed` on every Linux host: a permanent non-advisory warn, pinning `degraded` for a whole platform, which is the failure #136 exists to remove. Checked on a real Debian 12 / procps-ng 4.0.2 host rather than reasoned from the manual: `ps -x -o pid=,command=` exits 0 and reports one user across 22 lines, while `-ax` on the same box reports seven users across 39. procps accepts it as the BSD `x`, so the argv is correct on both supported platforms and #129's shipped code is correct too. Recorded in the comment so nobody has to re-derive it from a manpage. The array is now frozen. Two modules share it, and the claim written in the previous commit — that the two scans "cannot drift apart" — was only true of editorial drift. Importing one constant does not stop `PS_ARGS.push("-a")`; freezing does, and the test asserts the mutation throws rather than asserting the flag alone. Refs #132
pitimon
added a commit
that referenced
this pull request
Aug 2, 2026
…user (#138) * fix(usage-limits): scope the Antigravity process scan to the current user `detectAntigravityProcess` ran `/bin/ps -ax`, which walks every account on the box, and attached to whichever Antigravity language server matched first. Tracing what that could expose settles the question #132 left open: the CSRF token and the pid never reach an HTTP response. `processInfo` is read field by field and is never spread into a returned object — the pid goes to `listAntigravityPorts`, the token becomes a request header, and all four return shapes of `fetchAntigravityLimits` carry neither. What the token *fetches* does reach the response. `normalizeAntigravityResponse` returns `account_email` and `account_plan`, `finalize` spreads them into the result, and `getUsageLimits` serves that at `/functions/tokentracker-usage-limits`. On a shared host this displayed another person's email, plan and quota as the local user's own, and `writeAntigravityLimitsCache` persisted the address to disk, where the not-configured branch kept serving it after their process exited. So: not credential exposure, but cross-account PII — on multi-user hosts only, which is why no observed output could have caught it. The scan now uses the PS_BINARY / PS_ARGS already exported by process-list.js rather than its own inline argv. Sharing the constant is the point: #129 fixed the other scan, and two scans that must both stay own-user should not be able to drift apart. The regression test captures the literal argv from an injected commandRunner, because a real `ps` run on a single-user machine returns identical lines either way. Verified as a guard rather than a restatement: restoring `-ax` fails it. Anyone who has run this on a shared host may have another user's account_email cached in ~/.tokentracker/tracker/usage-limits-cache.json. It is a cache; deleting the file is the whole remedy. Closes #132 * fix(process-list): verify `-x` on Linux and freeze the shared argv Two gaps a review found in the previous commit. The `-x` scoping was only ever demonstrated on macOS, where `ps` is BSD and `-x` plainly means "own user, tty restriction lifted". Linux `ps` is procps and parses dash-prefixed options as UNIX-style, where `-x` is not an option — and `isProcessListSupported` returns true for every platform except win32, so Linux runs this argv. Had procps rejected it, `listProcessLines` would have returned `process_list_failed` on every Linux host: a permanent non-advisory warn, pinning `degraded` for a whole platform, which is the failure #136 exists to remove. Checked on a real Debian 12 / procps-ng 4.0.2 host rather than reasoned from the manual: `ps -x -o pid=,command=` exits 0 and reports one user across 22 lines, while `-ax` on the same box reports seven users across 39. procps accepts it as the BSD `x`, so the argv is correct on both supported platforms and #129's shipped code is correct too. Recorded in the comment so nobody has to re-derive it from a manpage. The array is now frozen. Two modules share it, and the claim written in the previous commit — that the two scans "cannot drift apart" — was only true of editorial drift. Importing one constant does not stop `PS_ARGS.push("-a")`; freezing does, and the test asserts the mutation throws rather than asserting the flag alone. Refs #132 --------- Co-authored-by: itarun.p <itarun.p@somapait.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Goal (one sentence)
Make TokenTracker able to say "some Claude usage is not being recorded" instead of silently reporting a source that has gone unobservable — the failure behind #128.
Why
Every Claude number in this product is parsed out of the session transcripts under
~/.claude/projects. A CLI started with--no-session-persistencewrites none, so those calls burn real tokens and contribute zero. The only symptom is a source reporting less than it should, which is indistinguishable from a quiet day. On the machine that found this, a gateway-routed stream stopped being counted on 2026-07-28 and nothing said so for two days —tokentracker doctorreadok 9 | warn 1 | fail 0 | critical 0the entire time.This follows the same principle as #105: a chip that silently disappears when its fetcher breaks is worse than no chip, because the user has been trained to read absence as "nothing happened".
What was tried first and rejected
The original proposal in #128 was source-staleness detection. It does not work, and the replay is in the issue: every Claude model aggregates under
source=claude, so the source read as active 0.4h ago while the affected model had been silent 37h. It would have missed the exact incident it was written for. At(source, model)granularity it does fire, but against the one machine it has been tested on it produced 3 false positives for 2 true ones — a user retiring a model is indistinguishable from a stream breaking. Parked rather than shipped.What ships here is the live signal instead: read the process list, answer "is something running right now that I cannot see". No baseline, no threshold, no history, and it would have fired the same evening.
Scope
src/)dashboard/)TokenTrackerBar/)TokenTrackerWin/)Checklist
npm run ci:localpasses (exit 0: dashboard build, 41 dashboard test files / 308 tests, 992 CLI tests, all six validators, OpenWiki fact check 0 findings)dashboard/src/content/copy.csv0.40.0across all four)npm-publish.ymlfires on push tomain, so merging this releases it. The bump is a separate commit (6e38d1c) precisely so it can be dropped if this should ride a later release instead.What changed
src/lib/process-list.jsparseProcessLinemoved here so two callers share one copy;listProcessLines()returns{supported, ok, lines, reason}so "no/bin/pson this platform" and "psrefused" stay distinguishablesrc/lib/transcript-suppression.jscommandRunnersrc/lib/doctor.jsingest.transcript_suppressedcheck + thedegradedfieldsrc/commands/doctor.jssrc/lib/local-api.jsGET /functions/tokentracker-ingest-healthsrc/lib/usage-limits.jsparseProcessLine; behaviour and tests unchanged (45/45)dashboard/…/IngestHealthNotice.jsxCopilotOtelHint,role="note"dashboard/src/pages/DashboardPage.jsxgit diff -w, which shows 31openwiki/degradeddocumented;source-facts.jsonregenerated (14 endpoints)Live output on a machine reproducing the bug
--jsonon the same run:"ok": true, "degraded": true.Codex Context (required when requesting @codex review)
xhighreviewed the design before any code existed and returned FIX. Everything in it is addressed or explicitly parked: source-level detection misses the incident (redesigned — see above); four wrong metrics in [bug]: claude-mem's gateway (GLM) usage silently stops being counted — claude-mem 13.12.4 spawns the CLI with --no-session-persistence #128 (corrected publicly in this comment);parseProcessLinenot exported (moved to a shared module); pids over an unauthenticated endpoint (removed); no Windows/bin/ps(guarded); process spawn per dashboard request (TTL cache); doctor WARN being invisible to automation (degraded). Parked with reasons: monotonic high-water mark, timezone-awareactiveDays, and gateway reconciliation — all belong to the silence detector, which is not in this PR.okand the exit code are unchanged for every existing caller.--modelabsent;--model=valueform; a non-Claude process whose argv merely contains the flag (a shell, an editor, this check itself);psfailing; Windows; TTL boundary; a command line carrying a user path in--model.npm run ci:local→ exit 0. New:test/transcript-suppression.test.js(13),test/local-api-ingest-health.test.js(3), 4 cases intest/doctor.test.js, 6 inIngestHealthNotice.test.jsx.userStatuseffect), so a session started later is not shown until reload —doctoris current on demand. Detection is macOS/Linux only. Anode-launched CLI (node cli.js --no-session-persistence) is not matched; argv[0] must be theclaudebinary, which is the trade for not matching every shell that mentions the flag.Risk Layer Trigger (if any)
Risk Layer Addendum
Rules / Invariants
GETs, like every other read endpoint here. Therefore its payload — not the module's internals — is the security boundary.meta.[A-Za-z0-9._:-]{1,64}, so a path or arbitrary argv text cannot ride out through the field.Boundary Matrix (must list at least 3)
psstdout → detector return valuenull | "unsupported_platform" | "process_list_failed"findSuppressedModelsreturns models only, never the parsed pid;test/transcript-suppression.test.js"never returns a pid or a raw command line" serialises the result and asserts the pid, the path, and the raw flag are all absenttest/local-api-ingest-health.test.js"leaks no pid, argv, or path" asserts on the raw response body--modelvalue → user-visible text (doctor detail, dashboard notice)--model[A-Za-z0-9._:-]{1,64}MODEL_FLAGcharset clamp; test feeds--model /Users/example/secret-project/notes.mdand asserts the process is still counted while no path is returnedprocess.platform,psexit status[OK]only when a scan actually ran and found nothingbuildTranscriptSuppressionCheckreturnsnullwhensupported === false(check omitted, not[OK]) andwarnwhenchecked === false; three doctor tests pin all three statesEvidence (tests or repro)
IngestHealthNoticeremedy test asserts literal text rather thancopy(...). That is deliberate: the row it guards originally shipped with an unquoted comma incopy.csv, which truncated the sentence at "this," — andvalidate:copystill reportedCopy registry ok, because all six required columns were non-empty. Verified by reverting the row and re-running: validator still green, test fails.false) and the current one (true) before the test was added, so it is a guard rather than a restatement.Public Exposure Checklist
GET /functions/*endpoints; no new auth surfacesupported,checked,count,models,reason,checked_at, and nothing else; the handler names each field rather than spreading the detector resultRegression Test Gate
Most likely regression surface
src/lib/usage-limits.js—parseProcessLinemoved out from underdetectAntigravityProcess. A bad move breaks Antigravity quota detection, which has no other guard on this path.Verification method (choose at least one)
node --test test/usage-limits.test.js→ 45/45 pass, including thedetectAntigravityProcesscase that exercises the moved parser through an injectedcommandRunnernpm run ci:local→ exit 0node bin/tracker.js doctoron a machine actually reproducing [bug]: claude-mem's gateway (GLM) usage silently stops being counted — claude-mem 13.12.4 spawns the CLI with --no-session-persistence #128 (output above)Uncovered scope
platform: "win32", but nothing has run on real Windows.Refs #128. Does not close it — proposal #1 (source silence) stays open there with the replay data that shows why it needs rework.