Skip to content

AGENTS.md

Latest commit

 

History

History
420 lines (299 loc) · 15 KB

AGENTS.md

File metadata and controls

420 lines (299 loc) · 15 KB
Raw

AGENTS.md — csctf Project

RULE 1 – ABSOLUTE (DO NOT EVER VIOLATE THIS)

You may NOT delete any file or directory unless I explicitly give the exact command in this session.

  • This includes files you just created (tests, tmp files, scripts, etc.).
  • You do not get to decide that something is "safe" to remove.
  • If you think something should be removed, stop and ask. You must receive clear written approval before any deletion command is even proposed.

Treat "never delete files without permission" as a hard invariant.

IRREVERSIBLE GIT & FILESYSTEM ACTIONS

Absolutely forbidden unless I give the exact command and explicit approval in the same message:

  • git reset --hard
  • git clean -fd
  • rm -rf
  • Any command that can delete or overwrite code/data

Rules:

  1. If you are not 100% sure what a command will delete, do not propose or run it. Ask first.
  2. Prefer safe tools: git status, git diff, git stash, copying to backups, etc.
  3. After approval, restate the command verbatim, list what it will affect, and wait for confirmation.
  4. When a destructive command is run, record in your response:
    • The exact user text authorizing it
    • The command run
    • When you ran it

If that audit trail is missing, then you must act as if the operation never happened.

Node / JS Toolchain

  • Use bun for everything JS/TS.
  • ❌ Never use npm, yarn, or pnpm.
  • Lockfiles: only bun.lock. Do not introduce any other lockfile.
  • Target latest Node.js. No need to support old Node versions.

Project Architecture

This is a single-file CLI tool (src/index.ts, ~3000 lines). The architecture is intentionally monolithic for simplicity and to produce a single compiled executable.

Key patterns:

  • Provider detection: URL patterns → Provider type (chatgpt, gemini, grok, claude)
  • Selector discovery: Provider-specific CSS selector arrays with fallback chains
  • Browser automation: Playwright with stealth measures and CDP fallback for Cloudflare-protected sites
  • Output formatting: Markdown with optional HTML (syntax-highlighted via highlight.js)

Build targets:

bun run build              # Local binary → dist/csctf
bun run build:all          # Cross-platform builds
bun run check              # Lint + typecheck

When adding features:

  • Add to src/index.ts directly; do not split into modules unless absolutely necessary.
  • Follow existing patterns for provider support, selector fallbacks, and error handling.
  • Test with bun run src/index.ts <url> before compiling.

Code Editing Discipline

  • Do not run scripts that bulk-modify code (codemods, invented one-off scripts, giant sed/regex refactors).
  • Large mechanical changes: break into smaller, explicit edits and review diffs.
  • Subtle/complex changes: edit by hand, file-by-file, with careful reasoning.

Backwards Compatibility & File Sprawl

We optimize for a clean architecture now, not backwards compatibility.

  • No "compat shims" or "v2" file clones.
  • When changing behavior, migrate callers and remove old code inside the same file.
  • New files are only for genuinely new domains that don't fit existing modules.
  • The bar for adding files is very high.

Console Output

This CLI uses chalk for colored console output. Patterns to follow:

console.error(chalk.blue('[1/8] Step description'))     // Progress steps
console.error(chalk.gray('    Details...'))             // Indented details
console.error(chalk.yellow('\n⚠️  Warning message'))    // Warnings
console.error(chalk.red('✖ Error message'))             // Errors
console.error(chalk.green('✔ Success message'))         // Success

Rules:

  • All progress/status goes to stderr (so stdout remains clean for piping)
  • Main output (markdown/HTML) goes to stdout
  • Quiet mode (--quiet) suppresses progress messages but not errors

Third-Party Libraries

When unsure of an API, look up current docs (late-2025) rather than guessing.

Key dependencies:

  • playwright-chromium: Browser automation with stealth measures
  • chalk: Terminal coloring
  • markdown-it: Markdown rendering
  • highlight.js: Syntax highlighting for code blocks
  • turndown: HTML-to-Markdown conversion

Provider-Specific Patterns

When adding a new provider:

  1. Add to Provider type union
  2. Add URL patterns to PROVIDER_PATTERNS
  3. Add CSS selector fallback chains to SELECTOR_FALLBACKS
  4. Update sharePattern regex for URL validation
  5. Test in both headless and headful modes
  6. Handle Cloudflare/bot-detection if present (may need CDP mode)

CDP Mode (for Cloudflare-protected sites):

  • Connects to user's real Chrome via --remote-debugging-port=9222
  • Saves/restores user's open tabs (macOS only via AppleScript)
  • Prompts user to solve Cloudflare challenges manually

MCP Agent Mail — Multi-Agent Coordination

Agent Mail is available as an MCP server for coordinating work across agents.

What Agent Mail gives:

  • Identities, inbox/outbox, searchable threads.
  • Advisory file reservations (leases) to avoid agents clobbering each other.
  • Persistent artifacts in git (human-auditable).

Core patterns:

  1. Same repo

    • Register identity:
      • ensure_project then register_agent with the repo's absolute path as project_key.
    • Reserve files before editing:
      • file_reservation_paths(project_key, agent_name, ["src/**"], ttl_seconds=3600, exclusive=true).
    • Communicate:
      • send_message(..., thread_id="FEAT-123").
      • fetch_inbox, then acknowledge_message.
    • Fast reads:
      • resource://inbox/{Agent}?project=<abs-path>&limit=20.
      • resource://thread/{id}?project=<abs-path>&include_bodies=true.

  2. Macros vs granular:

    • Prefer macros when speed is more important than fine-grained control:
      • macro_start_session, macro_prepare_thread, macro_file_reservation_cycle, macro_contact_handshake.
    • Use granular tools when you need explicit behavior.

Common pitfalls:

  • "from_agent not registered" → call register_agent with correct project_key.
  • FILE_RESERVATION_CONFLICT → adjust patterns, wait for expiry, or use non-exclusive reservation.

Testing

bun test                   # Unit tests
bun run test:e2e           # E2E tests (requires CSCTF_E2E=1)

For manual testing:

# Test a provider
bun run src/index.ts https://chatgpt.com/share/<id>
bun run src/index.ts https://gemini.google.com/share/<id>
bun run src/index.ts https://x.com/i/grok/share/<id>
bun run src/index.ts https://claude.ai/share/<id>  # Requires CDP mode

# Test output formats
bun run src/index.ts <url> --html        # Markdown + HTML
bun run src/index.ts <url> --json        # JSON output
bun run src/index.ts <url> -o output.md  # Write to file

Using bv as an AI Sidecar

bv is a graph-aware triage engine for Beads projects (.beads/beads.jsonl). Instead of parsing JSONL or hallucinating graph traversal, use robot flags for deterministic, dependency-aware outputs with precomputed metrics (PageRank, betweenness, critical path, cycles, HITS, eigenvector, k-core).

Scope boundary: bv handles what to work on (triage, priority, planning). For agent-to-agent coordination (messaging, work claiming, file reservations), use MCP Agent Mail.

CRITICAL: Use ONLY --robot-* flags. Bare bv launches an interactive TUI that blocks your session.

The Workflow: Start With Triage

bv --robot-triage is your single entry point. It returns everything you need in one call:

  • quick_ref: at-a-glance counts + top 3 picks
  • recommendations: ranked actionable items with scores, reasons, unblock info
  • quick_wins: low-effort high-impact items
  • blockers_to_clear: items that unblock the most downstream work
  • project_health: status/type/priority distributions, graph metrics
  • commands: copy-paste shell commands for next steps
bv --robot-triage        # THE MEGA-COMMAND: start here
bv --robot-next          # Minimal: just the single top pick + claim command

Other bv Commands

Planning:

Command Returns
--robot-plan Parallel execution tracks with unblocks lists
--robot-priority Priority misalignment detection with confidence

Graph Analysis:

Command Returns
--robot-insights Full metrics: PageRank, betweenness, HITS, eigenvector, critical path, cycles, k-core, articulation points, slack
--robot-label-health Per-label health: health_level (healthy|warning|critical), velocity_score, staleness, blocked_count
--robot-label-flow Cross-label dependency: flow_matrix, dependencies, bottleneck_labels

History & Change Tracking:

Command Returns
--robot-history Bead-to-commit correlations
--robot-diff --diff-since <ref> Changes since ref: new/closed/modified issues, cycles introduced/resolved

jq Quick Reference

bv --robot-triage | jq '.quick_ref'                        # At-a-glance summary
bv --robot-triage | jq '.recommendations[0]'               # Top recommendation
bv --robot-plan | jq '.plan.summary.highest_impact'        # Best unblock target
bv --robot-insights | jq '.status'                         # Check metric readiness
bv --robot-insights | jq '.Cycles'                         # Circular deps (must fix!)

Use bv instead of parsing beads.jsonl—it computes PageRank, critical paths, cycles, and parallel tracks deterministically.

ast-grep vs ripgrep (quick guidance)

Use ast-grep when structure matters. It parses code and matches AST nodes, so results ignore comments/strings, understand syntax, and can safely rewrite code.

  • Refactors/codemods: rename APIs, change import forms, rewrite call sites or variable kinds.
  • Policy checks: enforce patterns across a repo (scan with rules + test).
  • Editor/automation: LSP mode; --json output for tooling.

Use ripgrep when text is enough. It's the fastest way to grep literals/regex across files.

  • Recon: find strings, TODOs, log lines, config values, or non-code assets.
  • Pre-filter: narrow candidate files before a precise pass.

Rule of thumb

  • Need correctness over speed, or you'll apply changes → start with ast-grep.
  • Need raw speed or you're just hunting text → start with rg.
  • Often combine: rg to shortlist files, then ast-grep to match/modify with precision.

Snippets

Find structured code (ignores comments/strings):

ast-grep run -l TypeScript -p 'import $X from "$P"'

Codemod (only real var declarations become let):

ast-grep run -l TypeScript -p 'var $A = $B' -r 'let $A = $B' -U

Quick textual hunt:

rg -n 'console\.log\(' -t ts

Combine speed + precision:

rg -l -t ts 'chalk\.' | xargs ast-grep run -l TypeScript -p 'chalk.$METHOD($ARGS)' --json

Mental model

  • Unit of match: ast-grep = node; rg = line.
  • False positives: ast-grep low; rg depends on your regex.
  • Rewrites: ast-grep first-class; rg requires ad-hoc sed/awk and risks collateral edits.

Morph Warp Grep — AI-Powered Code Search

Use mcp__morph-mcp__warp_grep for "how does X work?" discovery across the codebase.

When to use:

  • You don't know where something lives.
  • You want data flow across multiple files (CLI → provider → parser → output).
  • You want all touchpoints of a cross-cutting concern (e.g., selector discovery, browser automation).

Example:

mcp__morph-mcp__warp_grep(
  repoPath: "/data/projects/chat_shared_conversation_to_file",
  query: "How does the provider detection work for different chat services?"
)

Warp Grep:

  • Expands a natural-language query to multiple search patterns.
  • Runs targeted greps, reads code, follows imports, then returns concise snippets with line numbers.
  • Reduces token usage by returning only relevant slices, not entire files.

When not to use Warp Grep:

  • You already know the function/identifier name; use rg.
  • You know the exact file; just open it.
  • You only need a yes/no existence check.

Comparison:

Scenario Tool
"How does provider detection work?" warp_grep
"Where is SELECTOR_FALLBACKS defined?" rg
"Replace var with let" ast-grep

Beads Workflow Integration

Note: br (beads_rust) is non-invasive and never executes git commands. You must run git commands manually after br sync --flush-only.

When starting a beads-tracked task:

  1. Pick ready work (Beads)
    • br ready --json → choose one item (highest priority, no blockers)
  2. Reserve edit surface (Mail)
    • file_reservation_paths(project_key, agent_name, ["src/**"], ttl_seconds=3600, exclusive=true, reason="br-123")
  3. Announce start (Mail)
    • send_message(..., thread_id="br-123", subject="[br-123] Start: <short title>", ack_required=true)
  4. Work and update
    • Reply in-thread with progress and attach artifacts/images; keep the discussion in one thread per issue id
  5. Complete and release
    • br close br-123 --reason "Completed" (Beads is status authority)
    • release_file_reservations(project_key, agent_name, paths=["src/**"])
    • Final Mail reply: [br-123] Completed with summary and links

Mapping cheat-sheet:

  • Mail thread_id ↔ br-###
  • Mail subject: [br-###] ...
  • File reservation reason: br-###
  • Commit messages (optional): include br-### for traceability

Landing the Plane (Session Completion)

When ending a work session, you MUST complete ALL steps below. Work is NOT complete until git push succeeds.

MANDATORY WORKFLOW:

  1. File issues for remaining work - Create issues for anything that needs follow-up
  2. Run quality gates (if code changed) - Tests, linters, builds
  3. Update issue status - Close finished work, update in-progress items
  4. PUSH TO REMOTE - This is MANDATORY: 
    git pull --rebase
br sync --flush-only
git add .beads/
git commit -m "sync beads" --allow-empty
git push
git status  # MUST show "up to date with origin"
  5. Clean up - Clear stashes, prune remote branches
  6. Verify - All changes committed AND pushed
  7. Hand off - Provide context for next session

CRITICAL RULES:

  • Work is NOT complete until git push succeeds
  • NEVER stop before pushing - that leaves work stranded locally
  • NEVER say "ready to push when you are" - YOU must push
  • If push fails, resolve and retry until it succeeds

Note for Codex/GPT-5.2

If you are Codex or GPT-5.2 (or any non-Claude agent): another agent (often Claude Code) may have made changes to the working tree since you last saw it. Before assuming your mental model of the code is correct:

  1. Run git status to see uncommitted changes
  2. Run git log --oneline -5 to see recent commits
  3. Re-read any files you plan to modify

This prevents you from overwriting another agent's work or making edits based on stale context

Contribution Policy

Remove any mention of contributing/contributors from README and don't reinsert it.