Skip to content

docs: set an explicit contribution bar for agent-authored PRs - #2602

Merged
douenergy merged 4 commits into
mainfrom
docs/claude-md-contribution-bar
Jul 28, 2026
Merged

docs: set an explicit contribution bar for agent-authored PRs#2602
douenergy merged 4 commits into
mainfrom
docs/claude-md-contribution-bar

Conversation

@goldmedal

@goldmedal goldmedal commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a ## Contribution Bar section to the root CLAUDE.md, one design-point entry to the wren package CLAUDE.md, and a pull request template.

What problem does this solve?

Review load, not a code defect. A growing share of incoming PRs share a few recurring shapes:

  • Unevidenced claims about external systems. Code comments and PR descriptions asserting that an engine or driver rejects some input, where testing shows it does not.
  • Guards for unreachable states. Defensive isinstance checks over manifest structures that wren-core's serde layer has already rejected a few lines earlier.
  • fix: on changes with no observable effect, which makes the changelog claim repairs that did not happen.
  • Tests that don't exercise the change — asserting on source text read as a string, assertions that cannot fail, or tests placed in a file the default CI jobs skip.
  • Duplicate PRs, including several pairs against the same file with the same change.
  • Long-lived branches merged on stale CI. fix(redshift): strip trailing semicolon on unlimited query path #2482 merged cleanly and still broke main for about a day, because its last CI run predated a refactor that renamed the helper it called. See CI: stale required checks let a semantic merge conflict (F821) reach main #2598.

None of this was written down anywhere, so there was no shared standard to point at in review.

What's in it

  • .claude/CLAUDE.md — a Contribution Bar section: prove the problem before fixing it, label honestly, make tests exercise the changed code and confirm they run in CI, one change per PR, don't silently reverse documented decisions, rebase before review. Each rule is written to be self-checkable before opening a PR and citable by name in review.
  • core/wren/.claude/CLAUDE.md — a "Validation boundaries" design point recording where MDL manifest schema is actually enforced (wren-core serde, via ManifestExtractor / SessionContext), and the contrast with genuine input boundaries — raw user JSON, hand-edited project YAML, external API responses — where validating and reporting is the right behaviour.
  • AGENTS.md (new, repo root) — a pointer to .claude/CLAUDE.md, matching the existing per-module AGENTS.md files, so agents that look for AGENTS.md rather than CLAUDE.md reach the contribution bar. The root was the one level that lacked this file.
  • .github/pull_request_template.md — the same questions at authoring time, for contributors who don't read CLAUDE.md. Explicitly skippable for typo and docs changes.

Notes

Deliberately describes patterns rather than naming contributors or citing the PRs that prompted it; this file stays in the repo long after any individual PR is closed. The two incident references above (#2482, #2598) are process history rather than attribution.

Docs only — no code or CI changes.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Added a “Contribution Bar” section with clearer guidance on evidence-based bug reports, avoiding speculative/untested claims, ensuring tests cover the changed code, keeping cohesive scope, and rebasing before review.
    • Added a standardized pull request template to streamline concise descriptions, required failure-mode details for fixes, testing notes, and duplicate checks.
    • Documented manifest validation boundaries, clarifying where schema validation happens and how errors should be handled.
    • Added contributor index files pointing to shared and module-specific contribution instructions.

A growing share of incoming PRs assert engine behaviour that turns out to be
false when tested, guard states that cannot occur, or carry `fix:` for changes
with no observable effect. Reviewing them costs more than the changes are
worth, and nothing in the repo told contributors where the bar is.

State it: reproduce before claiming, label honestly, make tests exercise the
code they cover and verify they actually run in CI, one change per PR, and
rebase before review. The wren package doc gains the specific fact behind the
most common case — manifest schema is validated once in wren-core, so
per-callsite guards downstream are dead code, while real input boundaries
should validate and report rather than crash.

The PR template asks the same questions at the point of authoring, since not
every contributor reads CLAUDE.md.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added documentation Improvements or additions to documentation python Pull requests that update Python code core ci labels Jul 28, 2026
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 8c4afca2-2b08-4988-9265-04aef9072c58

📥 Commits

Reviewing files that changed from the base of the PR and between 09feffb and ea3a4b5.

📒 Files selected for processing (1)
  • core/wren/.claude/CLAUDE.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • core/wren/.claude/CLAUDE.md

Walkthrough

Repository guidance adds contribution standards, instruction pointers, a pull request template, and documentation describing the MDL manifest validation boundary.

Changes

Repository guidance

Layer / File(s) Summary
Contribution and pull request workflow
AGENTS.md, .claude/CLAUDE.md, .github/pull_request_template.md
Contributor guidance and the PR template define evidence-based reports, testing expectations, labeling, rebasing, summaries, failure context, duplicate checks, and instruction locations.
MDL validation boundary
core/wren/.claude/CLAUDE.md
Documentation states that wren-core validates MDL manifests during deserialization and that external or manually edited inputs should be validated at the boundary.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Suggested reviewers: douenergy

Poem

I’m a rabbit with a checklist bright,
Guiding commits through review’s moonlight.
Tests hop in, claims stay true,
Validation waits at the edge in view.
Thump, thump—clean guidance grows!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: adding contribution guidance for agent-authored pull requests.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/claude-md-contribution-bar

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/pull_request_template.md (1)

6-38: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Add author checks for every mandatory Contribution Bar gate.

The template prompts for reproduction, tests, and duplicate checks, but not honest PR labeling, documented-decision changes, or rebasing. Add explicit checkboxes (or equivalent fields) so authors attest to all requirements from .claude/CLAUDE.md Lines 136-169.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/pull_request_template.md around lines 6 - 38, Add explicit
author-attestation fields to the pull request template covering honest PR
labeling, documented-decision changes, and rebasing, alongside the existing
failure, testing, and duplicate checks. Align the new requirements with the
mandatory Contribution Bar gates defined in CLAUDE.md, while preserving the
current prompts and template structure.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/wren/.claude/CLAUDE.md`:
- Line 49: Revise the “Validation boundaries — validate once, at the edge”
guidance to acknowledge that context.py decodes the manifest and accesses
manifest["views"] before WrenEngine construction, so malformed raw JSON/base64
input can reach Python unvalidated. State that wren-core remains the
schema-validation authority, but raw input must be structurally validated before
Python dereferences it; retain the prohibition on redundant guards only for
post-deserialization paths.

---

Outside diff comments:
In @.github/pull_request_template.md:
- Around line 6-38: Add explicit author-attestation fields to the pull request
template covering honest PR labeling, documented-decision changes, and rebasing,
alongside the existing failure, testing, and duplicate checks. Align the new
requirements with the mandatory Contribution Bar gates defined in CLAUDE.md,
while preserving the current prompts and template structure.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 70aad793-baa5-4db3-91b6-34973eb998f5

📥 Commits

Reviewing files that changed from the base of the PR and between a93184d and d8ef043.

📒 Files selected for processing (3)
  • .claude/CLAUDE.md
  • .github/pull_request_template.md
  • core/wren/.claude/CLAUDE.md

Comment thread core/wren/.claude/CLAUDE.md Outdated
The per-module crates already carry this pointer file; the repository root
did not, so agents that read AGENTS.md rather than CLAUDE.md found nothing
at the top level and never reached the contribution bar.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@AGENTS.md`:
- Around line 3-7: Revise the repository-wide guidance statement in AGENTS.md to
clarify that the root .claude/CLAUDE.md contains repository-wide instructions,
while applicable module-specific guidance must also be read from
core/<module>/.claude/CLAUDE.md. Preserve the existing direction to consult the
Contribution Bar and the per-module AGENTS.md references.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: aee595e4-c701-4321-b2fb-6ca7aa2a81dd

📥 Commits

Reviewing files that changed from the base of the PR and between d8ef043 and 464fcdf.

📒 Files selected for processing (1)
  • AGENTS.md

Comment thread AGENTS.md Outdated
context.validate_manifest decodes the manifest and iterates manifest["views"]
before WrenEngine is constructed, and construction does not deserialize — so
Python can and does observe an unvalidated dict. The guarantee only holds
downstream of a successful wren-core deserialization; say so, and name that
function as the counter-example.

Also narrow the root AGENTS.md opening line, which claimed all instructions
live in .claude/CLAUDE.md while the same file points at per-module ones.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@goldmedal

Copy link
Copy Markdown
Collaborator Author

Pushed 09feffbc addressing two of the three.

Validation-boundary timing claim — correct, thanks. context.validate_manifest decodes the manifest and iterates manifest["views"] before WrenEngine is constructed, and construction doesn't deserialize, so Python does observe an unvalidated dict there. The bullet now scopes the guarantee to downstream of a successful deserialization and names that function as the counter-example. I kept the concrete list of input boundaries (raw JSON before wren-core, hand-edited project YAML, external API responses) rather than taking the suggested wording verbatim — that list is the part a contributor can actually check their change against.

Root AGENTS.md opening line — agreed, "all repository instructions" contradicted the per-module pointer three lines below. Narrowed to "Repository-wide instructions".

Attestation checkboxes in the PR template — declining, deliberately. Three reasons:

  1. Checkboxes don't filter the population this template exists for. An agent generating PRs at volume will tick every box correctly and still submit the same unevidenced change. What has actual signal is the open-ended prompts that require pasting a real reproduction and a real error — those can't be satisfied by ticking.
  2. "Rebased before review" isn't something an author can attest at authoring time; it's a property at review time, and GitHub already surfaces behind-by status.
  3. The lightweight shape is intentional. The template opens by telling typo and docs contributors to fill in the summary and delete the rest, precisely so the bar falls on behaviour-changing PRs rather than on everyone. Expanding four questions into six attestations pushes it toward the checklist that gets ticked without being read.

The Contribution Bar in CLAUDE.md remains the full statement; the template is the subset that benefits from being asked at authoring time.

@goldmedal
goldmedal requested a review from douenergy July 28, 2026 04:13
validate_manifest was the wrong illustration. It does dereference a decoded
manifest before wren-core sees it, but its only caller rebuilds the manifest
via build_json(), so malformed input cannot actually reach it — citing it as
a "genuine input boundary" would invite exactly the unreachable-state guard
the paragraph above forbids.

convert_mdl_to_project is the real case: `wren context init --from-mdl` reads
a user-supplied JSON file and passes it straight in, with nothing in between.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci core documentation Improvements or additions to documentation python Pull requests that update Python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants