feat(metamorphic): analyzer-robustness harness for the .own core

[PhysShell]

Adds scripts/metamorphic.py — a metamorphic testing harness for the Own.NET core checker (analyzer QA; the next backlog brick after SARIF). It generates semantically-equivalent variants of a .own program and asserts the checker's diagnostics are invariant. A divergence is a robustness bug in the analyzer itself (it keyed on a name or textual order) — found with no labels and no oracle (the StaAgent / Statfier line). Standalone value, no LLM; it is also the behavior-preservation half of a future RLVR reward (see research-landscape-2026.md), but that is a bonus.

How

parse(.own) → mutate the AST (dataclasses.replace; nodes are frozen) → check_module → compare the (code, line) multiset. dotnet-free — drives the same parser + core as the CLI. The (code, line) key is the right invariant: the v1 transforms can only move a caret column or keep each node's own line, so a correct checker yields the identical set.

Two provably meaning-preserving transforms:

• alpha-rename — a local bound exactly once (no shadowing) + all its references; pure alpha-equivalence.
• reorder — adjacent simple statements with disjoint touched-variable sets; independent statements commute. Conservative (skips control flow / borrow blocks / shared-variable pairs), so it never emits an unsound variant.

Result

The whole .own corpus (gallery + examples + corpus, 28 programs) is invariant under both — the expected baseline for a symbol-identity/dataflow core, now ratcheted in CI. The harness is not vacuous: a teeth test pins that the (code, line) key distinguishes a leak from a clean run and that the transforms actually fire.

Honest scope

v1 confirms core robustness rather than finding a bug (the core is principled — name/order-agnostic by design). Bug-finding power grows on the follow-ups in docs/notes/metamorphic.md: more sound transforms; an OwnIR-fact target (mutate the JSON facts → check_facts; higher-signal, still dotnet-free); and a C# source target (where the P-014 syntactic-FP bugs actually lived — but needs Roslyn, so CI-only).

Validation

metamorphic --selftest 6/6 (28 files swept), wired into the CI script selftests step beside the miner/oracle; ruff clean, mypy --strict clean (incl. the new script), full suite green, miner 15/15, oracle 24/24.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Rg8kSk1YT14x7A1vo5zgED

---

Generated by Claude Code

Summary by CodeRabbit

• Testing

  • Enhanced CI “lint” with an additional metamorphic analyzer self-test.
  • The metamorphic checker now verifies diagnostic consistency across semantically-equivalent program variants (using diagnostic-code multiset comparison so line shifts from reordering don’t break invariance).

• Documentation

  • Added a guide to metamorphic testing robustness for the checker, including the kinds of program variants generated and how results are validated.

• Chores

  • Added a standalone, dotnet-free metamorphic testing harness/CLI with a --selftest mode and file/directory sweep capability.

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

Adds scripts/metamorphic.py, a metamorphic testing harness that parses .own sources, applies two AST transforms (alpha-rename and adjacent independent statement reorder), and asserts diagnostic code invariance across variants. Also adds docs/notes/metamorphic.md and extends the CI lint job to run --selftest.

Changes

Metamorphic Testing Harness

Layer / File(s)	Summary
Script setup, diag_key, and AST transform utilities scripts/metamorphic.py	Module docstring and shebang, sys.path injection, imports, diag_key (sorted diagnostic codes from check_module), and all AST name-analysis and substitution helpers (refs_in_expr, rename_expr, rename_stmt, count_bindings, all_idents, _rename_fn, _with_fn).
Transform generators and violations checking loop scripts/metamorphic.py	alpha_rename_variants and reorder_variants generators produce labeled Module variants; violations parses a source string, computes the base key, iterates all variants, catches checker exceptions, and returns a list of invariant-break strings.
Sweep, selftest, main CLI, CI wiring, and docs scripts/metamorphic.py, .github/workflows/ci.yml, docs/notes/metamorphic.md	sweep collects .own files recursively and returns a nonzero exit code on any violation; _selftest runs corpus invariance, a diag_key discriminability ("teeth") test, and a fixture transform-fires check; main dispatches --selftest vs sweep; CI lint step extended to invoke --selftest; design note added.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐇 Hop, hop through the AST,
rename a binding — diagnostics agree!
Swap two statements, disjoint sets,
the checker holds, no false regrets.
With teeth that bite and corpus wide,
invariance runs on every CI ride! 🌟

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat(metamorphic): analyzer-robustness harness for the .own core' accurately describes the main change—adding a metamorphic testing harness for analyzer robustness—and is concise, specific, and directly related to the core functionality introduced.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch claude/metamorphic-harness

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f23a3cf851

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (1)

scripts/metamorphic.py (1)

282-285: ⚡ Quick win

sweep() docstring and return contract diverge.

The docstring says “Returns the number of files with a violation,” but Line 304 returns exit status (0/1). Aligning this avoids misuse by callers.

Suggested fix

 def sweep(paths: list[str]) -> int:
-    """Run the harness over every .own file under the given files/dirs. Returns the
-    number of files with a violation (0 == all robust)."""
+    """Run the harness over every .own file under the given files/dirs.
+    Returns process exit status: 0 if all files are invariant, else 1."""

Also applies to: 304-304

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/metamorphic.py` around lines 282 - 285, The docstring for the sweep()
function claims it returns the number of files with a violation, but the actual
return statement at line 304 returns an exit status (0 or 1) instead. Update the
docstring to accurately describe the actual return behavior, clarifying that it
returns 0 if all files are robust and 1 if any violations are found, rather than
claiming it returns a count of violated files.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/metamorphic.py`:
- Around line 316-317: The roots variable excludes the gallery directory despite
the PR documentation stating that the selftest sweep should cover gallery,
examples, and corpus. Add repo / "gallery" to the roots list alongside repo /
"examples" and repo / "corpus" to ensure all three directories are included when
collecting .own files via the rglob operation.
- Around line 264-267: The try-except block catching ParseError and LexError
exceptions in the parse function call is returning an empty list, which makes
parsing failures indistinguishable from successful parses with no violations.
This allows _selftest to pass even when corpus files become unparseable. Instead
of returning an empty list when these exceptions occur, either re-raise the
exception or return a distinct value that clearly indicates a parsing error
happened, so that parsing failures are properly detected and reported rather
than silently treated as non-violations.

---

Nitpick comments:
In `@scripts/metamorphic.py`:
- Around line 282-285: The docstring for the sweep() function claims it returns
the number of files with a violation, but the actual return statement at line
304 returns an exit status (0 or 1) instead. Update the docstring to accurately
describe the actual return behavior, clarifying that it returns 0 if all files
are robust and 1 if any violations are found, rather than claiming it returns a
count of violated files.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 98f0d909-20c2-4202-bb28-ba6642f30475

📥 Commits

Reviewing files that changed from the base of the PR and between 19b0bdc and f23a3cf.

📒 Files selected for processing (3)

• .github/workflows/ci.yml
• docs/notes/metamorphic.md
• scripts/metamorphic.py