test: make evals judge outcomes and self-explain failures

[angeloashmore]

Resolves:

Description

Two evals scored 0/3 in the first CI run for reasons that were not agent failures. "adds constrained fields" required the field add content-relationship command, but the agent used field add link --allow document, which models the same constrained relationship; the eval now asserts the resulting model shape instead of the command. "commits and pushes local model changes" seeded an article type with no fields, so the agent reasonably asked for confirmation instead of publishing an empty type; the fixture now seeds a title field.

toHaveRun failures now also print the agent's final message, which is how the empty-type problem was found. Future CI failures explain themselves in the log without a local rerun.

Checklist

• A comprehensive Linear ticket, providing sufficient context and details to facilitate the review of the PR, is linked to the PR.
• If my changes require tests, I added them.
• If my changes affect backward compatibility, it has been discussed.
• If my changes require an update to the CONTRIBUTING.md guide, I updated it.

Preview

How to QA ¹

# Run the two adjusted evals:
# PRISMIC_ALLOW_EVALS=true node --run evals -- edit-models-precisely -t "adds constrained fields"
# PRISMIC_ALLOW_EVALS=true node --run evals -- sync-models -t "commits and pushes"

🤖 Generated with Claude Code

---

Note

Low Risk
Test-only changes to eval assertions and fixtures; no production runtime behavior.

Overview
Evals now grade agent results by outcome instead of exact CLI spelling, so CI failures reflect real regressions rather than equivalent command choices.

In adds constrained fields, the test no longer requires field add content-relationship or field add link; it only checks that some field add ran and that the product model has the select options and a document link constrained to author (including select: "document").

In commits and pushes local model changes, the article fixture includes a Title text field so the agent is not stuck confirming an empty type before push.

When toHaveRun fails, the assertion message now includes the agent’s final message alongside the command list, making CI logs easier to debug without a local rerun.

^{Reviewed by Cursor Bugbot for commit 7f8c2d4. Bugbot is set up for automated code reviews on this repo. Configure here.}

Footnotes

1. Please use these labels when submitting a review:
   ❓ #ask: Ask a question.
   💡 #idea: Suggest an idea.
   ⚠️ #issue: Strongly suggest a change.
   🎉 #nice: Share a compliment. ↩