Fix JSX scanning contexts

[tannerlinsley]

Changes

• keep JSX prose out of JavaScript string and semantic tokenization
• support multiline JSX tags and attributes without confusing TSX generics
• preserve nested JSX, fragments, regex literals, comments, templates, and attribute strings
• close tsx: apostrophe in JSX text opens a string and swallows the closing tag #5, tsx: multiline JSX tags lose tag and attribute classification #6, and tsx: JSX text content gets code token classes (keywords, numbers, types) #7

Fixes #5
Fixes #6
Fixes #7

Release impact: patch release of @tanstack/highlight.

Validation

• pnpm run verify
• 55 tests pass
• TSX: 9,439 bytes minified / 3,959 gzip
• all-language bundle: 22,762 bytes minified / 8,227 gzip
• 10,020 blocks highlighted in 256 ms against the 1,200 ms budget

Summary by CodeRabbit

• Bug Fixes

  • Improved parsing of TypeScript and JSX code, including multiline generics, tags, attributes, fragments, and nested expressions.
  • Corrected tokenization around JSX text, apostrophes, quoted strings, and embedded expressions.
  • Improved handling of unclassified prose and syntax at JSX boundaries.

• Tests

  • Added expanded regression coverage for JSX and TSX parsing scenarios.

[coderabbitai]

Warning

Review limit reached

@tannerlinsley, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 50 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: bde90744-d038-4d81-80e5-69446acdf3f2

📥 Commits

Reviewing files that changed from the base of the PR and between e51a6f8 and 741cf59.

📒 Files selected for processing (2)

• src/internal/script.ts
• test/regressions.test.ts

📝 Walkthrough

Walkthrough

The TSX parser now combines lexical and JSX scanning, keeps JSX text separate from semantic classification, handles multiline tags and expressions, and corrects quote and brace boundary handling. Regression tests cover JSX text, attributes, fragments, generics, nesting, and token reconstruction.

Changes

TSX parser correction

Layer / File(s)	Summary
Unified JSX and script scanner src/internal/script.ts	collectScriptRanges now scans script syntax, JSX tags, attributes, fragments, embedded expressions, and JSX text through one recursive parser. Semantic ranges inside JSX text are excluded.
JSX boundary and quote handling src/internal/script.ts	Tag brace depth is decremented directly. Tag scanning no longer rejects newlines outside braces. Quoted content ends at newlines for all quote types.
TSX tokenization regression coverage test/regressions.test.ts	Tests cover multiline generics, JSX prose, apostrophes, multiline attributes, nested expressions, fragments, syntax boundaries, exact token classes, and source reconstruction.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant collectScriptRanges
  participant collectScriptInitialRanges
  participant semanticPatterns
  collectScriptRanges->>collectScriptInitialRanges: scan script syntax and JSX structure
  collectScriptInitialRanges->>collectScriptRanges: return lexical ranges and JSX-text ranges
  collectScriptRanges->>semanticPatterns: apply semantic patterns outside JSX text

Loading

Possibly related PRs

• TanStack/highlight#1: Both changes modify JSX and script range collection in src/internal/script.ts.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely identifies the primary change: fixing JSX scanning contexts.
Description check	✅ Passed	The description includes the required Changes and Validation sections with scope, issue references, tests, bundle sizes, and performance results.
Linked Issues check	✅ Passed	The implementation and regression tests address the requirements in issues [#5], [#6], and [#7].
Out of Scope Changes check	✅ Passed	The source and regression-test changes are limited to JSX scanning, TSX tokenization, and the linked issue requirements.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch taren/fix-jsx-scanning

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/internal/script.ts`:
- Line 416: Update findQuotedEnd and the callers used by findTagEnd and raw
quoted-source range scanning so JSX attribute strings continue through newline
characters and close only at their matching quote. Preserve newline termination
for JavaScript template and string literal handling, and ensure multiline JSX
attributes do not cause later quotes to open unintended lexical ranges.
- Around line 199-207: Update the recursive collectScriptInitialRanges call in
the nested JSX attribute-expression path to enable JSX scanning and pass the
shared jsxText mask, preserving the existing range and boundary arguments. Add a
registry regression covering nested JSX such as child={<Label>can't</Label>} and
verify the nested tag is classified correctly.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6c17674a-dbc7-4b15-b692-e13276fa1fdd

📥 Commits

Reviewing files that changed from the base of the PR and between e749554 and e51a6f8.

📒 Files selected for processing (2)

• src/internal/script.ts
• test/regressions.test.ts