Skip to content

fix(clickhouse): coerce LIMIT before SQL interpolation - #2627

Closed
Bartok9 wants to merge 5 commits into
Canner:mainfrom
Bartok9:fix/clickhouse-coerce-limit
Closed

fix(clickhouse): coerce LIMIT before SQL interpolation#2627
Bartok9 wants to merge 5 commits into
Canner:mainfrom
Bartok9:fix/clickhouse-coerce-limit

Conversation

@Bartok9

@Bartok9 Bartok9 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Coerce limit to a non-negative int before ClickHouse LIMIT subquery wrapping.

Failure / reproduction

Before this change, ClickHouseConnector.query interpolated the raw {limit} directly into the generated SQL:

sql = f"SELECT * FROM ({sql}) LIMIT {limit}"

So a client-supplied bad value flowed straight into the query text. E.g. calling query(sql, limit=-1) produced:

SELECT * FROM (...) LIMIT -1

and a non-numeric value like limit="1; DROP TABLE t" interpolated verbatim — an injection-shaped input that should never reach the driver.

Fix

Add _coerce_limit and apply it at the start of query: reject negatives and non-integer strings, accept numeric strings, so only a validated non-negative int is ever interpolated.

Verification

cd core/wren && .venv/bin/python -m pytest tests/unit/test_clickhouse_coerce_limit.py -q
3 passed

Covers: negative rejection, injection-like/non-numeric rejection, numeric-string acceptance.

Duplicate check

Searched open PRs/issues for existing ClickHouse limit coercion. Related sibling connector PRs (#2624/#2625/#2626) apply the same pattern to other connectors; this PR is the ClickHouse-specific counterpart and does not overlap their files.

Summary by CodeRabbit

  • Bug Fixes
    • Improved validation of query LIMIT inputs by rejecting negative numeric values, including fractional inputs (decimals and fractions), before execution.
    • Prevented negative fractional values from being truncated into non-negative LIMIT values.
    • Strengthened limit handling to ensure only valid, non-negative integers are used in generated queries.
  • Tests
    • Updated and tightened unit coverage for fractional negative LIMIT coercion scenarios.

Reject negative and non-numeric limits before ClickHouse LIMIT wrapping.
@github-actions github-actions Bot added python Pull requests that update Python code core labels Jul 30, 2026
@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c857de2f-f488-455f-9bda-6bc435fae0cf

📥 Commits

Reviewing files that changed from the base of the PR and between a9773cc and 2c8a436.

📒 Files selected for processing (1)
  • core/wren/src/wren/connector/clickhouse.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • core/wren/src/wren/connector/clickhouse.py

Walkthrough

The ClickHouse connector now rejects negative numeric limits before integer coercion and validates limits before SQL interpolation. Unit tests cover fractional negatives, invalid strings, and numeric-string SQL generation.

Changes

ClickHouse limit validation

Layer / File(s) Summary
Limit coercion and query wiring
core/wren/src/wren/connector/clickhouse.py
Rejects negative numeric values before coercion and applies validated limits before SQL rewriting.
Limit validation tests
core/wren/tests/unit/test_clickhouse_coerce_limit.py
Tests integer, float, Decimal, and Fraction negatives, invalid strings, and safe numeric-string limit interpolation.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Poem

I’m a bunny guarding LIMIT seven,
No sneaky DROP shall enter heaven.
Negative hops are turned away,
Safe little integers lead the way.
SQL sprouts clean beneath the moon.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the main behavioral change to ClickHouse LIMIT handling.
Description check ✅ Passed The description covers summary, reproduction, fix, verification, and duplicate check, with only minor template details left implicit.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/wren/src/wren/connector/clickhouse.py`:
- Around line 386-397: Update _coerce_limit to reject negative numeric inputs
before int() truncates fractional values, while preserving safe coercion for
valid limits and existing None handling. Add a regression test covering
limit=-0.5 and assert that it raises ValueError rather than producing LIMIT 0.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c1115fa7-3bde-4d77-8236-7ad7f3ace67c

📥 Commits

Reviewing files that changed from the base of the PR and between 32d76bf and 654545d.

📒 Files selected for processing (2)
  • core/wren/src/wren/connector/clickhouse.py
  • core/wren/tests/unit/test_clickhouse_coerce_limit.py

Comment thread core/wren/src/wren/connector/clickhouse.py
@Bartok9

Bartok9 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Good catch, @coderabbitaiint(-0.5) truncates to 0 and slipped past the guard. Pushed eebb5ad: _coerce_limit now rejects any negative numeric (int/float) before int() truncation, plus a regression test for limit=-0.5.

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

@Bartok9 Thanks for fixing this. Rejecting negative numeric inputs before int() truncation, with the -0.5 regression test, addresses the bypass.

🐇 ✅


If you found this review helpful, would you consider giving us a shout-out on X?

Thank you for using CodeRabbit!

You are interacting with an AI system.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@core/wren/src/wren/connector/clickhouse.py`:
- Around line 394-400: Update the limit validation before int() coercion to
reject negative values for every accepted numeric type, including Decimal and
Fraction, so values such as -0.5 cannot become zero; alternatively, explicitly
reject unsupported numeric types. Preserve the existing non-negative coercion
and error behavior in the surrounding limit handling.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 088d79df-f424-4300-b0dc-eafd0e62395d

📥 Commits

Reviewing files that changed from the base of the PR and between 654545d and eebb5ad.

📒 Files selected for processing (2)
  • core/wren/src/wren/connector/clickhouse.py
  • core/wren/tests/unit/test_clickhouse_coerce_limit.py

Comment thread core/wren/src/wren/connector/clickhouse.py
Broaden the pre-int() negative guard from (int, float) to numbers.Number
so Decimal('-0.5') / Fraction(-1, 2) cannot slip through as LIMIT 0.
Adds regression tests for both.
@Bartok9

Bartok9 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @coderabbitai — good follow-up. Broadened the pre-int() negative guard from (int, float) to numbers.Number in 6c3053a, so Decimal('-0.5') and Fraction(-1, 2) are now rejected too rather than truncating to LIMIT 0. Added regression tests for both.

@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

git show --format=fuller --stat 6c3053a --
git diff 6c3053a^ 6c3053a -- \
  core/wren/src/wren/connector/clickhouse.py \
  core/wren/tests/unit/test_clickhouse_coerce_limit.py

Length of output: 3210


@Bartok9 Confirmed. numbers.Number now guards negative numeric values before int() in _coerce_limit, and the new Decimal("-0.5") and Fraction(-1, 2) regressions directly cover the prior truncation path. This closes the generalized negative-fraction bypass.

🐇 ✅


✏️ Learnings added
Learnt from: Bartok9
URL: https://github.com/Canner/WrenAI/pull/2627

Timestamp: 2026-07-30T06:19:05.981Z
Learning: In `core/wren/src/wren/connector/clickhouse.py`, `_coerce_limit` must reject negative `numbers.Number` inputs before calling `int()` because fractional negative values such as `Decimal("-0.5")` and `Fraction(-1, 2)` would otherwise truncate to `0` and be emitted as `LIMIT 0`.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

@Bartok9

Bartok9 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Fixed the lint failure in a9773cc — hoisted the Decimal/Fraction imports to module top-level (ruff PLC0415); they were function-local in the regression tests. ruff check is clean locally now.

@goldmedal

Copy link
Copy Markdown
Collaborator

Closing in favour of a single consolidated change — thanks for the work, the underlying tidy-up is worth doing, just not as one PR per connector.

Why this is being closed rather than reviewed:

  1. This is one mechanical change spread across eight PRs. The contribution bar added in docs: set an explicit contribution bar for agent-authored PRs #2602 asks for exactly this to be a single diff: "A mechanical change repeated across several files or connectors belongs in one PR, not one PR per file. Reviewers need to see the resulting convention in a single diff."

  2. A shared helper already exists in this same batch, and none of these PRs use it. refactor(connector): centralize LIMIT coercion #2624 adds coerce_limit() to connector/base.py. Every other PR in the series re-declares a private _coerce_limit in its own module instead of importing it. Merged as-is the repo would carry nine copies of the same function (the eight here plus the existing one at connector/mysql.py:44).

  3. The copies have already diverged before merge. base.py (refactor(connector): centralize LIMIT coercion #2624), postgres, trino, oracle, redshift and bigquery use int(limit) then a negativity check; fix(clickhouse): coerce LIMIT before SQL interpolation #2627 (clickhouse) additionally rejects fractional negatives such as -0.5 via numbers.Number; fix(duckdb): coerce LIMIT before SQL interpolation #2637 (duckdb) additionally rejects bool and non-integral float. Three different semantics for one contract is the specific outcome a shared helper prevents.

  4. Coverage is inconsistent. connector/canner.py:255 interpolates a bare {limit} and is not covered by any PR in the series, while bigquery, duckdb and redshift already interpolate {int(limit)} today — for those three the only behavioural delta is the negativity check.

On the fix: label and the stated failure. The reproduction in the description calls connector.query(sql, limit="1; DROP TABLE users") directly. Tracing the call paths:

  • run_sql in mcp_server.py declares limit: int | None, so a non-numeric string is rejected by tool-argument validation, and mcp_server.py:84 already rejects negatives and clamps to MAX_ROW_LIMIT.
  • The CLI declares --limit/-l as Optional[int], so a non-integer is rejected at parse time.
  • That leaves Engine.query(sql, limit) as a Python API. At that boundary the caller already supplies sql verbatim — anyone able to pass limit="1; DROP TABLE t" can pass that as sql instead. limit is not a lower-trust channel than sql there, so this is not an injection path.

What remains is genuine but smaller: a negative limit currently surfaces as a driver-level error instead of a clear ValueError, and the coercion contract is inconsistent across connectors. That is refactor:, per "fix: requires a reproducible failure that the change repairs."

What we would take instead — a single PR, refactor(connector): centralize LIMIT coercion, that:

  • keeps one coerce_limit() in connector/base.py, with the strictest semantics of the three variants above (reject bool, non-integral values, and negatives);
  • routes every interpolating connector through it — including canner.py, and replacing the private copy in mysql.py;
  • leaves ConnectorABC.query's signature as int | None (see the per-PR note on fix(duckdb): coerce LIMIT before SQL interpolation #2637 below);
  • tests the helper once in tests/unit/test_coerce_limit.py, with at most a smoke test per connector proving it is wired in, rather than repeating the same six cases eight times.

#2624 is the natural home for that; it is being kept open with a note to that effect.


On this PR specificallyclickhouse.py:401 is a bare-{limit} site. Your numbers.Number pre-check is the strictest variant in the series and is the one worth carrying into the shared helper.

@goldmedal goldmedal closed this Aug 3, 2026
@Bartok9

Bartok9 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @goldmedal — agreed on consolidating. Appreciate the detailed rationale (shared helper + one convention, not N private copies).

I'll fold this ClickHouse path into #2624 as the single refactor(connector): centralize LIMIT coercion home, carrying the stricter pre-int() semantics (numbers.Number negatives / fractional rejects) into the shared coerce_limit() and wiring the bare-{limit} sites (including here and canner.py) through it.

Closing as-is is the right call; thanks for the clear bar from #2602.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core python Pull requests that update Python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants