Skip to content

fix: exclude precision-losing integer-to-float conversions from CastExpr::check_bigger_cast (#23808) - #23809

Merged
alamb merged 3 commits into
apache:mainfrom
getChan:fix-23808-check-bigger-cast
Jul 26, 2026
Merged

fix: exclude precision-losing integer-to-float conversions from CastExpr::check_bigger_cast (#23808)#23809
alamb merged 3 commits into
apache:mainfrom
getChan:fix-23808-check-bigger-cast

Conversation

@getChan

@getChan getChan commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

ref. #23807

CastExpr::check_bigger_cast is used to determine whether a cast is a widening (order-preserving) conversion. Currently, it classifies Int32 -> Float32, UInt32 -> Float32, Int64 -> Float64, and UInt64 -> Float64 as widening casts.

However, integer-to-float conversions for 32-bit and 64-bit integers lose precision when values exceed the mantissa bit limit (24 bits for Float32, 53 bits for Float64). For example:
16_777_216_i32 as f32 == 16_777_217_i32 as f32 (both yield 16777216.0f32).

Because distinct integer inputs can collapse to the same float output, these casts are not strictly 1-to-1 (injective) and can break suffix sort key ordering in multi-column ordering analysis (e.g. [CAST(a AS Float32), b]).

What changes are included in this PR?

  • Updated CastExpr::check_bigger_cast to exclude precision-losing integer-to-float conversions (Int32/UInt32 -> Float32 and Int64/UInt64 -> Float64).
  • Added unit test test_check_bigger_cast_precision_loss to verify precision-losing casts return false while exact conversions (Int16 -> Float32, Int32 -> Float64, etc.) continue to return true.

Are these changes tested?

Yes, new unit test test_check_bigger_cast_precision_loss in cast.rs.

Are there any user-facing changes?

No breaking API changes. Internal optimizer behavior fix.

@github-actions github-actions Bot added the physical-expr Changes to the physical-expr crates label Jul 22, 2026
@codecov-commenter

codecov-commenter commented Jul 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.63%. Comparing base (9e3c71f) to head (0bc07d5).

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #23809   +/-   ##
=======================================
  Coverage   80.63%   80.63%           
=======================================
  Files        1091     1091           
  Lines      370720   370736   +16     
  Branches   370720   370736   +16     
=======================================
+ Hits       298934   298947   +13     
  Misses      53948    53948           
- Partials    17838    17841    +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nuno-faria nuno-faria left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @getChan. Since only EquivalenceProperties uses this, it should be ok. However, I think it's better to add some sqllogictests to force this bug. I explored this a bit and here is an example that returns invalid results for the int to float case:

Test data (test.csv):

k,v
1,16777217
2,16777216
-- t1 is defined with the sort order, t2 is not
> create external table t1 (k int, v int) stored as csv with order (v desc, k desc) location 'test.csv';
> create external table t2 (k int, v int) stored as csv location 'test.csv';

-- the result is wrong, since both v_ are equal the k should be switched
> select k, cast(v as float) v_ from t1 order by v_ desc, k desc;
+---+------------+
| k | v_         |
+---+------------+
| 1 | 16777216.0 |
| 2 | 16777216.0 |
+---+------------+

-- using the version without the sort works
> select k, cast(v as float) v_ from t2 order by v_ desc, k desc;
+---+------------+
| k | v_         |
+---+------------+
| 2 | 16777216.0 |
| 1 | 16777216.0 |
+---+------------+

Comment thread datafusion/physical-expr/src/expressions/cast.rs
@github-actions github-actions Bot added core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) labels Jul 25, 2026
@getChan
getChan force-pushed the fix-23808-check-bigger-cast branch from 48bc262 to 0bc07d5 Compare July 25, 2026 07:28

@nuno-faria nuno-faria left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @getChan.

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @getChan and @nuno-faria

@alamb
alamb added this pull request to the merge queue Jul 26, 2026
Merged via the queue into apache:main with commit 7576762 Jul 26, 2026
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate physical-expr Changes to the physical-expr crates sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix precision loss in CastExpr::check_bigger_cast for integer-to-float conversions

4 participants