[FLINK-37925][table] Define VARIANT cast rules so a cast never alters the stored value

[raminqaf]

What is the purpose of the change

Follow-up to the initial VARIANT to primitive cast support in FLINK-37925. It settles the cast semantics under one rule and turns the string cast into a real value cast.

A cast from a VARIANT succeeds only when the target holds the stored value without altering it. A value is never wrapped, rounded, truncated, or padded to make it fit; anything else fails CAST and returns NULL for TRY_CAST.

Stored kind	Succeeds for
integer kinds	an integer target in range, DECIMAL, FLOAT, DOUBLE
FLOAT, DOUBLE	FLOAT, DOUBLE
DECIMAL	DECIMAL without rounding, FLOAT, DOUBLE
BOOLEAN, DATE	the same type
TIMESTAMP	TIMESTAMP(p) that keeps the fractional seconds
TIMESTAMP_LTZ	TIMESTAMP_LTZ(p) that keeps the fractional seconds
BYTES	BINARY(n), VARBINARY(n)
any scalar	CHAR(n), VARCHAR(n), STRING
NULL	SQL NULL for any nullable target

The conditions in that table are:

• An integer target rejects a value out of its range, so CAST(PARSE_JSON('1000') AS TINYINT) fails instead of wrapping.
• A DECIMAL target has to fit the precision and the scale. Trailing zeros may be appended, so 42 reaches DECIMAL(5, 2) as 42.00, but a scale that would round is rejected.
• FLOAT and DOUBLE are the deliberate exception. They are approximate by definition, so they accept every numeric kind and drop decimal digits, rejecting only a magnitude they cannot represent, such as 1e40
  to a FLOAT.
• A timestamp keeps microseconds in a variant, so the target precision has to preserve its fractional seconds: .123 reaches TIMESTAMP(3) while .123456 does not.
• CHAR(n) and BINARY(n) require the exact length, VARCHAR(n) and VARBINARY(n) at most n. Nothing is padded or truncated.

Reading one kind as another is never implicit, so a DECIMAL is not read as an integer and a TIMESTAMP is not read as a TIMESTAMP_LTZ. To reach such a type the inner cast names the stored kind and a regular cast around it does the conversion, which then applies the usual rules and may round, truncate, or overflow:

CAST(CAST(PARSE_JSON('1000') AS SMALLINT) AS TINYINT)     -- returns -24 (after overflow)                                                                                                                           
CAST(CAST(PARSE_JSON('3.9') AS DECIMAL(2, 1)) AS INT)     -- returns 3 (truncated)                                                                                                                                  

CAST(VARIANT AS CHAR/VARCHAR) now extracts the scalar value, so a stored string is returned unquoted as foo. Previously it reused the JSON serialization and was indistinguishable from JSON_STRING, which
remains the way to obtain the JSON text where a string stays quoted as "foo". A variant holding an object, array, or binary value is not castable to a character string, and a variant storing a JSON null casts
to SQL NULL rather than to the text null.

Brief change log

• Add VariantCastUtils in flink-table-runtime holding the per-target checks: integer range, DECIMAL precision and scale, floating-point magnitude, timestamp precision, and character or binary length.
• VariantToPrimitiveCastRule: route every target through those checks. This also fixes two cases that silently altered the value, namely a timestamp keeping more precision than its declared type and a binary
  value being padded or truncated to the target length.
• VariantToStringCastRule: extract the scalar value instead of serializing to JSON, handle CHAR and VARCHAR directly to enforce the length, and map a null-valued variant to SQL NULL for a nullable target.
• LogicalTypeCasts: express VARIANT cast validation in the per-target rules and drop the JSON_STRING cast hint.

Verifying this change

This change added and updated tests:

• CastRulesTest: per-kind coverage of every numeric target including range rejection, DECIMAL precision and scale fitting, the FLOAT and DOUBLE acceptance of any numeric kind with overflow rejection, the timestamp precision rule, and the TIMESTAMP versus TIMESTAMP_LTZ split.
• CastFunctionITCase: the same through PARSE_JSON, plus string value extraction, strict CHAR and VARCHAR length, rejection of object and array values, a JSON null casting to SQL NULL, and the two nested casts above. Each failing case also asserts that TRY_CAST returns NULL.
• LogicalTypeCastsTest: the VARIANT cast support matrix.

Trade-offs

Rejecting a cast that would change the value is stricter than Snowflake and Spark, which coerce. For the kind-preserving part it matches Snowflake's AS_INTEGER and AS_DOUBLE accessor family, where a mismatch yields NULL. The reasoning is that a VARIANT cast is an extraction step, so any conversion decision belongs in an explicit second cast, and TRY_CAST provides the non-failing form.

Because the stored kind is only known per row, these are runtime failures and cannot be reported during validation.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): yes, javadoc-only change to the @PublicEvolving Variant interface, no signature change
• The serializers: no
• The runtime per-record code paths (performance sensitive): yes, only on the VARIANT cast path
• Anything that affects deployment or recovery: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no, it refines existing FLINK-37925 behavior
• If yes, how is the feature documented? docs and JavaDocs

[flinkbot]

CI report:

• f9bd3a0 UNKNOWN
• 83b53d4 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[twalthr]

Thank you @raminqaf. I left some feedback.

[twalthr]

Another round of comments.

[twalthr]

Thank you @raminqaf. We are almost there. But I had to post some more critical comments.

[twalthr]

This one we could rediscuss, right?

[twalthr]

This we did not discuss. Let's not allow this for now. Only string can be cast to string.

[twalthr]

only format on error, keep the hot path performant

[twalthr]

same comment as above

[twalthr]

this is again JSON and not SQL. e.g. TRUE vs true for variant boolean to string. as mentioned above. only string variant can cast to string. if you want to support more, come up with a proper string representation for each variant type. incl TS_LTZ in users time zone.