fix(amber): store execution sizes as BIGINT to stop silent >2GiB truncation

[mengw15]

What changes were proposed in this PR?

updateResultSize / updateRuntimeStatsSize / updateConsoleMessageSize (WorkflowExecutionsResource) stored Long byte counts into INT columns via Integer.valueOf(size.toInt). Scala's Long.toInt keeps only the low 32 bits without raising, so a size ≥ 2 GiB wrapped silently — values in [2 GiB, 4 GiB) became negative — and UserQuotaResource, which sums result_size / runtime_stats_size / console_messages_size into a user's storage quota, reported corrupted totals. With BigObject (#4067) supporting >2 GB results, such sizes are reachable in practice.

• Widen the three columns to BIGINT in sql/texera_ddl.sql, with migration sql/updates/29.sql (registered as changelog changeSet 29) for existing deployments — a lossless in-place ALTER COLUMN ... TYPE BIGINT for each.
• Store the Long directly at the three write sites, dropping the .toInt narrowing (java.lang.Long.valueOf(size); the jOOQ-generated fields become Long from the widened schema).
• Adapt the quota reads in UserQuotaResource: the getOrElse(0).asInstanceOf[Integer] pattern would throw ClassCastException on the now-Long fields; simplified to Option(...).map(_.toLong).getOrElse(0L).
• Split the size write out of updateRuntimeStatsSize / updateConsoleMessageSize into (eid, size) overloads, mirroring the existing updateResultSize shape. The outer signatures are unchanged (callers untouched), but the DB write is now reachable without an Iceberg/LakeFS-backed document — so all three writes are directly testable.

Any related issues, documentation, discussions?

Closes #6978. Size columns introduced with the execution result/stats storage; >2 GB results enabled by #4067 (BigObject).

How was this PR tested?

• Added regression cases to WorkflowExecutionsResourceSpec (unit spec on embedded Postgres, no external infra): each of the three size writes stores a 3 GiB value and is asserted to round-trip untruncated, plus the two no-URI no-op branches. Verified the truncation case fails before the fix — -1073741824 did not equal 3221225472 (the low-32-bit wrap) — and passes after.
• Full spec run locally: 27/27 passed (jOOQ regenerated against the widened schema; the embedded test DB loads the updated texera_ddl.sql).
• sql/updates/29.sql applied cleanly to a local Postgres 15 texera_db (three ALTER TABLEs in one transaction); columns verified bigint afterwards.
• WorkflowExecutionService/scalafmtCheck (main + Test) passes.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (claude-opus-4-8)

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @kunwp1, @seongjinyoon, @Yicong-Huang
  You can notify them by mentioning @kunwp1, @seongjinyoon, @Yicong-Huang in a comment.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.68%. Comparing base (ab58d47) to head (ab1f925).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6980      +/-   ##
============================================
- Coverage     79.07%   78.68%   -0.40%     
- Complexity     3784     3787       +3     
============================================
  Files          1160     1160              
  Lines         46105    46041      -64     
  Branches       5115     5109       -6     
============================================
- Hits          36457    36226     -231     
- Misses         8025     8187     +162     
- Partials       1623     1628       +5     

Flag	Coverage Δ		*Carryforward flag
access-control-service	70.00% <ø> (ø)
agent-service	76.76% <ø> (ø)		Carriedforward from 6c42c5f
amber	72.60% <100.00%> (+0.42%)	⬆️
computing-unit-managing-service	20.49% <ø> (ø)
config-service	66.66% <ø> (ø)
file-service	67.21% <ø> (ø)
frontend	82.64% <ø> (-0.36%)	⬇️	Carriedforward from 6c42c5f
notebook-migration-service	78.94% <ø> (ø)
pyamber	92.46% <ø> (-3.88%)	⬇️	Carriedforward from 6c42c5f
workflow-compiling-service	26.31% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 15 worse · ⚪ 0 noise (<±5%) · 0 without baseline

CI benchmark results are noisy; treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	407	0.248	22,992/32,067/32,067 us	🔴 +126.7% / 🔴 +102.9%
🔴	bs=100 sw=10 sl=64	929	0.567	108,500/131,498/131,498 us	🔴 +46.4% / 🔴 +22.5%
🔴	bs=1000 sw=10 sl=64	1,094	0.667	916,077/976,026/976,026 us	🔴 +16.8% / 🟢 -7.5%

Baseline details

Latest main 5040cad from 2026-07-28T15:16:10.471Z

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	407 tuples/sec	920.31 tuples/sec	787.55 tuples/sec	-55.8%	-48.3%
bs=10 sw=10 sl=64	MB/s	0.248 MB/s	0.562 MB/s	0.481 MB/s	-55.8%	-48.4%
bs=10 sw=10 sl=64	p50	22,992 us	10,402 us	12,255 us	+121.0%	+87.6%
bs=10 sw=10 sl=64	p95	32,067 us	14,143 us	15,802 us	+126.7%	+102.9%
bs=10 sw=10 sl=64	p99	32,067 us	18,427 us	19,008 us	+74.0%	+68.7%
bs=100 sw=10 sl=64	throughput	929 tuples/sec	1,224 tuples/sec	997.81 tuples/sec	-24.1%	-6.9%
bs=100 sw=10 sl=64	MB/s	0.567 MB/s	0.747 MB/s	0.609 MB/s	-24.1%	-6.9%
bs=100 sw=10 sl=64	p50	108,500 us	81,361 us	100,690 us	+33.4%	+7.8%
bs=100 sw=10 sl=64	p95	131,498 us	89,832 us	107,316 us	+46.4%	+22.5%
bs=100 sw=10 sl=64	p99	131,498 us	91,051 us	113,823 us	+44.4%	+15.5%
bs=1000 sw=10 sl=64	throughput	1,094 tuples/sec	1,270 tuples/sec	1,030 tuples/sec	-13.9%	+6.2%
bs=1000 sw=10 sl=64	MB/s	0.667 MB/s	0.775 MB/s	0.629 MB/s	-14.0%	+6.1%
bs=1000 sw=10 sl=64	p50	916,077 us	784,430 us	981,213 us	+16.8%	-6.6%
bs=1000 sw=10 sl=64	p95	976,026 us	838,600 us	1,027,605 us	+16.4%	-5.0%
bs=1000 sw=10 sl=64	p99	976,026 us	859,192 us	1,055,466 us	+13.6%	-7.5%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,491.58,200,128000,407,0.248,22992.15,32067.24,32067.24
1,100,10,64,20,2153.99,2000,1280000,929,0.567,108500.41,131498.36,131498.36
2,1000,10,64,20,18289.81,20000,12800000,1094,0.667,916077.34,976025.89,976025.89

[Copilot]

Pull request overview

This PR fixes a quota/accounting correctness bug in Amber by eliminating silent truncation when persisting execution-related byte sizes that can exceed 2GiB (now reachable with BigObject support). It updates the database schema to store these sizes as BIGINT, updates the write/read paths to use Long end-to-end, and adds a regression test to prevent reintroduction.

Changes:

• Widened result_size, runtime_stats_size, and console_messages_size columns from INT to BIGINT (DDL + Liquibase migration).
• Updated WorkflowExecutionsResource to persist byte counts as Long without narrowing to Int.
• Updated UserQuotaResource to read/sum these fields as Long safely, and added a >2GiB round-trip regression test.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
sql/updates/29.sql	Adds migration to widen the three execution-size columns to BIGINT.
sql/texera_ddl.sql	Updates baseline schema so new deployments create the columns as BIGINT.
sql/changelog.xml	Registers the new migration as changeSet 29.
amber/src/test/scala/org/apache/texera/web/resource/dashboard/user/workflow/WorkflowExecutionsResourceSpec.scala	Adds regression coverage for storing a 3GiB size without truncation.
amber/src/main/scala/org/apache/texera/web/resource/dashboard/user/workflow/WorkflowExecutionsResource.scala	Stops narrowing Long sizes to Int when persisting to the DB.
amber/src/main/scala/org/apache/texera/web/resource/dashboard/user/quota/UserQuotaResource.scala	Adjusts quota aggregation to read/sum size fields as Long after the schema widening.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[aglinxinyuan]

LGTM!

[Yicong-Huang]

@mengw15 is it possible to increase the test coverage of your patch change?

[mengw15]

@mengw15 is it possible to increase the test coverage of your patch change?

I increased from 33% to 73%, I will check tomorrow if I can increase further

[mengw15]

@mengw15 is it possible to increase the test coverage of your patch change?

increased to 100%

[github-actions]

Backport PR opened: draft #7050 (#7050) to release/v1.2, assigned to @mengw15 — needs manual work because the cherry-pick conflicts.

[Mrudhulraj]

Raised a regression type-mimatch error #7125

cc: @mengw15

[Mrudhulraj]

Raised a regression type-mimatch error #7125

cc: @mengw15

Closed as it was a ddl change. Apologise for raising qrong bug.