[SYSTEMDS-3946] Enable sending of large (>2GiB) FederatedRequests and…

[Biranavan-Parameswaran]

Federated transfers previously failed for payloads above 2GiB because the
single Netty frame size is bounded by a 32-bit length field, capping any
request or response at Integer.MAX_VALUE bytes.

This patch adds a streaming chunked codec that splits a large payload into
bounded frames on the sender and reassembles them on the receiver, so the
on-wire size is no longer limited by a single frame. A format detector and
format encoder select the chunked path only when the payload exceeds the
frame limit, leaving the existing small-message path unchanged to avoid
added overhead for the common case.

Adds FederatedMaxPayloadTest to exercise the boundary around the former
2GiB cap.

[ywcb00]

Thank you very much for the PR @Biranavan-Parameswaran :)
I left some minor comments in the code. Could you please have a look at it and resolve it if you find the time. Thanks.

[ywcb00]

We should use the regular encoder as long as we can, i.e., up to the largest possible message size. Can we increase this default threshold from 1.5GB to (INT_MAX - 1) bytes?

[Biranavan-Parameswaran]

Good suggestion, but INT_MAX-1 is actually unsafe here. Routing uses a size estimate, not the exact wire size, and the ObjectEncoder overflows its Integer.MAX_VALUE ByteBuf once serialization framing is added. I measured that cliff at about 1.990 GiB, so an estimate just under INT_MAX can still overflow on the wire. I set the threshold to 2000L << 20 (about 1.953 GiB), roughly 40 MB under the cliff, so the regular encoder is used as high as we safely can.

Note the routing also changed since your review. Responses now route by lineage cacheability instead of size, so STREAM_THRESHOLD is now the size guard on the object encoder path rather than the main router.

[Biranavan-Parameswaran]

Thanks for the review @ywcb00. All three comments are addressed, and I also ran experiments to back the design choices. Summary of what changed since the reviewed commit.

Comments

• Threshold (1): INT_MAX-1 can overflow on the wire because routing uses a size estimate. I measured the overflow cliff at about 1.990 GiB and set the threshold about 40 MB under it (2000L<<20). Full reasoning in the inline reply.
• Redundant catch (2): removed.
• Rename (3): FederatedFormatDetector is now FederatedFormatDecoder.

Follow up work

• Chunk size 4 MB (1<<22). Swept 256K, 1M, 4M and 8M across four payloads, 48 runs. Median round trip in seconds:

  payload	256K	1M	4M	8M
  0.08 MB	0.755	0.787	0.900	0.835
  8 MB	1.208	1.165	1.099	1.066
  512 MB	10.707	10.203	10.013	10.443
  1074 MB	17.439	16.020	15.578	15.917

  4 MB is the robust optimum. It wins on large and xl and stays stable as the payload doubles. 8M starts to regress on big payloads and 256K pays too much framing overhead. Small payloads are one frame so they are chunk size indifferent. It also caps in flight buffering at 16 x 4 MB, so 64 MB.

• Streaming vs object encoder. Mean elapsed in seconds, object encoder vs chunked:

  payload	object encoder	chunked
  tiny	1.36	1.46
  mid	2.16	2.21
  512 MB	74.1	18.5
  1 GiB	305.5	29.9

  Chunked matches the object encoder on small payloads and is 4x to 10x faster on large ones.

• Scaling past 2 GB. Swept 2.2 to 4.0 GB, all correct with 0 exceptions and 0 OOM:

  payload	round trip s	MB/s	peak worker MB
  2.2 GB	11.56	190	5961
  2.6 GB	14.26	182	7689
  3.0 GB	16.14	186	6744
  4.0 GB	25.74	155	9795

  Throughput stays flat around 155 to 190 MB/s and worker peak memory tracks payload at about 2.2 to 3.0x with no second full payload buffer. The old object encoder crashes near 1.99 GiB, so this is the range the codec exists for.

• Routing now based on lineage cacheability instead of size. Lineage cacheable responses use the object encoder, everything else streams. New FederatedFormatRoutingTest covers the four cases.

• Close race fix. The worker CloseListener treated a client close during a chunked response drain as a write failure and tried a second write on the dead channel. It now treats an inactive channel mid stream as a normal end of stream.

• Slimmed FederatedChunkEncoder to a factory and tightened the routing predicate.

[codecov]

Codecov Report

❌ Patch coverage is 76.27907% with 51 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.53%. Comparing base (e4f0987) to head (1567c28).
⚠️ Report is 55 commits behind head on main.

Files with missing lines	Patch %	Lines
...ontrolprogram/federated/FederatedChunkEncoder.java	73.41%	19 Missing and 2 partials ⚠️
...ontrolprogram/federated/FederatedChunkDecoder.java	72.97%	14 Missing and 6 partials ⚠️
...ntrolprogram/federated/FederatedWorkerHandler.java	60.00%	3 Missing and 1 partial ⚠️
...ntrolprogram/federated/FederatedFormatEncoder.java	85.71%	2 Missing and 1 partial ⚠️
...ntrolprogram/federated/FederatedFormatDecoder.java	81.81%	1 Missing and 1 partial ⚠️
...me/controlprogram/federated/FederatedResponse.java	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2496      +/-   ##
============================================
+ Coverage     71.47%   71.53%   +0.06%     
- Complexity    48883    50264    +1381     
============================================
  Files          1573     1628      +55     
  Lines        189238   194506    +5268     
  Branches      37128    37983     +855     
============================================
+ Hits         135261   139149    +3888     
- Misses        43530    44430     +900     
- Partials      10447    10927     +480     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.