feat(computing-unit): out-of-pod dataset mount infrastructure

[aicam]

What changes were proposed in this PR?

Perform the FUSE mount for dataset repositories outside the (unprivileged) computing-unit pod — the infrastructure foundation of the dataset-mounting feature (#6606).

• texera-mounter DaemonSet — a per-node privileged agent (bin/mounter/mounter.py + tests, dockerfile, helm daemonset/rbac/values) that runs GeeseFS on a pod's behalf. The read-only mount is exposed back into the CU pod via Kubernetes mount propagation, scoped to that computing unit, so the pod that runs untrusted user code stays unprivileged.
• Unprivileged CU pod wiring (KubernetesClient) — the propagation volume + node-IP / mounter env.
• File-service JWT S3 proxy (S3ProxyServlet) — fronts the LakeFS S3 gateway: verifies the pod's JWT, checks the user's read access, and re-signs to LakeFS with credentials held only server-side. No global credential ever enters the pod.

Foundation only — nothing triggers a mount yet; the platform integration (engine client + per-CU mount API + UI + UDF bindings) comes in the follow-up PR.

Any related issues, documentation, discussions?

Closes #6862 · part of #6606.

How was this PR tested?

• sbt FileService/compile ComputingUnitManagingService/compile green.
• The mounter has its own 397-line pytest suite (bin/mounter/tests) covering mount/unmount, the pod-deletion reaper, and dead-mount self-heal; the proxy's request-parsing helpers are unit-tested (S3ProxyServletSpec).
• Validated end-to-end on a single-node minikube: a Python UDF read a ~2 GB sharded PyTorch model from a propagated mount via torch.load with bit-exact output; the proxy's JWT authorization was exercised for both an authorized user (200) and an unauthorized repository (403 + refused to mount).

Note on patch coverage: most of this PR is inherently integration/IO code — the S3 proxy's request forwarding + re-signing (needs a live LakeFS gateway) and the Kubernetes pod wiring (fabric8 has no mock server in this repo). codecov's patch % therefore reads low even though the behavior is covered by the end-to-end validation above and by the mounter's pytest suite — which codecov does not instrument, being a bin/ script rather than a build module. We'd appreciate reviewers weighing the patch-coverage signal in that light.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 4.8

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @Ma77Ball, @eugenegujing
  You can notify them by mentioning @Ma77Ball, @eugenegujing in a comment.

[github-actions]

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 2 worse · ⚪ 11 noise (<±5%) · 0 without baseline

Compared against main f02dd2f benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	453	0.276	20,926/30,741/30,741 us	🔴 +5.4% / 🔴 +85.2%
🟢	bs=100 sw=10 sl=64	963	0.588	103,144/116,161/116,161 us	🟢 -11.8% / 🔴 +6.5%
⚪	bs=1000 sw=10 sl=64	1,083	0.661	923,020/973,037/973,037 us	⚪ within ±5% / 🟢 -9.4%

Baseline details

Latest main f02dd2f from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	453 tuples/sec	473 tuples/sec	754.55 tuples/sec	-4.2%	-40.0%
bs=10 sw=10 sl=64	MB/s	0.276 MB/s	0.289 MB/s	0.461 MB/s	-4.5%	-40.1%
bs=10 sw=10 sl=64	p50	20,926 us	21,252 us	12,816 us	-1.5%	+63.3%
bs=10 sw=10 sl=64	p95	30,741 us	29,165 us	16,594 us	+5.4%	+85.2%
bs=10 sw=10 sl=64	p99	30,741 us	29,165 us	19,806 us	+5.4%	+55.2%
bs=100 sw=10 sl=64	throughput	963 tuples/sec	964 tuples/sec	969.38 tuples/sec	-0.1%	-0.7%
bs=100 sw=10 sl=64	MB/s	0.588 MB/s	0.588 MB/s	0.592 MB/s	0.0%	-0.6%
bs=100 sw=10 sl=64	p50	103,144 us	102,846 us	103,584 us	+0.3%	-0.4%
bs=100 sw=10 sl=64	p95	116,161 us	131,641 us	109,097 us	-11.8%	+6.5%
bs=100 sw=10 sl=64	p99	116,161 us	131,641 us	117,304 us	-11.8%	-1.0%
bs=1000 sw=10 sl=64	throughput	1,083 tuples/sec	1,085 tuples/sec	1,004 tuples/sec	-0.2%	+7.9%
bs=1000 sw=10 sl=64	MB/s	0.661 MB/s	0.662 MB/s	0.613 MB/s	-0.2%	+7.9%
bs=1000 sw=10 sl=64	p50	923,020 us	921,358 us	1,002,357 us	+0.2%	-7.9%
bs=1000 sw=10 sl=64	p95	973,037 us	983,661 us	1,046,463 us	-1.1%	-7.0%
bs=1000 sw=10 sl=64	p99	973,037 us	983,661 us	1,073,661 us	-1.1%	-9.4%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,441.71,200,128000,453,0.276,20926.45,30740.64,30740.64
1,100,10,64,20,2075.83,2000,1280000,963,0.588,103143.72,116160.56,116160.56
2,1000,10,64,20,18462.71,20000,12800000,1083,0.661,923020.33,973036.86,973036.86

[codecov-commenter]

Codecov Report

❌ Patch coverage is 11.76471% with 90 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.77%. Comparing base (f02dd2f) to head (ddf160e).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...rg/apache/texera/service/util/S3ProxyServlet.scala	5.79%	65 Missing ⚠️
.../apache/texera/service/util/KubernetesClient.scala	4.00%	24 Missing ⚠️
.../scala/org/apache/texera/service/FileService.scala	0.00%	1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (11.76%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6866      +/-   ##
============================================
+ Coverage     77.30%   82.77%   +5.46%     
+ Complexity     3524     2354    -1170     
============================================
  Files          1161      893     -268     
  Lines         45922    37691    -8231     
  Branches       5101     3968    -1133     
============================================
- Hits          35501    31199    -4302     
+ Misses         8840     5357    -3483     
+ Partials       1581     1135     -446     

Flag	Coverage Δ		*Carryforward flag
access-control-service	70.00% <ø> (ø)
agent-service	76.76% <ø> (ø)		Carriedforward from f02dd2f
amber	85.90% <100.00%> (+16.80%)	⬆️
computing-unit-managing-service	19.37% <4.00%> (-1.13%)	⬇️
config-service	66.66% <ø> (ø)
file-service	59.49% <5.71%> (-7.72%)	⬇️
frontend	82.58% <ø> (ø)		Carriedforward from f02dd2f
notebook-migration-service	78.94% <ø> (ø)
pyamber	92.15% <ø> (ø)		Carriedforward from f02dd2f
workflow-compiling-service	55.14% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[chenlica]

@aicam Please revise the diagram to use a layout consistent with other diagrams.