feat(storage): add model metadata tables - #6799
Conversation
Automated Reviewer SuggestionsBased on the
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| ⚪ | bs=10 sw=10 sl=64 | 419 | 0.256 | 23,401/31,031/31,031 us | ⚪ within ±5% / 🔴 +92.5% |
| 🔴 | bs=100 sw=10 sl=64 | 927 | 0.566 | 105,383/155,687/155,687 us | 🔴 +16.5% / 🔴 +44.9% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,127 | 0.688 | 885,944/918,819/918,819 us | ⚪ within ±5% / 🟢 -13.0% |
Baseline details
Latest main a9f384e from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 419 tuples/sec | 439 tuples/sec | 760.2 tuples/sec | -4.6% | -44.9% |
| bs=10 sw=10 sl=64 | MB/s | 0.256 MB/s | 0.268 MB/s | 0.464 MB/s | -4.5% | -44.8% |
| bs=10 sw=10 sl=64 | p50 | 23,401 us | 22,596 us | 12,781 us | +3.6% | +83.1% |
| bs=10 sw=10 sl=64 | p95 | 31,031 us | 31,851 us | 16,118 us | -2.6% | +92.5% |
| bs=10 sw=10 sl=64 | p99 | 31,031 us | 31,851 us | 19,074 us | -2.6% | +62.7% |
| bs=100 sw=10 sl=64 | throughput | 927 tuples/sec | 960 tuples/sec | 988.68 tuples/sec | -3.4% | -6.2% |
| bs=100 sw=10 sl=64 | MB/s | 0.566 MB/s | 0.586 MB/s | 0.603 MB/s | -3.4% | -6.2% |
| bs=100 sw=10 sl=64 | p50 | 105,383 us | 100,768 us | 101,448 us | +4.6% | +3.9% |
| bs=100 sw=10 sl=64 | p95 | 155,687 us | 133,606 us | 107,410 us | +16.5% | +44.9% |
| bs=100 sw=10 sl=64 | p99 | 155,687 us | 133,606 us | 120,258 us | +16.5% | +29.5% |
| bs=1000 sw=10 sl=64 | throughput | 1,127 tuples/sec | 1,122 tuples/sec | 1,023 tuples/sec | +0.4% | +10.2% |
| bs=1000 sw=10 sl=64 | MB/s | 0.688 MB/s | 0.685 MB/s | 0.624 MB/s | +0.4% | +10.2% |
| bs=1000 sw=10 sl=64 | p50 | 885,944 us | 888,810 us | 983,888 us | -0.3% | -10.0% |
| bs=1000 sw=10 sl=64 | p95 | 918,819 us | 930,691 us | 1,030,100 us | -1.3% | -10.8% |
| bs=1000 sw=10 sl=64 | p99 | 918,819 us | 930,691 us | 1,056,513 us | -1.3% | -13.0% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,476.80,200,128000,419,0.256,23401.03,31030.57,31030.57
1,100,10,64,20,2157.09,2000,1280000,927,0.566,105383.05,155687.17,155687.17
2,1000,10,64,20,17739.14,20000,12800000,1127,0.688,885944.17,918818.90,918818.90|
/request-review @aicam |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6799 +/- ##
============================================
- Coverage 75.20% 75.20% -0.01%
+ Complexity 3447 3446 -1
============================================
Files 1160 1160
Lines 45807 45807
Branches 5071 5071
============================================
- Hits 34451 34450 -1
Misses 9745 9745
- Partials 1611 1612 +1
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
This PR is atomic but does not include any logic, please add more code to this PR to show the minimum way it will be used, its not possible to review a schema change without checking its effect on logics. |
What changes were proposed in this PR?
Adds the database tables that track ML models as a first-class resource in Texera. The
model*tables mirror the existingdataset*tables — their own primary key (mid), their own LakeFS repo namespace — andadd model-specific attributes (
framework,format):model— the model asset (owner, name, visibility, description, cover image, framework, format)model_user_access— per-user privilege (reusesprivilege_enum)model_version— versions of a model (mvid,version_hash)model_upload_session/model_upload_session_part— multipart upload bookkeepingmodel_user_likes/model_view_count— like/view countersThe change is pure schema, with no application code and no behavior change:
sql/texera_ddl.sql(with matchingDROP TABLEentries and FK-safe ordering).sql/updates/30.sql(allCREATE TABLE IF NOT EXISTS, no data backfill), registered aschangeSet id="30"insql/changelog.xml.Any related issues, documentation, discussions?
Closes #6496.
How was this PR tested?
Validated against a throwaway
groonga/pgroonga:4.0.1-debian-15Postgres:sql/texera_ddl.sqlruns end-to-end with no errors; all 7model*tables are created with the expected primary keys, foreign keys (ON DELETE CASCADE), unique constraints, andCHECKconstraints.sql/updates/30.sqlapplies cleanly on a DB that already has the dataset tables but no model tables, and is idempotentWas this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)