From 5fe9a2fbf7d6c9811f2ae12d975baf2479841045 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Wed, 15 Jul 2026 19:55:08 +1000 Subject: [PATCH 1/4] feat(stack-supabase): EQL v3 encrypted-JSON querying (containment + selectors) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #650. The v3 Supabase adapter gains encrypted-JSON querying on types.Json columns: - contains(col, subDocument) — exact encrypted ste_vec containment. A live PostgREST probe established the wire form: PostgREST casts the `cs.` filter operand to the column's own domain, so a storage-encrypted needle reaches eql_v3."@>"(eql_v3_json, eql_v3_json) → ste_vec_contains. The operand is a full storage envelope, the same INTERIM shape (and GET-query-string caveat) as every other v3 filter operand. - selectorEq/selectorNe(col, path, value) — JSONPath equality at a dot-notation path, compiled to containment of the reconstructed needle ({user:{role:'admin'}}); ne compiles to not.cs and INCLUDES rows lacking the path, mirroring the Drizzle selector's documented semantics (verified live). Paths/leaves share the Drizzle adapter's validation. - Selector ORDERING is not expressible over PostgREST: arrow paths get to_jsonb-wrapped (domain stripped) and bare-column comparison is blocked by design. Filed cipherstash/encrypt-query-language#407 proposing a needle-comparison overload; until then the methods don't exist and the skill points at Drizzle's ops.selector() for ordering. Core: QueryTypesForColumn gains the searchableJson arm (types.Json no longer resolves to never, so the typed key sets stop erasing JSON columns), and the JSONPath selector-path helpers move from stack-drizzle to @cipherstash/stack/adapter-kit (drizzle re-exports them unchanged) so both adapters share one validation surface. Adapter plumbing: V3SearchableJsonKeys key set + typed contains overload + selector methods on both builder interfaces; the capability resolver re-types the cs-collected term to searchableJson on JSON columns (free-text and JSON capabilities are mutually exclusive by construction); matches() steers to contains()/selectorEq() on JSON columns; not(col,'contains',…) is allowed on JSON (honest negated exact containment) and stays rejected on text. Tests: 17 unit (operand routing, guards, path validation), 6 type-level (key-set narrowing, operand shapes), 11 live integration with real crypto against real PostgREST as anon (containment, selector eq/ne incl. absent-path inclusion, raw cs, capability rejections). Full suites: stack 816, drizzle 368 unit + 16 live JSON, supabase 468 unit + 677 integration — all green. Claude-Session: https://claude.ai/code/session_01MxTTPaPP16m6br7Hoab94w --- .changeset/supabase-v3-json-querying.md | 22 ++ packages/stack-drizzle/src/v3/operators.ts | 115 +-------- .../__tests__/helpers/supabase-mock.ts | 9 +- .../__tests__/supabase-v3-builder.test.ts | 13 +- .../__tests__/supabase-v3-json.test.ts | 225 ++++++++++++++++++ .../__tests__/supabase-v3.test-d.ts | 59 +++++ .../integration/json.integration.test.ts | 178 ++++++++++++++ packages/stack-supabase/src/helpers.ts | 8 +- .../stack-supabase/src/query-builder-v3.ts | 155 ++++++++++-- packages/stack-supabase/src/types.ts | 96 +++++++- packages/stack/src/adapter-kit.ts | 11 + packages/stack/src/eql/v3/columns.ts | 9 +- packages/stack/src/eql/v3/selector-path.ts | 117 +++++++++ skills/stash-supabase/SKILL.md | 45 +++- 14 files changed, 924 insertions(+), 138 deletions(-) create mode 100644 .changeset/supabase-v3-json-querying.md create mode 100644 packages/stack-supabase/__tests__/supabase-v3-json.test.ts create mode 100644 packages/stack-supabase/integration/json.integration.test.ts create mode 100644 packages/stack/src/eql/v3/selector-path.ts diff --git a/.changeset/supabase-v3-json-querying.md b/.changeset/supabase-v3-json-querying.md new file mode 100644 index 000000000..eb6fe44fb --- /dev/null +++ b/.changeset/supabase-v3-json-querying.md @@ -0,0 +1,22 @@ +--- +'@cipherstash/stack-supabase': minor +'@cipherstash/stack': minor +--- + +Encrypted-JSON querying on the v3 Supabase surface (#650). A `types.Json` +column now supports exact encrypted containment — `contains(col, subDocument)` +(ste_vec `@>` via PostgREST `cs`, with the sub-document storage-encrypted +against the column) — and JSONPath selector predicates: `selectorEq(col, path, +value)` and `selectorNe(col, path, value)` (dot-notation paths; `ne` includes +rows where the path is absent, mirroring the Drizzle selector's semantics). +Raw `.filter(col, 'cs', subDocument)` and `not(col, 'contains', …)` route +through the same encrypted path. Selector ordering is not expressible over +PostgREST yet (needs an EQL-bundle overload — see +cipherstash/encrypt-query-language#407); the Drizzle integration's +`ops.selector()` covers ordering today. + +In core, `QueryTypesForColumn` gains the `searchableJson` arm (a `types.Json` +column no longer resolves to `never`, so typed adapter key sets can include +it), and the JSONPath selector-path helpers the Drizzle adapter introduced in +#651 moved to `@cipherstash/stack/adapter-kit` so both adapters share one +validation surface (`@cipherstash/stack-drizzle` re-exports them unchanged). diff --git a/packages/stack-drizzle/src/v3/operators.ts b/packages/stack-drizzle/src/v3/operators.ts index abba645d0..35830ee1a 100644 --- a/packages/stack-drizzle/src/v3/operators.ts +++ b/packages/stack-drizzle/src/v3/operators.ts @@ -1,8 +1,12 @@ import type { Result } from '@byteslice/result' import type { AuditConfig } from '@cipherstash/stack/adapter-kit' import { + jsonPathOf, matchNeedleError, + parseSelectorSegments, + reconstructSelectorDocument, stripDomainSchema, + unsupportedLeafReason, } from '@cipherstash/stack/adapter-kit' import type { AnyEncryptedV3Column, @@ -76,113 +80,6 @@ type OperandEncryptionClient = { encrypt(value: never, opts: never): ChainableOperation } -/** - * Object keys that are prototype-pollution vectors — rejected outright (mirrors - * core's `FORBIDDEN_KEYS`), so a selector can never address them. - */ -const FORBIDDEN_SEGMENTS: ReadonlySet = new Set([ - '__proto__', - 'prototype', - 'constructor', -]) - -/** - * Parse a dot-notation JSONPath into its object-key segments. Rejects, each with - * a clear message: array-index/wildcard syntax (v1 is object-keys-only), the - * empty/root path, malformed paths (`..`, stray/leading/trailing dots, so we - * never silently query a *different* path), and prototype-pollution keys. - * `'$.a.b'` / `' a.b '` → `['a', 'b']`. Mirrors core's JSONPath normalization - * (not exported publicly — see the follow-up to share it via adapter-kit). - * - * Exported for unit testing; NOT re-exported from the package index. - */ -export function parseSelectorSegments(path: string): string[] { - const trimmed = path.trim() - let body = trimmed.startsWith('$') ? trimmed.slice(1) : trimmed - if (body.startsWith('.')) body = body.slice(1) - if (body === '') { - throw new Error( - `JSON selector path "${path}" addresses no field — use e.g. "$.a" or "$.a.b".`, - ) - } - if (/[[\]*]/.test(body)) { - throw new Error( - `JSON selector path "${path}" uses array/wildcard syntax, which is not yet supported — use dot-notation object keys (e.g. "$.a.b").`, - ) - } - const segments = body.split('.') - for (const segment of segments) { - if (segment === '') { - throw new Error( - `JSON selector path "${path}" is malformed (empty segment / ".." / stray dot) — use dot-notation object keys (e.g. "$.a.b").`, - ) - } - if (FORBIDDEN_SEGMENTS.has(segment)) { - throw new Error( - `JSON selector path "${path}" addresses the forbidden key "${segment}".`, - ) - } - } - return segments -} - -/** `$`-rooted JSONPath for `encryptQuery`'s selector needle. */ -function jsonPathOf(segments: string[]): string { - return `$.${segments.join('.')}` -} - -/** - * A selector compares a single scalar LEAF. Returns a reason string when `value` - * is unsupported — a non-scalar (object/array → that's `contains`), or a boolean - * under an ordering operator (no ordering term) — else `null`. The caller raises - * it as an {@link EncryptionOperatorError} with column context, so a bad value is - * an actionable SDK error rather than a deferred, opaque DB failure. - */ -function unsupportedLeafReason( - value: unknown, - ordering: boolean, -): string | null { - const isScalar = - value instanceof Date || - typeof value === 'number' || - typeof value === 'bigint' || - typeof value === 'string' || - typeof value === 'boolean' - if (!isScalar) { - return `a selector compares a scalar leaf, but got ${Array.isArray(value) ? 'an array' : 'an object'} — use contains() for sub-object matching.` - } - if (ordering && typeof value === 'boolean') { - return 'a boolean leaf has no ordering — use eq/ne (or contains()).' - } - return null -} - -/** - * Nest `value` under the segments: `['a','b']` → `{ a: { b: value } }`. The - * storage-needle document whose ste_vec entry at the path supplies the - * ciphertext-bearing comparison entry. - */ -export function reconstructSelectorDocument( - segments: string[], - value: unknown, -): Record { - // Null-prototype objects: a segment like `__proto__` must become an OWN key, - // not invoke the prototype setter (which would drop it and mis-serialize the - // needle). JSON.stringify ignores the [[Prototype]], so this serializes fine. - const root: Record = Object.create(null) - let cursor = root - segments.forEach((segment, index) => { - if (index === segments.length - 1) { - cursor[segment] = value - } else { - const next: Record = Object.create(null) - cursor[segment] = next - cursor = next - } - }) - return root -} - /** * A dedicated error for v3 operator gating and operand-encryption failures, * carrying the offending column/table/operator for diagnostics. @@ -192,6 +89,10 @@ export function reconstructSelectorDocument( * `./eql/v3/drizzle` — two independently-versioned public entry points — so the * duplication is deliberate, not an oversight. */ +// Path helpers now live in @cipherstash/stack/adapter-kit (shared with the +// Supabase adapter, #650); re-exported so existing imports keep working. +export { parseSelectorSegments, reconstructSelectorDocument } + export class EncryptionOperatorError extends Error { constructor( message: string, diff --git a/packages/stack-supabase/__tests__/helpers/supabase-mock.ts b/packages/stack-supabase/__tests__/helpers/supabase-mock.ts index a72efd619..26d8576a4 100644 --- a/packages/stack-supabase/__tests__/helpers/supabase-mock.ts +++ b/packages/stack-supabase/__tests__/helpers/supabase-mock.ts @@ -36,11 +36,16 @@ export function fakeEnvelope(value: unknown, column: string): FakeEnvelope { : typeof value === 'bigint' ? value.toString() : value + // `String(pt)` throws on a null-prototype object (no toString) — which the + // JSON selector needles deliberately are (prototype-pollution safety). Tag + // the fake ciphertext via JSON for objects instead. + const tag = + pt !== null && typeof pt === 'object' ? JSON.stringify(pt) : String(pt) return { v: 2, i: { t: 'tbl', c: column }, - c: `ct:${String(pt)}`, - hm: `hm:${String(pt)}`, + c: `ct:${tag}`, + hm: `hm:${tag}`, pt, } } diff --git a/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts b/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts index 48391c697..a4d7f7abc 100644 --- a/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts +++ b/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts @@ -1754,17 +1754,18 @@ describe('v3 raw filter() resolves the query type from the operator', () => { }) // `encryptCollectedTerms` rejects any queryType outside the three scalar EQL - // v3 kinds. No public call path can produce a fourth — `mapFilterOpToQueryType`, + // v3 kinds. No public call path can produce a fifth — `mapFilterOpToQueryType`, // `queryTypeForRawOp` and `queryTypeForOrOp` are exhaustive — so this backstop // is unreachable without breaking the internal contract, which is exactly what // the subclass below does. Keep the guard: it is what a future producer - // gaining a fourth QueryTypeName would trip over. - it('rejects a query type outside the scalar EQL v3 kinds', async () => { + // gaining a new QueryTypeName would trip over. (`searchableJson` used to be + // the exemplar here until #650 made it a real, supported kind.) + it('rejects a query type outside the supported EQL v3 kinds', async () => { const supabase = createMockSupabase() class BogusQueryType extends EncryptedQueryBuilderV3Impl { protected override queryTypeForRawOp(_operator: string) { - return 'searchableJson' as never + return 'steVecSelector' as never } } @@ -1781,8 +1782,8 @@ describe('v3 raw filter() resolves the query type from the operator', () => { .filter('email', 'eq', 'a@b.com') expect(status).toBe(500) - expect(error?.message).toContain('query type "searchableJson"') - expect(error?.message).toContain('not supported on scalar EQL v3 columns') + expect(error?.message).toContain('query type "steVecSelector"') + expect(error?.message).toContain('not supported on EQL v3 columns') }) }) diff --git a/packages/stack-supabase/__tests__/supabase-v3-json.test.ts b/packages/stack-supabase/__tests__/supabase-v3-json.test.ts new file mode 100644 index 000000000..3b37c779c --- /dev/null +++ b/packages/stack-supabase/__tests__/supabase-v3-json.test.ts @@ -0,0 +1,225 @@ +/** + * Encrypted-JSON querying on the v3 supabase surface (#650): `contains()` on a + * `types.Json` column (encrypted ste_vec containment), the JSONPath selector + * methods (`selectorEq`/`selectorNe`), and the guards that keep the capability- + * overloaded `cs` wire operator honest (free-text vs containment). + * + * Wire semantics (which overload PostgREST resolves, absent-path `ne` + * inclusion) are proven live in `integration/json.integration.test.ts`; this + * suite pins the ADAPTER's behaviour — operand routing, encryption calls, and + * rejection surfaces — against mocks. + */ + +import { encryptedTable, types } from '@cipherstash/stack/eql/v3' +import { describe, expect, it } from 'vitest' +import { EncryptedQueryBuilderV3Impl } from '../src/query-builder-v3' +import { + createMockEncryptionClient, + createMockSupabase, +} from './helpers/supabase-mock' + +const events = encryptedTable('events', { + payload: types.Json('payload'), + name: types.TextSearch('name'), +}) + +const EVENTS_ALL_COLUMNS = ['id', 'payload', 'name', 'note'] + +function makeBuilder(resultData: unknown = []) { + const supabase = createMockSupabase(resultData) + const encryptionClient = createMockEncryptionClient() + const builder = new EncryptedQueryBuilderV3Impl( + 'events', + events, + encryptionClient as never, + supabase.client as never, + EVENTS_ALL_COLUMNS, + ) + return { supabase, builder } +} + +/** The recorded wire call for a column's containment filter, parsed. */ +function containsCall( + supabase: ReturnType, + method: 'filter' | 'not', +) { + const call = supabase.calls.find((c) => c.method === method) + expect(call).toBeDefined() + return call as { method: string; args: unknown[] } +} + +describe('contains() on an encrypted types.Json column', () => { + it('storage-encrypts the sub-document and emits the cs wire operator', async () => { + const { supabase, builder } = makeBuilder() + await builder.select('id').contains('payload', { user: { role: 'admin' } }) + + const call = containsCall(supabase, 'filter') + expect(call.args[0]).toBe('payload') + expect(call.args[1]).toBe('cs') + // The operand is the JSON.stringify'd storage envelope of the WHOLE + // sub-document — encrypted via encrypt/bulkEncrypt, never encryptQuery. + const envelope = JSON.parse(call.args[2] as string) + expect(envelope.pt).toEqual({ user: { role: 'admin' } }) + expect(envelope.i.c).toBe('payload') + }) + + it('accepts an array sub-document', async () => { + const { supabase, builder } = makeBuilder() + await builder.select('id').contains('payload', [{ tag: 'vip' }]) + const call = containsCall(supabase, 'filter') + expect(JSON.parse(call.args[2] as string).pt).toEqual([{ tag: 'vip' }]) + }) + + it('rejects a scalar operand with a sub-document steer', () => { + const { builder } = makeBuilder() + expect(() => builder.contains('payload', 'admin' as never)).toThrow( + /takes a sub-document/, + ) + expect(() => builder.contains('payload', null as never)).toThrow( + /takes a sub-document/, + ) + }) + + it('still rejects contains() on an encrypted TEXT column (that is matches())', () => { + const { builder } = makeBuilder() + expect(() => builder.contains('name', { any: 'thing' })).toThrow( + /native \(exact\) containment.*matches\(\)/s, + ) + }) + + it('still passes contains() through natively on a plaintext column', async () => { + const { supabase, builder } = makeBuilder() + await builder.select('id').contains('note', ['x']) + // Plaintext containment takes the base path: the operand is formatted as a + // native containment literal (never encrypted — no envelope in the operand). + const call = containsCall(supabase, 'filter') + expect(call.args[0]).toBe('note') + expect(call.args[1]).toBe('cs') + expect(String(call.args[2])).not.toContain('pt') + }) +}) + +describe('matches() vs encrypted JSON', () => { + it('rejects matches() on a types.Json column with a contains/selector steer', () => { + const { builder } = makeBuilder() + expect(() => builder.matches('payload', 'admin')).toThrow( + /does not apply to encrypted JSON column .*contains\(.*selectorEq\(/s, + ) + }) +}) + +describe('selectorEq()', () => { + it('reconstructs the path-shaped needle and emits encrypted cs', async () => { + const { supabase, builder } = makeBuilder() + await builder.select('id').selectorEq('payload', '$.user.role', 'admin') + + const call = containsCall(supabase, 'filter') + expect(call.args[1]).toBe('cs') + expect(JSON.parse(call.args[2] as string).pt).toEqual({ + user: { role: 'admin' }, + }) + }) + + it('accepts a bare dot path (no $ prefix)', async () => { + const { supabase, builder } = makeBuilder() + await builder.select('id').selectorEq('payload', 'user.age', 30) + const call = containsCall(supabase, 'filter') + expect(JSON.parse(call.args[2] as string).pt).toEqual({ user: { age: 30 } }) + }) + + it('rejects invalid paths with the shared validation errors', () => { + const { builder } = makeBuilder() + expect(() => builder.selectorEq('payload', '$', 'x')).toThrow( + /addresses no field/, + ) + expect(() => builder.selectorEq('payload', '$.a[0]', 'x')).toThrow( + /array\/wildcard syntax/, + ) + expect(() => builder.selectorEq('payload', '$.a..b', 'x')).toThrow( + /malformed/, + ) + expect(() => builder.selectorEq('payload', '$.a.__proto__', 'x')).toThrow( + /forbidden key/, + ) + }) + + it('rejects non-scalar leaves and null', () => { + const { builder } = makeBuilder() + expect(() => + builder.selectorEq('payload', '$.user', { role: 'admin' } as never), + ).toThrow(/scalar leaf.*contains\(\)/s) + expect(() => + builder.selectorEq('payload', '$.tags', ['vip'] as never), + ).toThrow(/scalar leaf/) + expect(() => + builder.selectorEq('payload', '$.user.role', null as never), + ).toThrow(/non-null scalar/) + }) + + it('rejects a non-JSON column', () => { + const { builder } = makeBuilder() + expect(() => builder.selectorEq('name', '$.a', 'x')).toThrow( + /requires an encrypted JSON \(types\.Json\) column/, + ) + expect(() => builder.selectorEq('note', '$.a', 'x')).toThrow( + /requires an encrypted JSON \(types\.Json\) column/, + ) + }) +}) + +describe('selectorNe()', () => { + it('emits the negated containment (not + cs) of the reconstructed needle', async () => { + const { supabase, builder } = makeBuilder() + await builder.select('id').selectorNe('payload', '$.user.role', 'admin') + + const call = containsCall(supabase, 'not') + expect(call.args[0]).toBe('payload') + expect(call.args[1]).toBe('cs') + expect(JSON.parse(call.args[2] as string).pt).toEqual({ + user: { role: 'admin' }, + }) + }) + + it('shares selectorEq validation (path + leaf + column kind)', () => { + const { builder } = makeBuilder() + expect(() => builder.selectorNe('payload', '$.a[*]', 'x')).toThrow( + /array\/wildcard/, + ) + expect(() => builder.selectorNe('name', '$.a', 'x')).toThrow( + /requires an encrypted JSON/, + ) + }) +}) + +describe('raw filter routing on a JSON column', () => { + it("accepts .filter(col, 'cs', subdoc) — the raw containment spelling", async () => { + const { supabase, builder } = makeBuilder() + await builder.select('id').filter('payload', 'cs', { user: { age: 30 } }) + const call = containsCall(supabase, 'filter') + expect(call.args[1]).toBe('cs') + expect(JSON.parse(call.args[2] as string).pt).toEqual({ user: { age: 30 } }) + }) + + it("rejects scalar ops (.filter(col, 'eq', …)) by capability", async () => { + const { builder } = makeBuilder() + const { error, status } = await builder + .select('id') + .filter('payload', 'eq', { a: 1 }) + expect(status).toBe(500) + expect(error?.message).toMatch(/does not support equality queries/) + }) + + it('not(col, "contains", …) is allowed on a JSON column (exact negated containment)', async () => { + const { supabase, builder } = makeBuilder() + await builder.select('id').not('payload', 'contains', { a: 1 }) + const call = containsCall(supabase, 'not') + expect(call.args[1]).toBe('cs') + }) + + it('not(col, "contains", …) stays rejected on an encrypted TEXT column', () => { + const { builder } = makeBuilder() + expect(() => builder.not('name', 'contains', 'x')).toThrow( + /not.*fuzzy free-text matching/s, + ) + }) +}) diff --git a/packages/stack-supabase/__tests__/supabase-v3.test-d.ts b/packages/stack-supabase/__tests__/supabase-v3.test-d.ts index 10ab39b2d..782882abb 100644 --- a/packages/stack-supabase/__tests__/supabase-v3.test-d.ts +++ b/packages/stack-supabase/__tests__/supabase-v3.test-d.ts @@ -355,3 +355,62 @@ describe('encryptedSupabaseV3 untyped surface (no schemas)', () => { supabase.from('users').select() }) }) + +// --------------------------------------------------------------------------- +// Encrypted JSON querying (#650): contains on types.Json + selector methods +// --------------------------------------------------------------------------- + +const docs = encryptedTable('docs', { + payload: types.Json('payload'), + email: types.TextEq('email'), +}) + +declare const docsBuilder: EncryptedQueryBuilderV3< + typeof docs, + InferPlaintext & { note: string; meta: Record } +> + +describe('encrypted JSON keys and operands (#650)', () => { + it('contains() accepts a sub-document on the types.Json column', () => { + docsBuilder.contains('payload', { user: { role: 'admin' } }) + docsBuilder.contains('payload', [{ tag: 'vip' }]) + // A raw-string operand is the PLAINTEXT native form only; the encrypted + // JSON overload requires an object/array sub-document. + // @ts-expect-error — string operand is not a sub-document + docsBuilder.contains('payload', '{"user":{}}') + }) + + it('a types.Json column is filterable (no longer erased to never)', () => { + // Pre-#650, QueryTypesForColumn resolved types.Json to never, putting it in + // NonQueryableV3Keys and rejecting EVERY filter mention at compile time. + docsBuilder.selectorEq('payload', '$.user.role', 'admin') + }) + + it('selectorEq/selectorNe accept scalar leaves on the JSON column', () => { + docsBuilder.selectorEq('payload', '$.user.age', 30) + docsBuilder.selectorNe('payload', '$.user.active', true) + docsBuilder.selectorEq('payload', '$.user.joined', new Date()) + // @ts-expect-error — objects are contains(), not a selector leaf + docsBuilder.selectorEq('payload', '$.user', { role: 'admin' }) + // @ts-expect-error — null is not a comparable leaf + docsBuilder.selectorNe('payload', '$.user.role', null) + }) + + it('selector methods reject non-JSON columns at compile time', () => { + // @ts-expect-error — email is a TextEq column, not types.Json + docsBuilder.selectorEq('email', '$.a', 'x') + // @ts-expect-error — note is a plaintext column + docsBuilder.selectorNe('note', '$.a', 'x') + }) + + it('matches() still rejects the JSON column at compile time', () => { + // @ts-expect-error — payload carries searchableJson, not freeTextSearch + docsBuilder.matches('payload', 'admin') + }) + + it('plaintext contains() is unaffected', () => { + docsBuilder.contains('meta', { a: 1 }) + // @ts-expect-error — scalar plaintext column has no containment operand + docsBuilder.contains('note', 'x') + }) +}) diff --git a/packages/stack-supabase/integration/json.integration.test.ts b/packages/stack-supabase/integration/json.integration.test.ts new file mode 100644 index 000000000..68054cbf7 --- /dev/null +++ b/packages/stack-supabase/integration/json.integration.test.ts @@ -0,0 +1,178 @@ +/** + * Live encrypted-JSON querying for the v3 supabase adapter (#650): real crypto, + * real PostgREST (as `anon`), real `public.eql_v3_json` domain. + * + * What this uniquely proves (the unit suite runs against mocks): + * - `contains()` reaches the `eql_v3."@>"(eql_v3_json, eql_v3_json)` overload + * through PostgREST's column-type cast of the `cs.` operand, with a real + * storage-encrypted needle (probe-verified wire form; see #650). + * - `selectorEq`/`selectorNe` — equality-at-a-path compiled to containment of + * the reconstructed needle — return the right ROWS, including the documented + * `ne` semantics: rows whose document lacks the path entirely are INCLUDED. + * - The `anon` grants cover the ste_vec functions the operator expands to. + * + * Selector ORDERING is deliberately absent: not expressible over PostgREST + * until cipherstash/encrypt-query-language#407 lands. Drizzle's + * `json-selector.integration.test.ts` proves ordering semantics. + */ + +import type { JsonDocument } from '@cipherstash/stack/eql/v3' +import { encryptedTable, types } from '@cipherstash/stack/eql/v3' +import { databaseUrl } from '@cipherstash/test-kit' +import postgres from 'postgres' +import { afterAll, beforeAll, describe, expect, it } from 'vitest' +import { encryptedSupabaseV3 } from '../src/index.js' +import { makePostgrestClient, reloadSchemaCache } from './helpers/pgrest' + +const TABLE = 'protect_ci_v3_supabase_json' +const RUN = `run-${Date.now()}-${Math.random().toString(36).slice(2, 8)}` + +const sql = postgres(databaseUrl(), { prepare: false }) + +const docs = encryptedTable(TABLE, { + payload: types.Json('payload'), +}) + +const DOCS: Record = { + ada: { user: { email: 'ada@example.com', role: 'admin' }, age: 30 }, + grace: { user: { email: 'grace@example.com', role: 'admin' }, age: 20 }, + zoe: { user: { email: 'zoe@example.com', role: 'analyst' }, age: 40 }, + // No `$.age` and no `$.user.role` — exercises absent-path semantics + // (excluded by contains/selectorEq, INCLUDED by selectorNe). + norole: { user: { email: 'norole@example.com' } }, +} + +type Instance = Awaited> +let instance: Instance + +function from() { + return (instance as never as Instance) + .from<{ row_key: string; payload: JsonDocument }>(TABLE, docs as never) + .select('row_key') + .eq('test_run_id', RUN) +} + +async function rowKeys( + q: PromiseLike<{ data: unknown; error: { message: string } | null }>, +): Promise { + const { data, error } = await q + if (error) throw new Error(error.message) + return ((data as { row_key: string }[]) ?? []).map((r) => r.row_key).sort() +} + +beforeAll(async () => { + await sql.unsafe(` + CREATE TABLE IF NOT EXISTS public.${TABLE} ( + id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + row_key text NOT NULL, + test_run_id text NOT NULL, + payload public.eql_v3_json + ) + `) + await sql.unsafe( + `GRANT SELECT, INSERT ON public.${TABLE} TO anon, authenticated`, + ) + await reloadSchemaCache(sql, TABLE) + + instance = await encryptedSupabaseV3(makePostgrestClient(), { + schemas: { [TABLE]: docs } as never, + databaseUrl: databaseUrl(), + }) + + const models = Object.entries(DOCS).map(([rowKey, payload]) => ({ + row_key: rowKey, + test_run_id: RUN, + payload, + })) + const { error } = await (instance as never as Instance) + .from(TABLE, docs as never) + .insert(models) + if (error) throw new Error(`seed insert: ${error.message}`) +}, 120_000) + +afterAll(async () => { + await sql.unsafe(`DELETE FROM public.${TABLE} WHERE test_run_id = '${RUN}'`) + await sql.end() +}) + +describe('encrypted containment (contains)', () => { + it('matches rows containing a nested sub-document', async () => { + expect( + await rowKeys(from().contains('payload', { user: { role: 'admin' } })), + ).toEqual(['ada', 'grace']) + }) + + it('a multi-leaf needle requires EVERY leaf to match', async () => { + expect( + await rowKeys( + from().contains('payload', { + user: { role: 'admin' }, + age: 30, + }), + ), + ).toEqual(['ada']) + }) + + it('matches nothing for an absent value', async () => { + expect(await rowKeys(from().contains('payload', { age: 99 }))).toEqual([]) + }) + + it('raw .filter(col, "cs", subdoc) is the same query', async () => { + expect(await rowKeys(from().filter('payload', 'cs', { age: 40 }))).toEqual([ + 'zoe', + ]) + }) +}) + +describe('selector equality (selectorEq)', () => { + it('matches the row carrying the value at the path', async () => { + expect(await rowKeys(from().selectorEq('payload', '$.age', 30))).toEqual([ + 'ada', + ]) + }) + + it('works on nested paths and string leaves', async () => { + expect( + await rowKeys(from().selectorEq('payload', '$.user.role', 'analyst')), + ).toEqual(['zoe']) + }) + + it('an absent path matches nothing', async () => { + expect( + await rowKeys(from().selectorEq('payload', '$.user.missing', 'x')), + ).toEqual([]) + }) +}) + +describe('selector inequality (selectorNe)', () => { + it('includes rows with a different value AND rows lacking the path', async () => { + // Documented semantics (mirrors the Drizzle selector `ne`): NOT-contains, + // and a document without the path never contains — so `norole` is in. + expect( + await rowKeys(from().selectorNe('payload', '$.user.role', 'admin')), + ).toEqual(['norole', 'zoe']) + }) + + it('ne against an absent-everywhere value returns every row', async () => { + expect(await rowKeys(from().selectorNe('payload', '$.age', 99))).toEqual([ + 'ada', + 'grace', + 'norole', + 'zoe', + ]) + }) +}) + +describe('guards hold on the live surface', () => { + it('matches() on the JSON column throws the steer', () => { + expect(() => from().matches('payload', 'admin')).toThrow( + /encrypted JSON column/, + ) + }) + + it('scalar ops on the JSON column are rejected by capability', async () => { + const { error, status } = await from().filter('payload', 'eq', { a: 1 }) + expect(status).toBe(500) + expect(error?.message).toMatch(/does not support equality/) + }) +}) diff --git a/packages/stack-supabase/src/helpers.ts b/packages/stack-supabase/src/helpers.ts index d34697413..7d2ae5fc6 100644 --- a/packages/stack-supabase/src/helpers.ts +++ b/packages/stack-supabase/src/helpers.ts @@ -237,9 +237,11 @@ export function mapFilterOpToQueryType(op: FilterOp): QueryTypeName { return 'equality' case 'like': case 'ilike': - // `matches` is the encrypted free-text (bloom) operator. `contains` remains - // for completeness/v2 but is plaintext-only on the v3 surface, so it never - // reaches term encryption there (a plaintext operand is not encrypted). + // `matches` is the encrypted free-text (bloom) operator. `contains` is + // plaintext-native on scalar columns, but on an encrypted `types.Json` + // column it IS the encrypted ste_vec containment (#650) — the v3 dialect's + // capability resolver re-types the collected term to `searchableJson` when + // the column carries that capability instead of `freeTextSearch`. case 'contains': case 'matches': return 'freeTextSearch' diff --git a/packages/stack-supabase/src/query-builder-v3.ts b/packages/stack-supabase/src/query-builder-v3.ts index 608661ef6..66916d283 100644 --- a/packages/stack-supabase/src/query-builder-v3.ts +++ b/packages/stack-supabase/src/query-builder-v3.ts @@ -2,6 +2,9 @@ import { DATE_LIKE_CASTS, EncryptedV3Column, logger, + parseSelectorSegments, + reconstructSelectorDocument, + unsupportedLeafReason, } from '@cipherstash/stack/adapter-kit' import type { EncryptionClient } from '@cipherstash/stack/encryption' import type { AnyV3Table } from '@cipherstash/stack/eql/v3' @@ -50,6 +53,8 @@ type V3ColumnLike = { equality: boolean orderAndRange: boolean freeTextSearch: boolean + /** Optional: only `public.eql_v3_json` (`types.Json`) carries it. */ + searchableJson?: boolean } build(): ColumnSchema } @@ -402,19 +407,35 @@ export class EncryptedQueryBuilderV3Impl< */ private assertTermQueryable(term: ScalarQueryTerm): V3ColumnLike { const column = term.column as unknown as V3ColumnLike - const queryType = term.queryType ?? 'equality' + let queryType = term.queryType ?? 'equality' + const capabilities = column.getQueryCapabilities() + + // The `cs` wire operator is capability-overloaded: bloom free-text on a + // match/search TEXT column, encrypted ste_vec containment on a `types.Json` + // DOCUMENT column. Both arrive here as `freeTextSearch` (contains/matches/ + // raw `cs` all map there); resolve to the capability the column actually + // carries. The two are mutually exclusive by construction, so this can + // never reinterpret a real free-text column. + if ( + queryType === 'freeTextSearch' && + !capabilities.freeTextSearch && + capabilities.searchableJson + ) { + queryType = 'searchableJson' + } if ( queryType !== 'equality' && queryType !== 'orderAndRange' && - queryType !== 'freeTextSearch' + queryType !== 'freeTextSearch' && + queryType !== 'searchableJson' ) { throw new Error( - `[supabase v3]: query type "${queryType}" is not supported on scalar EQL v3 columns`, + `[supabase v3]: query type "${queryType}" is not supported on EQL v3 columns`, ) } - if (!column.getQueryCapabilities()[queryType]) { + if (!capabilities[queryType]) { throw new Error( `[supabase v3]: column "${column.getName()}" (${column.getEqlType()}) does not support ${queryType} queries — declare the column with a domain that carries that capability`, ) @@ -571,13 +592,31 @@ export class EncryptedQueryBuilderV3Impl< return Boolean(this.v3Columns[column]) } + /** True when `column` is an encrypted `types.Json` document column. */ + private isSearchableJsonColumn(column: string): boolean { + const builder = this.v3Columns[column] as unknown as + | V3ColumnLike + | undefined + return Boolean(builder?.getQueryCapabilities().searchableJson) + } + /** - * `contains` on the v3 surface is EXACT containment only — native jsonb/array - * `@>` on a plaintext column. On an encrypted match/search column containment - * is not the operation (that is the fuzzy `matches`), so refuse loudly rather + * `contains` on the v3 surface is EXACT containment: native jsonb/array `@>` + * on a plaintext column, ENCRYPTED ste_vec `@>` on a `types.Json` column (the + * sub-document operand is storage-encrypted whole; every leaf must match at + * its path — #650). On an encrypted match/search TEXT column containment is + * not the operation (that is the fuzzy `matches`), so refuse loudly rather * than silently emit a bloom match under a name that promises exactness. */ override contains(column: string, value: unknown): this { + if (this.isSearchableJsonColumn(column)) { + if (value === null || typeof value !== 'object') { + throw new Error( + `[supabase v3]: contains("${column}", …) on an encrypted JSON column takes a sub-document (object or array) to match, got ${value === null ? 'null' : typeof value}.`, + ) + } + return super.contains(column, value) + } if (this.isEncryptedV3Column(column)) { throw new Error( `[supabase v3]: contains() is native (exact) containment and does not apply to encrypted column "${column}". Use matches() for encrypted free-text search.`, @@ -590,9 +629,18 @@ export class EncryptedQueryBuilderV3Impl< * `matches` is the encrypted free-text operator: fuzzy bloom-filter token * matching, one-sided (may false-positive), NOT containment. It requires an * encrypted match/search column; on a plaintext column, `contains` (native - * `@>`) is what the caller means. + * `@>`) is what the caller means — and on an encrypted JSON column, + * `contains`/`selectorEq` are (matching a document is containment, not + * free-text). Guarded here because both spellings collect the same + * `freeTextSearch` term, which the capability resolver would otherwise + * silently accept as containment of the raw string. */ override matches(column: string, value: unknown): this { + if (this.isSearchableJsonColumn(column)) { + throw new Error( + `[supabase v3]: matches() is encrypted free-text search and does not apply to encrypted JSON column "${column}". Use contains("${column}", subDocument) or selectorEq("${column}", path, value).`, + ) + } if (!this.isEncryptedV3Column(column)) { throw new Error( `[supabase v3]: matches() is encrypted free-text search and requires an encrypted column; "${column}" is not one. Use contains() for native containment.`, @@ -602,15 +650,23 @@ export class EncryptedQueryBuilderV3Impl< } /** - * `not(col, 'contains', …)` on an encrypted column would negate a fuzzy bloom - * match under the `contains` name — the exact confusion #617 removes — because - * the base `not()` path rewrites the `contains` spelling to the `cs` wire - * operator. Reject it and steer to the `matches` spelling (or the raw `cs` - * operator, which is honest about the wire op). Plaintext columns keep native - * negated containment, and every other operator is delegated unchanged. + * `not(col, 'contains', …)` on an encrypted TEXT column would negate a fuzzy + * bloom match under the `contains` name — the exact confusion #617 removes — + * because the base `not()` path rewrites the `contains` spelling to the `cs` + * wire operator. Reject it and steer to the `matches` spelling (or the raw + * `cs` operator, which is honest about the wire op). + * + * On an encrypted JSON column negated containment IS the honest exact + * operation (`not.cs` over ste_vec containment — {@link selectorNe} compiles + * to it), so it passes through. Plaintext columns keep native negated + * containment, and every other operator is delegated unchanged. */ override not(column: string, operator: string, value: unknown): this { - if (operator === 'contains' && this.isEncryptedV3Column(column)) { + if ( + operator === 'contains' && + this.isEncryptedV3Column(column) && + !this.isSearchableJsonColumn(column) + ) { throw new Error( `[supabase v3]: not("${column}", 'contains', …) does not apply to encrypted column "${column}" — that is fuzzy free-text matching, not containment. Use not("${column}", 'matches', …) or the raw 'cs' operator.`, ) @@ -618,6 +674,75 @@ export class EncryptedQueryBuilderV3Impl< return super.not(column, operator, value) } + /** + * Validate + reconstruct a selector needle: `('$.user.role', 'admin')` → + * `{user: {role: 'admin'}}`. Shared by {@link selectorEq}/{@link selectorNe}; + * throws with column context for a non-JSON column, an invalid path, or a + * non-scalar leaf. + */ + private selectorNeedle( + method: string, + column: string, + path: string, + value: unknown, + ): Record { + if (!this.isSearchableJsonColumn(column)) { + throw new Error( + `[supabase v3]: ${method}() requires an encrypted JSON (types.Json) column; "${column}" is not one.`, + ) + } + if (value == null) { + throw new Error( + `[supabase v3]: ${method}("${column}", "${path}", …) requires a non-null scalar value.`, + ) + } + // Selector comparisons compare a scalar LEAF (eq/ne arm — `ordering: false`; + // PostgREST cannot express selector ordering yet, see + // cipherstash/encrypt-query-language#407). + const leafReason = unsupportedLeafReason(value, false) + if (leafReason) { + throw new Error( + `[supabase v3]: ${method}("${column}", "${path}", …): ${leafReason}`, + ) + } + let segments: string[] + try { + segments = parseSelectorSegments(path) + } catch (err) { + throw new Error( + `[supabase v3]: ${method}("${column}", …): ${err instanceof Error ? err.message : String(err)}`, + ) + } + return reconstructSelectorDocument(segments, value) + } + + /** + * Encrypted JSONPath-selector equality: matches rows whose document carries + * exactly `value` at `path`. Equality at a path IS containment of the + * path-shaped needle (`{user: {role: 'admin'}}`), so this compiles to + * {@link contains} — the ste_vec entry at the selector matches on its + * equality/ordering term. Selector ORDERING (`gt`/`lt`/…) is not expressible + * over PostgREST until the bundle grows a needle-comparison overload + * (cipherstash/encrypt-query-language#407); the Drizzle adapter's + * `ops.selector()` supports it today. + */ + selectorEq(column: string, path: string, value: unknown): this { + const needle = this.selectorNeedle('selectorEq', column, path, value) + return super.contains(column, needle) + } + + /** + * Encrypted JSONPath-selector inequality: rows whose document does NOT carry + * `value` at `path` — INCLUDING rows where the path is absent, mirroring the + * Drizzle selector's documented `ne` semantics (`NOT contains`; a missing + * path never contains). Compiles to `not(column, 'contains', needle)` → + * PostgREST `not.cs`. + */ + selectorNe(column: string, path: string, value: unknown): this { + const needle = this.selectorNeedle('selectorNe', column, path, value) + return super.not(column, 'contains', needle) + } + /** * `like`/`ilike` on an ENCRYPTED column are a best-effort compatibility shim, * delegated to `matches`. EQL v3 free-text search is fuzzy bloom token diff --git a/packages/stack-supabase/src/types.ts b/packages/stack-supabase/src/types.ts index df5fe7865..a2f1f90f0 100644 --- a/packages/stack-supabase/src/types.ts +++ b/packages/stack-supabase/src/types.ts @@ -145,6 +145,54 @@ export type V3EncryptedFreeTextKeys< > & Extract +/** + * JS property names of a v3 table's columns that carry NO `searchableJson` + * capability — i.e. every domain but `public.eql_v3_json`. Mirror of + * {@link NonFreeTextSearchV3Keys} for the encrypted-JSON capability. + */ +type NonSearchableJsonV3Keys = { + [K in Extract< + keyof V3ColumnsOfTable
, + string + >]: 'searchableJson' extends QueryTypesForColumn[K]> + ? never + : K +}[Extract, string>] + +/** + * Row keys the encrypted-JSON query methods accept (`contains()` on an + * encrypted column, `selectorEq()`, `selectorNe()`): ONLY the table's ENCRYPTED + * columns whose domain is `public.eql_v3_json` (`types.Json`). Plaintext + * columns are excluded — on those, `contains()` is PostgREST-native containment + * and the selector methods do not apply. Mirror of + * {@link V3EncryptedFreeTextKeys} for the `searchableJson` capability. + */ +export type V3SearchableJsonKeys< + Table extends AnyV3Table, + Row extends Record, +> = Exclude< + Extract, string>, + NonSearchableJsonV3Keys
+> & + Extract + +/** + * The operand `contains()` accepts on an ENCRYPTED `types.Json` column: a + * sub-document (object or array). The whole operand is storage-encrypted + * against the column and compared via encrypted ste_vec containment — never a + * raw PostgREST string form, which cannot be encrypted. + */ +export type EncryptedJsonContainsValue = + | Record + | readonly unknown[] + +/** + * The scalar leaf value the selector methods compare at a JSONPath. Objects and + * arrays are rejected (that shape is `contains()`); see the adapters' shared + * `unsupportedLeafReason`. + */ +export type SelectorLeafValue = string | number | bigint | boolean | Date + /** * The operand `contains()` accepts on a PLAINTEXT column, mirroring * postgrest-js's own untyped `contains` overload: a jsonb literal, an array, or @@ -310,6 +358,37 @@ export interface EncryptedQueryBuilderV3< column: K, value: PlaintextContainsValue, ): EncryptedQueryBuilderV3 + /** ENCRYPTED (exact) JSON containment on a `types.Json` column: the + * sub-document operand is storage-encrypted against the column and compared + * via the encrypted ste_vec `@>`. Every leaf of the operand must match at its + * path. Note the operand's ciphertext appears in the request's filter string + * (the same tradeoff as every v3 filter operand — see the class doc). */ + contains & StringKeyOf>( + column: K, + value: EncryptedJsonContainsValue, + ): EncryptedQueryBuilderV3 + /** Encrypted JSONPath-selector equality on a `types.Json` column: + * `selectorEq('doc', '$.user.role', 'admin')` matches rows whose document + * carries exactly that value at that path. Compiled to encrypted containment + * of the reconstructed `{user: {role: 'admin'}}` needle — equality at a path + * IS containment of the path-shaped document. Paths are dot-notation object + * keys (`'$.a.b'`); array/wildcard steps are rejected. */ + selectorEq & StringKeyOf>( + column: K, + path: string, + value: SelectorLeafValue, + ): EncryptedQueryBuilderV3 + /** Encrypted JSONPath-selector inequality. Matches rows whose document does + * NOT carry `value` at `path` — INCLUDING rows where the path is absent + * (mirrors the Drizzle selector's documented `ne` semantics; it is `NOT + * contains`, and a missing path never contains). Selector ordering + * (`gt`/`lt`/…) is not yet expressible over PostgREST — see + * cipherstash/encrypt-query-language#407. */ + selectorNe & StringKeyOf>( + column: K, + path: string, + value: SelectorLeafValue, + ): EncryptedQueryBuilderV3 } /** @@ -338,11 +417,26 @@ export interface EncryptedQueryBuilderV3Untyped< column: K, value: string, ): EncryptedQueryBuilderV3Untyped - /** Native jsonb/array containment on a plaintext column (PostgREST `cs`). */ + /** Native jsonb/array containment on a plaintext column (PostgREST `cs`), + * or ENCRYPTED ste_vec containment on an encrypted `types.Json` column — the + * runtime resolves the column kind and picks the encoding. */ contains>( column: K, value: NativeContainsValue, ): EncryptedQueryBuilderV3Untyped + /** Encrypted JSONPath-selector equality on an encrypted `types.Json` column + * (runtime-guarded on the untyped surface). */ + selectorEq>( + column: K, + path: string, + value: SelectorLeafValue, + ): EncryptedQueryBuilderV3Untyped + /** Encrypted JSONPath-selector inequality (includes absent-path rows). */ + selectorNe>( + column: K, + path: string, + value: SelectorLeafValue, + ): EncryptedQueryBuilderV3Untyped } /** Untyped instance (no `schemas`): rows default to `Record` diff --git a/packages/stack/src/adapter-kit.ts b/packages/stack/src/adapter-kit.ts index 8dcf65041..2d1fad591 100644 --- a/packages/stack/src/adapter-kit.ts +++ b/packages/stack/src/adapter-kit.ts @@ -43,6 +43,17 @@ export { stripDomainSchema, } from './eql/v3/domain-registry.js' +// Shared JSONPath-selector path handling (parse/validate, needle-document +// reconstruction, scalar-leaf guard) for encrypted-JSON querying — used by the +// Drizzle selector operators (#651) and the Supabase selector filters (#650) so +// the validation rules cannot drift between adapters. +export { + jsonPathOf, + parseSelectorSegments, + reconstructSelectorDocument, + unsupportedLeafReason, +} from './eql/v3/selector-path.js' + // Shared match-index guard (short-needle rejection), reused by both adapters. export { matchNeedleError } from './schema/match-defaults.js' // Shared structured logger (adapters log rejections through the same instance). diff --git a/packages/stack/src/eql/v3/columns.ts b/packages/stack/src/eql/v3/columns.ts index 6647088ab..bd8895afb 100644 --- a/packages/stack/src/eql/v3/columns.ts +++ b/packages/stack/src/eql/v3/columns.ts @@ -765,7 +765,7 @@ export type PlaintextForColumn = * The user-facing `queryType` names a v3 column supports, derived 1:1 from its * capability flags. Resolves to `never` for a storage-only column (all flags * `false`) and for any non-v3 value. The names mirror the {@link QueryCapabilities} - * keys and the first three {@link import('@/types').QueryTypeName} members. + * keys, each of which is also a {@link import('@/types').QueryTypeName} member. */ export type QueryTypesForColumn = C extends EncryptedV3Column @@ -777,6 +777,13 @@ export type QueryTypesForColumn = | (D['capabilities']['freeTextSearch'] extends true ? 'freeTextSearch' : never) + // The flag is optional (absent means "not a JSON document column"), so + // only a literal `true` contributes. Without this arm a `types.Json` + // column resolved to `never`, and every typed adapter key set derived + // from this type excluded encrypted-JSON columns entirely (#650). + | (D['capabilities']['searchableJson'] extends true + ? 'searchableJson' + : never) : never /** diff --git a/packages/stack/src/eql/v3/selector-path.ts b/packages/stack/src/eql/v3/selector-path.ts new file mode 100644 index 000000000..bff33d25d --- /dev/null +++ b/packages/stack/src/eql/v3/selector-path.ts @@ -0,0 +1,117 @@ +/** + * Shared JSONPath-selector path handling for the first-party adapters + * (`@cipherstash/stack-drizzle`, `@cipherstash/stack-supabase`), exposed via + * `@cipherstash/stack/adapter-kit`. + * + * Both adapters express "compare the value at `$.a.b`" over an encrypted + * `eql_v3_json` column, and both need the same three pieces: parse + validate + * the dot-notation path, reconstruct the `{ a: { b: value } }` needle document + * whose ste_vec entry at the path carries the comparison terms, and reject + * non-scalar leaves up front. Originally private to the Drizzle v3 operators + * (#651); moved here when the Supabase adapter grew the same querying (#650) so + * the validation rules cannot drift between adapters. + */ + +/** + * Object keys that are prototype-pollution vectors — rejected outright (mirrors + * core's `FORBIDDEN_KEYS`), so a selector can never address them. + */ +const FORBIDDEN_SEGMENTS: ReadonlySet = new Set([ + '__proto__', + 'prototype', + 'constructor', +]) + +/** + * Parse a dot-notation JSONPath into its object-key segments. Rejects, each with + * a clear message: array-index/wildcard syntax (v1 is object-keys-only), the + * empty/root path, malformed paths (`..`, stray/leading/trailing dots, so we + * never silently query a *different* path), and prototype-pollution keys. + * `'$.a.b'` / `' a.b '` → `['a', 'b']`. + */ +export function parseSelectorSegments(path: string): string[] { + const trimmed = path.trim() + let body = trimmed.startsWith('$') ? trimmed.slice(1) : trimmed + if (body.startsWith('.')) body = body.slice(1) + if (body === '') { + throw new Error( + `JSON selector path "${path}" addresses no field — use e.g. "$.a" or "$.a.b".`, + ) + } + if (/[[\]*]/.test(body)) { + throw new Error( + `JSON selector path "${path}" uses array/wildcard syntax, which is not yet supported — use dot-notation object keys (e.g. "$.a.b").`, + ) + } + const segments = body.split('.') + for (const segment of segments) { + if (segment === '') { + throw new Error( + `JSON selector path "${path}" is malformed (empty segment / ".." / stray dot) — use dot-notation object keys (e.g. "$.a.b").`, + ) + } + if (FORBIDDEN_SEGMENTS.has(segment)) { + throw new Error( + `JSON selector path "${path}" addresses the forbidden key "${segment}".`, + ) + } + } + return segments +} + +/** `$`-rooted JSONPath for `encryptQuery`'s selector needle. */ +export function jsonPathOf(segments: string[]): string { + return `$.${segments.join('.')}` +} + +/** + * A selector compares a single scalar LEAF. Returns a reason string when `value` + * is unsupported — a non-scalar (object/array → that's `contains`), or a boolean + * under an ordering operator (no ordering term) — else `null`. Callers raise it + * as their adapter's operator error with column context, so a bad value is an + * actionable SDK error rather than a deferred, opaque DB failure. + */ +export function unsupportedLeafReason( + value: unknown, + ordering: boolean, +): string | null { + const isScalar = + value instanceof Date || + typeof value === 'number' || + typeof value === 'bigint' || + typeof value === 'string' || + typeof value === 'boolean' + if (!isScalar) { + return `a selector compares a scalar leaf, but got ${Array.isArray(value) ? 'an array' : 'an object'} — use contains() for sub-object matching.` + } + if (ordering && typeof value === 'boolean') { + return 'a boolean leaf has no ordering — use eq/ne (or contains()).' + } + return null +} + +/** + * Nest `value` under the segments: `['a','b']` → `{ a: { b: value } }`. The + * storage-needle document whose ste_vec entry at the path supplies the + * ciphertext-bearing comparison entry. + */ +export function reconstructSelectorDocument( + segments: string[], + value: unknown, +): Record { + // Null-prototype objects: a segment like `__proto__` must become an OWN key, + // not invoke the prototype setter (which would drop it and mis-serialize the + // needle). JSON.stringify ignores the [[Prototype]], so this serializes fine. + const root: Record = Object.create(null) + let cursor = root + segments.forEach((segment, index) => { + if (index === segments.length - 1) { + cursor[segment] = value + } else { + const next: Record = Object.create(null) + cursor[segment] = next + cursor = next + } + }) + return root +} diff --git a/skills/stash-supabase/SKILL.md b/skills/stash-supabase/SKILL.md index d8046c9ad..51de3537c 100644 --- a/skills/stash-supabase/SKILL.md +++ b/skills/stash-supabase/SKILL.md @@ -527,9 +527,10 @@ All envelopes (stored payloads and filter operands) are versioned `v: 3`. bloom-filter token matching (PostgREST `cs` / SQL `@>`): one-sided (a match may be a false positive, a non-match never is) and order-/multiplicity- insensitive, where `%` is tokenized like any other character. `matches(col, - needle)` is the operator; `contains()` on an encrypted column throws an error - pointing at `matches()`. `contains()` is reserved for native (exact) - jsonb/array containment on plaintext columns, which pass through unchanged. + needle)` is the operator; `contains()` on an encrypted TEXT column throws an + error pointing at `matches()`. `contains()` is native (exact) jsonb/array + containment on plaintext columns — and ENCRYPTED (exact) containment on a + `types.Json` column (see "Encrypted JSON querying" below). - **`like`/`ilike` on an encrypted column are an approximate compatibility shim** delegating to `matches()`: leading/trailing `%` are stripped and the residual term is fuzzy-matched (same `cs` wire, plus a one-time warning). @@ -569,6 +570,44 @@ All envelopes (stored payloads and filter operands) are versioned `v: 3`. - **Null filter values are rejected** with a pointer to `.is(column, null)` — a null cannot be encrypted into an operand. +### Encrypted JSON querying (`types.Json`) + +A `types.Json("payload")` column (`public.eql_v3_json`) stores an encrypted +JSONB document and supports two query forms: + +```typescript +// Exact encrypted containment: every leaf of the sub-document must match at +// its path (ste_vec `@>` — PostgREST `cs`). +es.from("events").select("id").contains("payload", { user: { role: "admin" } }) + +// JSONPath selector equality / inequality at a dot-notation path: +es.from("events").select("id").selectorEq("payload", "$.user.role", "admin") +es.from("events").select("id").selectorNe("payload", "$.user.role", "admin") +``` + +- `selectorEq(col, path, value)` matches rows carrying exactly `value` at + `path` — it compiles to containment of the path-shaped needle + (`{user: {role: "admin"}}`), which is the same operation. Paths are + dot-notation object keys (`"$.a.b"` or `"a.b"`); array/wildcard steps are + rejected, and values must be scalars (an object operand belongs to + `contains()`). +- `selectorNe` matches rows that do NOT carry the value — **including rows + where the path is absent entirely** (it is NOT-contains, and a missing path + never contains). This mirrors the Drizzle selector's `ne` semantics. +- **Selector ordering (`gt`/`gte`/`lt`/`lte`) is not available on Supabase.** + PostgREST cannot reach the entry-comparison operators (it wraps JSON arrow + paths in `to_jsonb`, and bare-column comparison is blocked by design); it + needs an EQL-bundle overload, tracked in + cipherstash/encrypt-query-language#407. The Drizzle integration's + `ops.selector()` supports ordering today — use it where ordering at a path + is required. +- `matches()` does not apply to JSON columns (it is text free-text search) and + throws with a steer; scalar filters (`eq`, `gt`, `in`, …) on the column are + rejected by capability. +- The containment/selector operand is a **full storage envelope** of the + needle document — the same INTERIM shape (and the same security caveat about + ciphertext in GET query strings) as every other v3 filter operand above. + ## Migrating an Existing Column to Encrypted The hard case: a Supabase table that already exists with live data in a plaintext column you want to encrypt. You can't just change the column type — that would drop the data. From b2579533f31eadba2348e2b8ec8bdf05f2717b6e Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Wed, 15 Jul 2026 21:04:29 +1000 Subject: [PATCH 2/4] fix(stack-supabase): address code-review findings on encrypted-JSON querying MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ten findings from the high-effort review of #653, all fixed: 1. Guard bypass on non-method spellings: the freeTextSearch→searchableJson re-typing now enforces the operand boundary AT THE RESOLVER — raw .filter(col,'cs',string), .or() string/structured conditions, and not(col,'contains',scalar) get the actionable sub-document steer instead of silently storage-encrypting a JSON scalar; not(col,'matches',…) on a JSON column gets the same steer matches() gives. 2. Type-surface regression: scalar predicates (eq/gt/in/…) are compile-excluded again for types.Json columns (V3FilterableKeys now excludes columns with no scalar capability); dedicated filter('cs')/not('contains') overloads keep the raw spellings typed. Pinned with @ts-expect-error tests. 3. Empty/non-plain needles: contains('payload', {}) and [] are rejected ("matches every row" — Drizzle parity), and Date/Map/RegExp instances get the sub-document steer instead of serializing to scalars or {}. 4. Array-leaf semantics: the live probe showed finder-claimed ANY-element matching is WRONG with real protect-ffi — a scalar needle does NOT match an array at the path (elements are encoded under their own selectors). Docs corrected to the observed truth and pinned by live fixtures (scalar needle misses the array row; the full array through contains() hits it). 5. SQL-NULL documents: selectorNe now compiles to a structured OR (payload.is.null, payload.not.cs.) so NULL-document rows are INCLUDED, matching Drizzle's ne for both absence cases — verified live. 6. Integration test: removed the phantom two-arg from(TABLE, schema) (v3 from() takes one argument) and the identity casts; tsc over integration/ is clean. 7. Exposure honesty: the skill now states the JSON-needle ciphertext scales per-node (Drizzle containment ships none) and cites the new tracking issue #654; empty-needle rejection documented. 8. Changeset gains the required 'stash': patch for the skills change. 9. stash-encryption skill now names the Supabase route beside Drizzle's for both containment and selectors (with the ordering caveat). 10. Live decrypt round-trip added (select payload → toEqual original doc), plus live not-contains, and the unit not-contains asserts the encrypted envelope. Also from below-the-fold: the INTENTIONAL FORK JSDoc re-attached to EncryptionOperatorError (the re-export had split them); parseSelectorSegments rejects whitespace at segment boundaries (was silently addressing 'user '); unsupportedLeafReason handles null explicitly (was steering null to contains()); the as-unknown-as double-cast replaced with a plain widening; dead V3ContainsValue removed; core's matrix test-d pins QueryTypesForColumn = 'searchableJson'; stale "three scalar kinds" comment fixed. Suites: supabase 476 unit + 31 type + 681 live integration (15 in the JSON suite incl. NULL-doc and array-leaf fixtures), stack 62 type tests, drizzle 368 — all green. Claude-Session: https://claude.ai/code/session_01MxTTPaPP16m6br7Hoab94w --- .changeset/supabase-v3-json-querying.md | 6 + packages/stack-drizzle/src/v3/operators.ts | 8 +- .../__tests__/supabase-v3-builder.test.ts | 2 +- .../__tests__/supabase-v3-json.test.ts | 95 ++++++++++++-- .../__tests__/supabase-v3.test-d.ts | 15 ++- .../integration/json.integration.test.ts | 74 +++++++++-- .../stack-supabase/src/query-builder-v3.ts | 95 +++++++++++--- packages/stack-supabase/src/types.ts | 124 +++++++++++------- .../__tests__/v3-matrix/matrix.test-d.ts | 12 ++ packages/stack/src/eql/v3/selector-path.ts | 14 ++ skills/stash-encryption/SKILL.md | 9 +- skills/stash-supabase/SKILL.md | 21 ++- 12 files changed, 373 insertions(+), 102 deletions(-) diff --git a/.changeset/supabase-v3-json-querying.md b/.changeset/supabase-v3-json-querying.md index eb6fe44fb..93a8e0465 100644 --- a/.changeset/supabase-v3-json-querying.md +++ b/.changeset/supabase-v3-json-querying.md @@ -1,6 +1,7 @@ --- '@cipherstash/stack-supabase': minor '@cipherstash/stack': minor +'stash': patch --- Encrypted-JSON querying on the v3 Supabase surface (#650). A `types.Json` @@ -20,3 +21,8 @@ column no longer resolves to `never`, so typed adapter key sets can include it), and the JSONPath selector-path helpers the Drizzle adapter introduced in #651 moved to `@cipherstash/stack/adapter-kit` so both adapters share one validation surface (`@cipherstash/stack-drizzle` re-exports them unchanged). + +The bundled `stash-supabase` and `stash-encryption` skills are updated to +document the new querying surface (including the array-leaf and SQL-NULL +semantics, and the operand-exposure caveat) — skills ship inside the `stash` +tarball, hence the patch. diff --git a/packages/stack-drizzle/src/v3/operators.ts b/packages/stack-drizzle/src/v3/operators.ts index 35830ee1a..0e0ed3654 100644 --- a/packages/stack-drizzle/src/v3/operators.ts +++ b/packages/stack-drizzle/src/v3/operators.ts @@ -80,6 +80,10 @@ type OperandEncryptionClient = { encrypt(value: never, opts: never): ChainableOperation } +// Path helpers now live in @cipherstash/stack/adapter-kit (shared with the +// Supabase adapter, #650); re-exported so existing imports keep working. +export { parseSelectorSegments, reconstructSelectorDocument } + /** * A dedicated error for v3 operator gating and operand-encryption failures, * carrying the offending column/table/operator for diagnostics. @@ -89,10 +93,6 @@ type OperandEncryptionClient = { * `./eql/v3/drizzle` — two independently-versioned public entry points — so the * duplication is deliberate, not an oversight. */ -// Path helpers now live in @cipherstash/stack/adapter-kit (shared with the -// Supabase adapter, #650); re-exported so existing imports keep working. -export { parseSelectorSegments, reconstructSelectorDocument } - export class EncryptionOperatorError extends Error { constructor( message: string, diff --git a/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts b/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts index a4d7f7abc..b12d2554f 100644 --- a/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts +++ b/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts @@ -1753,7 +1753,7 @@ describe('v3 raw filter() resolves the query type from the operator', () => { ]) }) - // `encryptCollectedTerms` rejects any queryType outside the three scalar EQL + // `encryptCollectedTerms` rejects any queryType outside the four supported EQL // v3 kinds. No public call path can produce a fifth — `mapFilterOpToQueryType`, // `queryTypeForRawOp` and `queryTypeForOrOp` are exhaustive — so this backstop // is unreachable without breaking the internal contract, which is exactly what diff --git a/packages/stack-supabase/__tests__/supabase-v3-json.test.ts b/packages/stack-supabase/__tests__/supabase-v3-json.test.ts index 3b37c779c..d9bdbc709 100644 --- a/packages/stack-supabase/__tests__/supabase-v3-json.test.ts +++ b/packages/stack-supabase/__tests__/supabase-v3-json.test.ts @@ -80,6 +80,22 @@ describe('contains() on an encrypted types.Json column', () => { ) }) + it('rejects empty needles — {} and [] match every row', () => { + const { builder } = makeBuilder() + expect(() => builder.contains('payload', {})).toThrow(/matches every row/) + expect(() => builder.contains('payload', [])).toThrow(/matches every row/) + }) + + it('rejects non-plain objects (Date/Map) that would serialize to scalars or {}', () => { + const { builder } = makeBuilder() + expect(() => builder.contains('payload', new Date() as never)).toThrow( + /takes a sub-document/, + ) + expect(() => + builder.contains('payload', new Map([['a', 1]]) as never), + ).toThrow(/takes a sub-document/) + }) + it('still rejects contains() on an encrypted TEXT column (that is matches())', () => { const { builder } = makeBuilder() expect(() => builder.contains('name', { any: 'thing' })).toThrow( @@ -153,7 +169,7 @@ describe('selectorEq()', () => { ).toThrow(/scalar leaf/) expect(() => builder.selectorEq('payload', '$.user.role', null as never), - ).toThrow(/non-null scalar/) + ).toThrow(/non-null scalar leaf.*is\(column, null\)/s) }) it('rejects a non-JSON column', () => { @@ -168,16 +184,22 @@ describe('selectorEq()', () => { }) describe('selectorNe()', () => { - it('emits the negated containment (not + cs) of the reconstructed needle', async () => { + it('emits the IS-NULL-inclusive or: payload.is.null, payload.not.cs.', async () => { const { supabase, builder } = makeBuilder() await builder.select('id').selectorNe('payload', '$.user.role', 'admin') - const call = containsCall(supabase, 'not') - expect(call.args[0]).toBe('payload') - expect(call.args[1]).toBe('cs') - expect(JSON.parse(call.args[2] as string).pt).toEqual({ - user: { role: 'admin' }, - }) + // NULL-document parity with Drizzle's ne: a bare not.cs would drop rows + // whose payload column is SQL NULL (three-valued logic), so selectorNe + // compiles to a structured OR whose containment arm is encrypted. + const call = supabase.calls.find((c) => c.method === 'or') + expect(call).toBeDefined() + const orString = String(call?.args[0]) + expect(orString).toContain('payload.is.null') + expect(orString).toContain('payload.not.cs.') + // The needle in the or-string is the ENCRYPTED envelope (quote-escaped by + // the or-value formatter), not a plaintext containment literal. + expect(orString).toContain('\\"pt\\"') + expect(orString).toContain('admin') }) it('shares selectorEq validation (path + leaf + column kind)', () => { @@ -191,6 +213,53 @@ describe('selectorNe()', () => { }) }) +describe('the resolver operand boundary (non-method spellings)', () => { + it("raw .filter(col, 'cs', string) throws the sub-document steer, not a silent scalar query", async () => { + // Pre-#650 this failed on capability; the searchableJson resolution must + // not turn it into a silently-empty containment of a JSON string scalar. + const { builder } = makeBuilder() + const { error, status } = await builder + .select('id') + .filter('payload', 'cs', '{"role":"admin"}') + expect(status).toBe(500) + expect(error?.message).toMatch(/takes a sub-document/) + }) + + it("not(col, 'matches', …) on a JSON column throws the steer", () => { + const { builder } = makeBuilder() + expect(() => builder.not('payload', 'matches', 'admin')).toThrow( + /does not apply to encrypted JSON column/, + ) + }) + + it('an .or() string condition with a raw cs term is rejected loudly', async () => { + const { builder } = makeBuilder() + const { error, status } = await builder.select('id').or('payload.cs.admin') + expect(status).toBe(500) + expect(error?.message).toMatch(/takes a sub-document/) + }) + + it('a structured .or() contains condition with a sub-document is encrypted', async () => { + const { supabase, builder } = makeBuilder() + await builder + .select('id') + .or([{ column: 'payload', op: 'contains', value: { a: 1 } }]) + const call = supabase.calls.find((c) => c.method === 'or') + const orString = String(call?.args[0]) + expect(orString).toContain('payload.cs.') + expect(orString).toContain('\\"pt\\"') + }) + + it('a structured .or() contains condition with an empty needle is rejected', async () => { + const { builder } = makeBuilder() + const { error, status } = await builder + .select('id') + .or([{ column: 'payload', op: 'contains', value: {} }]) + expect(status).toBe(500) + expect(error?.message).toMatch(/matches every row/) + }) +}) + describe('raw filter routing on a JSON column', () => { it("accepts .filter(col, 'cs', subdoc) — the raw containment spelling", async () => { const { supabase, builder } = makeBuilder() @@ -214,6 +283,16 @@ describe('raw filter routing on a JSON column', () => { await builder.select('id').not('payload', 'contains', { a: 1 }) const call = containsCall(supabase, 'not') expect(call.args[1]).toBe('cs') + // The operand must be the ENCRYPTED envelope — a plaintext containment + // literal here would mean the term skipped encryption entirely. + expect(JSON.parse(call.args[2] as string).pt).toEqual({ a: 1 }) + }) + + it('rejects a whitespace-at-boundary selector path', () => { + const { builder } = makeBuilder() + expect(() => + builder.selectorEq('payload', '$.user .role', 'admin'), + ).toThrow(/whitespace at a segment boundary/) }) it('not(col, "contains", …) stays rejected on an encrypted TEXT column', () => { diff --git a/packages/stack-supabase/__tests__/supabase-v3.test-d.ts b/packages/stack-supabase/__tests__/supabase-v3.test-d.ts index 782882abb..b9140c5dd 100644 --- a/packages/stack-supabase/__tests__/supabase-v3.test-d.ts +++ b/packages/stack-supabase/__tests__/supabase-v3.test-d.ts @@ -380,10 +380,23 @@ describe('encrypted JSON keys and operands (#650)', () => { docsBuilder.contains('payload', '{"user":{}}') }) - it('a types.Json column is filterable (no longer erased to never)', () => { + it('a types.Json column is queryable via its OWN methods, not scalar predicates', () => { // Pre-#650, QueryTypesForColumn resolved types.Json to never, putting it in // NonQueryableV3Keys and rejecting EVERY filter mention at compile time. + // Post-#650 the JSON methods are open… docsBuilder.selectorEq('payload', '$.user.role', 'admin') + docsBuilder.filter('payload', 'cs', { user: { role: 'admin' } }) + docsBuilder.not('payload', 'contains', { user: { role: 'admin' } }) + // …but the scalar predicates stay compile-excluded: an encrypted document + // has no scalar terms, so eq/gt/in against it can only fail at runtime. + // @ts-expect-error — eq on a JSON column has no term to compare + docsBuilder.eq('payload', { a: 1 }) + // @ts-expect-error — gt on a JSON column has no ordering term + docsBuilder.gt('payload', { a: 1 }) + // @ts-expect-error — in-lists have no equality terms on a JSON column + docsBuilder.in('payload', [{ a: 1 }]) + // @ts-expect-error — the raw filter overload for JSON keys admits only 'cs' + docsBuilder.filter('payload', 'eq', { a: 1 }) }) it('selectorEq/selectorNe accept scalar leaves on the JSON column', () => { diff --git a/packages/stack-supabase/integration/json.integration.test.ts b/packages/stack-supabase/integration/json.integration.test.ts index 68054cbf7..2a86015a5 100644 --- a/packages/stack-supabase/integration/json.integration.test.ts +++ b/packages/stack-supabase/integration/json.integration.test.ts @@ -40,16 +40,25 @@ const DOCS: Record = { // No `$.age` and no `$.user.role` — exercises absent-path semantics // (excluded by contains/selectorEq, INCLUDED by selectorNe). norole: { user: { email: 'norole@example.com' } }, + // ARRAY-VALUED path: pins the OBSERVED ste_vec semantics — a scalar-leaf + // needle does NOT match an array at the path (protect-ffi encodes array + // elements under their own selectors, not the parent path's), so selectorEq + // treats an array leaf as not-equal and selectorNe includes it. The docs + // carry this caveat; this fixture keeps it honest against the real FFI+EQL. + multi: { user: { email: 'multi@example.com', role: ['admin', 'analyst'] } }, } +/** A row whose payload COLUMN is SQL NULL — the maximally-absent document. + * selectorNe must include it (the `is.null` arm of its or-compilation). */ +const NULL_ROW_KEY = 'nulldoc' + type Instance = Awaited> let instance: Instance +type Row = { row_key: string; payload: JsonDocument; test_run_id: string } + function from() { - return (instance as never as Instance) - .from<{ row_key: string; payload: JsonDocument }>(TABLE, docs as never) - .select('row_key') - .eq('test_run_id', RUN) + return instance.from(TABLE).select('row_key').eq('test_run_id', RUN) } async function rowKeys( @@ -84,10 +93,14 @@ beforeAll(async () => { test_run_id: RUN, payload, })) - const { error } = await (instance as never as Instance) - .from(TABLE, docs as never) - .insert(models) + const { error } = await instance.from(TABLE).insert(models) if (error) throw new Error(`seed insert: ${error.message}`) + + // NULL-payload row: inserted directly (the adapter's insert path encrypts + // model values; a SQL NULL column needs no encryption). + await sql.unsafe( + `INSERT INTO public.${TABLE} (row_key, test_run_id, payload) VALUES ('${NULL_ROW_KEY}', '${RUN}', NULL)`, + ) }, 120_000) afterAll(async () => { @@ -95,6 +108,19 @@ afterAll(async () => { await sql.end() }) +describe('decrypt round-trip (the read path)', () => { + it('a filtered row decrypts back to the original document', async () => { + const { data, error } = await instance + .from(TABLE) + .select('row_key, payload') + .eq('test_run_id', RUN) + .selectorEq('payload', '$.user.role', 'analyst') + if (error) throw new Error(error.message) + expect(data).toHaveLength(1) + expect(data?.[0].payload).toEqual(DOCS.zoe) + }) +}) + describe('encrypted containment (contains)', () => { it('matches rows containing a nested sub-document', async () => { expect( @@ -102,6 +128,12 @@ describe('encrypted containment (contains)', () => { ).toEqual(['ada', 'grace']) }) + it('not(col, "contains", …) is the bare negated containment (excludes the NULL row)', async () => { + expect( + await rowKeys(from().not('payload', 'contains', { age: 30 })), + ).toEqual(['grace', 'multi', 'norole', 'zoe']) + }) + it('a multi-leaf needle requires EVERY leaf to match', async () => { expect( await rowKeys( @@ -137,6 +169,22 @@ describe('selector equality (selectorEq)', () => { ).toEqual(['zoe']) }) + it('ARRAY-LEAF: a scalar needle does NOT match an array at the path', async () => { + // multi.user.role = ['admin','analyst'] is NOT matched by the scalar + // needle 'admin' — array elements are encoded under their own selectors. + expect( + await rowKeys(from().selectorEq('payload', '$.user.role', 'admin')), + ).toEqual(['ada', 'grace']) + }) + + it('ARRAY-LEAF: the full array as a contains() needle matches the row', async () => { + expect( + await rowKeys( + from().contains('payload', { user: { role: ['admin', 'analyst'] } }), + ), + ).toEqual(['multi']) + }) + it('an absent path matches nothing', async () => { expect( await rowKeys(from().selectorEq('payload', '$.user.missing', 'x')), @@ -145,19 +193,23 @@ describe('selector equality (selectorEq)', () => { }) describe('selector inequality (selectorNe)', () => { - it('includes rows with a different value AND rows lacking the path', async () => { - // Documented semantics (mirrors the Drizzle selector `ne`): NOT-contains, - // and a document without the path never contains — so `norole` is in. + it('includes different-value rows, absent-path rows, AND SQL-NULL documents', async () => { + // Drizzle-parity semantics: NOT-contains OR IS NULL. `norole` lacks the + // path; `nulldoc` lacks the whole document (the case a bare not.cs drops + // under three-valued logic); `multi` is included because its role ARRAY is + // not-equal to the scalar needle (the array-leaf caveat). expect( await rowKeys(from().selectorNe('payload', '$.user.role', 'admin')), - ).toEqual(['norole', 'zoe']) + ).toEqual(['multi', 'norole', NULL_ROW_KEY, 'zoe']) }) it('ne against an absent-everywhere value returns every row', async () => { expect(await rowKeys(from().selectorNe('payload', '$.age', 99))).toEqual([ 'ada', 'grace', + 'multi', 'norole', + NULL_ROW_KEY, 'zoe', ]) }) diff --git a/packages/stack-supabase/src/query-builder-v3.ts b/packages/stack-supabase/src/query-builder-v3.ts index 66916d283..4163a8909 100644 --- a/packages/stack-supabase/src/query-builder-v3.ts +++ b/packages/stack-supabase/src/query-builder-v3.ts @@ -59,6 +59,42 @@ type V3ColumnLike = { build(): ColumnSchema } +/** + * Validate an encrypted-JSON containment operand: a NON-EMPTY plain object or a + * non-empty array. Everything else is rejected with an actionable steer: + * + * - Scalars/strings: the caller meant free-text (`matches` on a text column) or + * a selector — a raw JSON string is NOT parsed, by design (parsing would make + * `'{"a":1}'` and `{a:1}` silently different queries on other surfaces). + * - Non-plain objects (`Date`, `Map`, `RegExp`, class instances): these JSON- + * serialize to scalars or `{}` — not the sub-document the caller believes. + * - `{}` and `[]`: jsonb containment holds for EVERY document (`doc @> '{}'`), + * so an accidentally-empty needle would silently return (and decrypt) the + * whole table. The Drizzle adapter rejects the same needle for the same + * reason — the two first-party adapters must agree that this is an error. + */ +function assertJsonContainmentOperand(column: string, value: unknown): void { + const isPlainObject = + value !== null && + typeof value === 'object' && + !Array.isArray(value) && + (Object.getPrototypeOf(value) === Object.prototype || + Object.getPrototypeOf(value) === null) + if (!isPlainObject && !Array.isArray(value)) { + throw new Error( + `[supabase v3]: encrypted JSON containment on column "${column}" takes a sub-document (plain object or array) to match, got ${value === null ? 'null' : Array.isArray(value) ? 'an array' : typeof value === 'object' ? (value as object).constructor?.name || 'a non-plain object' : typeof value}.`, + ) + } + const empty = Array.isArray(value) + ? value.length === 0 + : Object.keys(value as object).length === 0 + if (empty) { + throw new Error( + `[supabase v3]: encrypted JSON containment on column "${column}" cannot take an empty ${Array.isArray(value) ? 'array' : 'object'} needle: it matches every row. Pass a non-empty sub-document, or omit the predicate to select all rows.`, + ) + } +} + /** * Reject a declared property name that is also a DIFFERENT physical column. * @@ -422,6 +458,15 @@ export class EncryptedQueryBuilderV3Impl< capabilities.searchableJson ) { queryType = 'searchableJson' + // THE single enforced operand boundary for encrypted-JSON containment. + // Terms reach this resolver from every spelling — contains(), raw + // .filter(col,'cs',…), not(col,'contains'|'matches',…), and .or() + // string/structured conditions — and only contains() has a method-level + // guard. Without this check a raw string (e.g. a free-text term ported + // from a text column, or an .or() condition value, which is always a + // string) would be storage-encrypted as a JSON SCALAR and silently match + // nothing; pre-#650 every such spelling failed loudly on capability. + assertJsonContainmentOperand(column.getName(), term.value) } if ( @@ -594,9 +639,7 @@ export class EncryptedQueryBuilderV3Impl< /** True when `column` is an encrypted `types.Json` document column. */ private isSearchableJsonColumn(column: string): boolean { - const builder = this.v3Columns[column] as unknown as - | V3ColumnLike - | undefined + const builder: V3ColumnLike | undefined = this.v3Columns[column] return Boolean(builder?.getQueryCapabilities().searchableJson) } @@ -610,11 +653,9 @@ export class EncryptedQueryBuilderV3Impl< */ override contains(column: string, value: unknown): this { if (this.isSearchableJsonColumn(column)) { - if (value === null || typeof value !== 'object') { - throw new Error( - `[supabase v3]: contains("${column}", …) on an encrypted JSON column takes a sub-document (object or array) to match, got ${value === null ? 'null' : typeof value}.`, - ) - } + // Same validator the term resolver enforces — failing here just surfaces + // the error at the call site instead of at execution. + assertJsonContainmentOperand(column, value) return super.contains(column, value) } if (this.isEncryptedV3Column(column)) { @@ -671,6 +712,14 @@ export class EncryptedQueryBuilderV3Impl< `[supabase v3]: not("${column}", 'contains', …) does not apply to encrypted column "${column}" — that is fuzzy free-text matching, not containment. Use not("${column}", 'matches', …) or the raw 'cs' operator.`, ) } + // Mirror of the matches() guard: a `matches` spelling on a JSON column + // would otherwise resolve to containment (the two share the `cs` wire op), + // silently negating an EXACT operation under a name that promises FUZZY. + if (operator === 'matches' && this.isSearchableJsonColumn(column)) { + throw new Error( + `[supabase v3]: not("${column}", 'matches', …) does not apply to encrypted JSON column "${column}" — matches() is free-text search. Use not("${column}", 'contains', subDocument) or selectorNe("${column}", path, value).`, + ) + } return super.not(column, operator, value) } @@ -691,12 +740,8 @@ export class EncryptedQueryBuilderV3Impl< `[supabase v3]: ${method}() requires an encrypted JSON (types.Json) column; "${column}" is not one.`, ) } - if (value == null) { - throw new Error( - `[supabase v3]: ${method}("${column}", "${path}", …) requires a non-null scalar value.`, - ) - } - // Selector comparisons compare a scalar LEAF (eq/ne arm — `ordering: false`; + // Selector comparisons compare a scalar LEAF (null included in the shared + // helper's rejection; eq/ne arm — `ordering: false`; // PostgREST cannot express selector ordering yet, see // cipherstash/encrypt-query-language#407). const leafReason = unsupportedLeafReason(value, false) @@ -733,14 +778,20 @@ export class EncryptedQueryBuilderV3Impl< /** * Encrypted JSONPath-selector inequality: rows whose document does NOT carry - * `value` at `path` — INCLUDING rows where the path is absent, mirroring the - * Drizzle selector's documented `ne` semantics (`NOT contains`; a missing - * path never contains). Compiles to `not(column, 'contains', needle)` → - * PostgREST `not.cs`. + * `value` at `path` — INCLUDING rows where the path is absent AND rows whose + * document column is SQL NULL, matching the Drizzle selector's `ne` (whose + * `OR entry IS NULL` arm covers both absence cases). A bare `not.cs` would + * drop NULL documents under three-valued logic (`NOT (NULL @> x)` is NULL), + * so this compiles to a structured OR: + * `column.is.null, column.not.cs.` — the containment condition's + * operand is encrypted through the normal or-condition term path. */ selectorNe(column: string, path: string, value: unknown): this { const needle = this.selectorNeedle('selectorNe', column, path, value) - return super.not(column, 'contains', needle) + return super.or([ + { column, op: 'is', value: null }, + { column, op: 'contains', negate: true, value: needle }, + ]) } /** @@ -818,6 +869,12 @@ export class EncryptedQueryBuilderV3Impl< */ protected override queryTypeForOrOp(op: FilterOp): QueryTypeName { if (op === 'matches') return 'freeTextSearch' + // Structured conditions may carry the `contains` METHOD spelling (the wire + // token becomes `cs` in rebuildOrString). It maps to the same capability + // gate as `cs`; on a JSON column the term resolver then re-types it to + // searchableJson and validates the operand. selectorNe's IS-NULL-inclusive + // or-form relies on this arm. + if (op === 'contains') return 'freeTextSearch' return this.queryTypeForRawOp(op) } diff --git a/packages/stack-supabase/src/types.ts b/packages/stack-supabase/src/types.ts index a2f1f90f0..60f77c4ea 100644 --- a/packages/stack-supabase/src/types.ts +++ b/packages/stack-supabase/src/types.ts @@ -91,14 +91,43 @@ export type NonQueryableV3Keys
= { }[Extract, string>] /** - * Row keys a v3 builder accepts in filter methods: every row key except the - * table's storage-only encrypted columns. Plaintext (non-schema) columns pass - * through untouched, exactly as in v2. + * The capability names a SCALAR predicate (`eq`/`neq`/`gt`/`gte`/`lt`/`lte`/ + * `in`) can exercise. `searchableJson` is deliberately absent: an encrypted + * JSON document has no scalar terms, so a scalar predicate against it can only + * ever fail at runtime. + */ +type ScalarQueryTypeName = 'equality' | 'orderAndRange' | 'freeTextSearch' + +/** + * JS property names of a v3 table's columns that support NO scalar predicate: + * storage-only columns (no capability at all) AND `types.Json` columns (whose + * only capability is `searchableJson` — containment/selector querying, reached + * via `contains()`/`selectorEq()`/`selectorNe()` or the dedicated + * `filter(col, 'cs', …)` / `not(col, 'contains', …)` overloads, never via a + * scalar predicate). + */ +type NonScalarQueryableV3Keys
= { + [K in Extract, string>]: [ + Extract< + QueryTypesForColumn[K]>, + ScalarQueryTypeName + >, + ] extends [never] + ? K + : never +}[Extract, string>] + +/** + * Row keys a v3 builder accepts in SCALAR filter methods: every row key except + * the table's encrypted columns with no scalar capability (storage-only + * columns, and `types.Json` documents — see {@link NonScalarQueryableV3Keys}; + * before #650's `searchableJson` arm the two sets coincided). Plaintext + * (non-schema) columns pass through untouched, exactly as in v2. */ export type V3FilterableKeys< Table extends AnyV3Table, Row extends Record, -> = Exclude, NonQueryableV3Keys
> +> = Exclude, NonScalarQueryableV3Keys
> /** * JS property names of a v3 table's columns that carry NO `freeTextSearch` @@ -220,46 +249,6 @@ type PlaintextContainsValue = V extends readonly unknown[] ? V | string : never -/** - * The `contains()` operand for column `K`. - * - * A DECLARED encrypted column reaching `contains` necessarily carries a - * `freeTextSearch` capability — only `public.eql_v3_text_match` and - * `public.eql_v3_text_search` do, and both `cast_as` to `string` — so its operand is - * the string to tokenize into a bloom-filter query term. - * - * Any other key is a plaintext passthrough, where `contains` means PostgREST's - * native jsonb/array containment and the operand follows the column - * ({@link PlaintextContainsValue}). A blanket `value: string` made that half of - * {@link V3FreeTextSearchableKeys} unreachable from TypeScript; a blanket - * {@link NativeContainsValue} then over-corrected, admitting an array on a - * plaintext scalar. - * - * A UNION key is only as permissive as its strictest member: if ANY member is a - * declared column, the operand is the string term. `[K] extends [declared]` gets - * this backwards — a mixed `'email' | 'tags'` fails that test and falls to the - * plaintext branch, so an array typechecks for a key that may resolve to the - * encrypted `email` at runtime. Nothing downstream catches it: the operand goes - * straight to `encrypt()`, which has no plaintext-type guard. - * - * So test the INTERSECTION instead, wrapped in a tuple to stop the naked type - * parameter distributing (which would rebuild the same permissive union). - * - * `PlaintextContainsValue` DOES distribute over `Row[K]`, which is safe only - * because the tuple guard above has already excluded every encrypted member: a - * union of plaintext keys (`'tags' | 'meta'`) must accept the operands of each. - * The residual imprecision is a plaintext scalar unioned with a container column - * (`'note' | 'tags'`), where an array still typechecks — and yields a loud 42883 - * rather than a silent mis-encryption. - */ -export type V3ContainsValue< - Table extends AnyV3Table, - Row extends Record, - K extends string, -> = [Extract, string>>] extends [never] - ? PlaintextContainsValue - : string - /** * JS property names of a v3 table's columns that `order()` cannot sort by. Two * cases: @@ -369,19 +358,26 @@ export interface EncryptedQueryBuilderV3< ): EncryptedQueryBuilderV3 /** Encrypted JSONPath-selector equality on a `types.Json` column: * `selectorEq('doc', '$.user.role', 'admin')` matches rows whose document - * carries exactly that value at that path. Compiled to encrypted containment - * of the reconstructed `{user: {role: 'admin'}}` needle — equality at a path - * IS containment of the path-shaped document. Paths are dot-notation object - * keys (`'$.a.b'`); array/wildcard steps are rejected. */ + * carries that value at that path, compiled to encrypted containment of the + * reconstructed `{user: {role: 'admin'}}` needle. ARRAY-LEAF CAVEAT (pinned + * by the live integration suite): an ARRAY at the path does NOT match a + * scalar needle — ste_vec encodes array elements under their own selectors — + * so `{a:[40,30]}` is NOT matched by `selectorEq('$.a', 30)`; to match an + * array-valued path, pass the full array through `contains()`. Paths are + * dot-notation object keys (`'$.a.b'`); array/wildcard steps are + * rejected. */ selectorEq & StringKeyOf>( column: K, path: string, value: SelectorLeafValue, ): EncryptedQueryBuilderV3 /** Encrypted JSONPath-selector inequality. Matches rows whose document does - * NOT carry `value` at `path` — INCLUDING rows where the path is absent - * (mirrors the Drizzle selector's documented `ne` semantics; it is `NOT - * contains`, and a missing path never contains). Selector ordering + * NOT carry `value` at `path` — INCLUDING rows where the path is absent and + * rows whose document column is SQL NULL (compiled to + * `payload IS NULL OR NOT contains`, matching the Drizzle selector's `ne` + * semantics for both absence cases). The array-leaf caveat on + * {@link selectorEq} carries over: an ARRAY at `path` is never equal to a + * scalar needle, so such rows are INCLUDED here. Selector ordering * (`gt`/`lt`/…) is not yet expressible over PostgREST — see * cipherstash/encrypt-query-language#407. */ selectorNe & StringKeyOf>( @@ -389,6 +385,32 @@ export interface EncryptedQueryBuilderV3< path: string, value: SelectorLeafValue, ): EncryptedQueryBuilderV3 + /** Raw containment spelling on an encrypted `types.Json` column — the ONLY + * `filter()` form a JSON column supports (scalar operators have no terms to + * compare; they are compile-excluded via {@link V3FilterableKeys}). */ + filter & StringKeyOf>( + column: K, + operator: 'cs', + value: EncryptedJsonContainsValue, + ): EncryptedQueryBuilderV3 + filter & StringKeyOf>( + column: K, + operator: string, + value: Row[K], + ): EncryptedQueryBuilderV3 + /** Negated exact containment on an encrypted `types.Json` column (PostgREST + * `not.cs`). Note the bare form EXCLUDES rows whose document column is SQL + * NULL (three-valued logic) — {@link selectorNe} adds the IS NULL arm. */ + not & StringKeyOf>( + column: K, + operator: 'contains', + value: EncryptedJsonContainsValue, + ): EncryptedQueryBuilderV3 + not & StringKeyOf>( + column: K, + operator: string, + value: Row[K], + ): EncryptedQueryBuilderV3 } /** diff --git a/packages/stack/__tests__/v3-matrix/matrix.test-d.ts b/packages/stack/__tests__/v3-matrix/matrix.test-d.ts index bbd23891e..750abcdcb 100644 --- a/packages/stack/__tests__/v3-matrix/matrix.test-d.ts +++ b/packages/stack/__tests__/v3-matrix/matrix.test-d.ts @@ -14,6 +14,7 @@ import { describe, expectTypeOf, it } from 'vitest' import { encryptedTable, type InferPlaintext, + type JsonDocument, type QueryableColumnsOf, type QueryTypesForColumn, } from '@/eql/v3' @@ -29,6 +30,9 @@ const records = encryptedTable('records', { createdAt: V3_MATRIX['public.eql_v3_timestamp_ord'].builder('created_at'), // date email: V3_MATRIX['public.eql_v3_text_search'].builder('email'), // string, full-text active: V3_MATRIX['public.eql_v3_boolean'].builder('active'), // boolean, storage-only + // Pinned so core's OWN suite guards the `searchableJson` arm rather than + // leaving it to a downstream adapter's typecheck (#650). + payload: V3_MATRIX['public.eql_v3_json'].builder('payload'), // JsonDocument, containment/selector only }) describe('eql_v3 type-driven matrix (types)', () => { @@ -41,6 +45,7 @@ describe('eql_v3 type-driven matrix (types)', () => { createdAt: Date email: string active: boolean + payload: JsonDocument }>() }) @@ -66,6 +71,13 @@ describe('eql_v3 type-driven matrix (types)', () => { expectTypeOf< QueryTypesForColumn >().toEqualTypeOf() + // The #650 arm: exactly 'searchableJson' — never the scalar kinds (a + // regression to `never` re-erases JSON columns from every typed adapter + // key set; leaking a scalar kind would open scalar predicates that cannot + // succeed at runtime). + expectTypeOf< + QueryTypesForColumn + >().toEqualTypeOf<'searchableJson'>() }) it('excludes storage-only columns from the queryable set', () => { diff --git a/packages/stack/src/eql/v3/selector-path.ts b/packages/stack/src/eql/v3/selector-path.ts index bff33d25d..8e17843ae 100644 --- a/packages/stack/src/eql/v3/selector-path.ts +++ b/packages/stack/src/eql/v3/selector-path.ts @@ -50,6 +50,15 @@ export function parseSelectorSegments(path: string): string[] { `JSON selector path "${path}" is malformed (empty segment / ".." / stray dot) — use dot-notation object keys (e.g. "$.a.b").`, ) } + // Leading/trailing whitespace in a segment is almost certainly spacing + // around a dot ('$.user .role'), which would silently address a DIFFERENT + // key ('user '). Interior whitespace stays legal — 'a b' is a valid JSON + // key a caller may genuinely have. + if (segment !== segment.trim()) { + throw new Error( + `JSON selector path "${path}" has whitespace at a segment boundary ("${segment}") — it would address a different key. Remove the spacing around the dot.`, + ) + } if (FORBIDDEN_SEGMENTS.has(segment)) { throw new Error( `JSON selector path "${path}" addresses the forbidden key "${segment}".`, @@ -75,6 +84,11 @@ export function unsupportedLeafReason( value: unknown, ordering: boolean, ): string | null { + // Explicit null arm: `typeof null === 'object'`, so without it a null leaf + // would get the actively-wrong "got an object — use contains()" steer. + if (value == null) { + return 'a selector compares a non-null scalar leaf, but got null/undefined — SQL NULL never equals anything; use is(column, null) for null checks.' + } const isScalar = value instanceof Date || typeof value === 'number' || diff --git a/skills/stash-encryption/SKILL.md b/skills/stash-encryption/SKILL.md index 2e9273889..85e68b3a3 100644 --- a/skills/stash-encryption/SKILL.md +++ b/skills/stash-encryption/SKILL.md @@ -685,13 +685,16 @@ const events = encryptedTable("events", { metadata: types.Json("metadata") }) await client.encryptQuery({ roles: ["admin"] }, { column: events.metadata, table: events }) ``` -Through the Drizzle v3 integration this is `ops.contains(col, subObject)` — see +Through the Drizzle v3 integration this is `ops.contains(col, subObject)` (on Supabase: `contains(col, subObject)` — see the `stash-supabase` skill) — see the `stash-drizzle` skill. **JSONPath selector-with-constraint** is the second query pattern: it compares the encrypted value at a JSONPath, e.g. `metadata->'age' > 21`. Through the -Drizzle v3 integration this is `ops.selector(col, '$.path').{eq,ne,gt,gte,lt,lte}(value)`. -Its unique power over containment is **ordering at a path** +Drizzle v3 integration this is `ops.selector(col, '$.path').{eq,ne,gt,gte,lt,lte}(value)`; +on Supabase it is `selectorEq(col, path, value)` / `selectorNe(col, path, value)` +— equality/inequality only, since selector ORDERING over PostgREST needs an +EQL-bundle change (cipherstash/encrypt-query-language#407); see the +`stash-supabase` skill. Drizzle's unique power over containment is **ordering at a path** (`ops.selector(events.metadata, '$.age').gt(21)`); equality at a path is also expressible as containment (`contains(col, { age: 21 })`). v1 supports dot-notation object paths (`$.a`, `$.a.b`); array-index/wildcard and empty/root diff --git a/skills/stash-supabase/SKILL.md b/skills/stash-supabase/SKILL.md index 51de3537c..36db369f2 100644 --- a/skills/stash-supabase/SKILL.md +++ b/skills/stash-supabase/SKILL.md @@ -592,8 +592,14 @@ es.from("events").select("id").selectorNe("payload", "$.user.role", "admin") rejected, and values must be scalars (an object operand belongs to `contains()`). - `selectorNe` matches rows that do NOT carry the value — **including rows - where the path is absent entirely** (it is NOT-contains, and a missing path - never contains). This mirrors the Drizzle selector's `ne` semantics. + where the path is absent entirely, and rows whose document column is SQL + NULL** (it compiles to `payload.is.null OR payload.not.cs.`, matching + the Drizzle selector's `ne` semantics for both absence cases). +- **Array-valued paths:** a scalar needle does NOT match an array at the path — + ste_vec encodes array elements under their own selectors — so + `selectorEq("payload", "$.roles", "admin")` does not match + `{roles: ["admin", "analyst"]}` (and `selectorNe` includes that row). To + match an array-valued path, pass the full array through `contains()`. - **Selector ordering (`gt`/`gte`/`lt`/`lte`) is not available on Supabase.** PostgREST cannot reach the entry-comparison operators (it wraps JSON arrow paths in `to_jsonb`, and bare-column comparison is blocked by design); it @@ -605,8 +611,15 @@ es.from("events").select("id").selectorNe("payload", "$.user.role", "admin") throws with a steer; scalar filters (`eq`, `gt`, `in`, …) on the column are rejected by capability. - The containment/selector operand is a **full storage envelope** of the - needle document — the same INTERIM shape (and the same security caveat about - ciphertext in GET query strings) as every other v3 filter operand above. + needle document. The GET-query-string security caveat above applies with an + important difference in degree: a JSON needle carries the root decryptable + ciphertext PLUS one ciphertext-bearing entry **per node of the sub-document** + — the exposure scales with needle size (the equivalent Drizzle containment + ships no ciphertext at all). Keep needles minimal; removing the ciphertext on + this surface is tracked in cipherstash/stack#654. +- Empty needles (`{}` / `[]`) are rejected — jsonb containment holds for every + document, so an accidentally-empty filter would silently return the whole + table (the Drizzle adapter rejects the same needle). ## Migrating an Existing Column to Encrypted From 1d5addf443008742b18a14b4d661e6e0e8a9f015 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Wed, 15 Jul 2026 23:20:43 +1000 Subject: [PATCH 3/4] fix(stack-supabase): address Copilot feedback on the JSON querying surface MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - SelectorLeafValue narrowed to the JSON scalar kinds (string|number|boolean): a stored JsonDocument cannot contain Date/bigint, so such a needle could never match a stored leaf. The runtime guard rejects them too, with a serialization steer (pass date.toISOString() / the stored form) — stricter than the shared unsupportedLeafReason, whose Date/bigint arms serve the Drizzle surface (its operand is unknown-typed). Pinned by @ts-expect-error type tests and unit tests. - Integration Row type declares payload: JsonDocument | null — the suite seeds a SQL-NULL document row, so the type must not promise non-null. - Removed the unreachable 'an array' branch from the containment operand error message (Array.isArray is false on that path by construction). Claude-Session: https://claude.ai/code/session_01MxTTPaPP16m6br7Hoab94w --- .../__tests__/supabase-v3-json.test.ts | 10 ++++++++ .../__tests__/supabase-v3.test-d.ts | 6 ++++- .../integration/json.integration.test.ts | 8 ++++++- .../stack-supabase/src/query-builder-v3.ts | 23 ++++++++++++++++++- packages/stack-supabase/src/types.ts | 11 +++++---- 5 files changed, 51 insertions(+), 7 deletions(-) diff --git a/packages/stack-supabase/__tests__/supabase-v3-json.test.ts b/packages/stack-supabase/__tests__/supabase-v3-json.test.ts index d9bdbc709..d0122ada0 100644 --- a/packages/stack-supabase/__tests__/supabase-v3-json.test.ts +++ b/packages/stack-supabase/__tests__/supabase-v3-json.test.ts @@ -172,6 +172,16 @@ describe('selectorEq()', () => { ).toThrow(/non-null scalar leaf.*is\(column, null\)/s) }) + it('rejects Date/bigint leaves with the serialization steer (JSON scalars only)', () => { + const { builder } = makeBuilder() + expect(() => + builder.selectorEq('payload', '$.user.joined', new Date() as never), + ).toThrow(/got a Date — pass date\.toISOString\(\)/) + expect(() => + builder.selectorEq('payload', '$.user.balance', 10n as never), + ).toThrow(/JSON scalar.*got bigint/s) + }) + it('rejects a non-JSON column', () => { const { builder } = makeBuilder() expect(() => builder.selectorEq('name', '$.a', 'x')).toThrow( diff --git a/packages/stack-supabase/__tests__/supabase-v3.test-d.ts b/packages/stack-supabase/__tests__/supabase-v3.test-d.ts index b9140c5dd..51239b448 100644 --- a/packages/stack-supabase/__tests__/supabase-v3.test-d.ts +++ b/packages/stack-supabase/__tests__/supabase-v3.test-d.ts @@ -399,10 +399,14 @@ describe('encrypted JSON keys and operands (#650)', () => { docsBuilder.filter('payload', 'eq', { a: 1 }) }) - it('selectorEq/selectorNe accept scalar leaves on the JSON column', () => { + it('selectorEq/selectorNe accept exactly the JSON scalar leaves', () => { docsBuilder.selectorEq('payload', '$.user.age', 30) docsBuilder.selectorNe('payload', '$.user.active', true) + docsBuilder.selectorEq('payload', '$.user.role', 'admin') + // @ts-expect-error — a JsonDocument cannot contain a Date; pass toISOString() docsBuilder.selectorEq('payload', '$.user.joined', new Date()) + // @ts-expect-error — a JsonDocument cannot contain a bigint + docsBuilder.selectorEq('payload', '$.user.balance', 10n) // @ts-expect-error — objects are contains(), not a selector leaf docsBuilder.selectorEq('payload', '$.user', { role: 'admin' }) // @ts-expect-error — null is not a comparable leaf diff --git a/packages/stack-supabase/integration/json.integration.test.ts b/packages/stack-supabase/integration/json.integration.test.ts index 2a86015a5..1c502b488 100644 --- a/packages/stack-supabase/integration/json.integration.test.ts +++ b/packages/stack-supabase/integration/json.integration.test.ts @@ -55,7 +55,13 @@ const NULL_ROW_KEY = 'nulldoc' type Instance = Awaited> let instance: Instance -type Row = { row_key: string; payload: JsonDocument; test_run_id: string } +// `payload` is nullable: the suite seeds a SQL-NULL document row (the +// selectorNe is.null arm), so the type must not promise non-null. +type Row = { + row_key: string + payload: JsonDocument | null + test_run_id: string +} function from() { return instance.from(TABLE).select('row_key').eq('test_run_id', RUN) diff --git a/packages/stack-supabase/src/query-builder-v3.ts b/packages/stack-supabase/src/query-builder-v3.ts index 4163a8909..781ae50b4 100644 --- a/packages/stack-supabase/src/query-builder-v3.ts +++ b/packages/stack-supabase/src/query-builder-v3.ts @@ -81,8 +81,16 @@ function assertJsonContainmentOperand(column: string, value: unknown): void { (Object.getPrototypeOf(value) === Object.prototype || Object.getPrototypeOf(value) === null) if (!isPlainObject && !Array.isArray(value)) { + // Array.isArray is false on this branch by construction, so the label only + // distinguishes null / non-plain object / scalar. + const got = + value === null + ? 'null' + : typeof value === 'object' + ? (value as object).constructor?.name || 'a non-plain object' + : typeof value throw new Error( - `[supabase v3]: encrypted JSON containment on column "${column}" takes a sub-document (plain object or array) to match, got ${value === null ? 'null' : Array.isArray(value) ? 'an array' : typeof value === 'object' ? (value as object).constructor?.name || 'a non-plain object' : typeof value}.`, + `[supabase v3]: encrypted JSON containment on column "${column}" takes a sub-document (plain object or array) to match, got ${got}.`, ) } const empty = Array.isArray(value) @@ -750,6 +758,19 @@ export class EncryptedQueryBuilderV3Impl< `[supabase v3]: ${method}("${column}", "${path}", …): ${leafReason}`, ) } + // Stricter than the shared helper (whose Date/bigint arms serve the Drizzle + // surface): a stored JsonDocument leaf is a JSON scalar, so a Date/bigint + // needle could never match one — reject with the serialization steer + // instead of running a query that structurally returns nothing. + if ( + typeof value !== 'string' && + typeof value !== 'number' && + typeof value !== 'boolean' + ) { + throw new Error( + `[supabase v3]: ${method}("${column}", "${path}", …): a JSON document leaf is a JSON scalar (string/number/boolean); got ${value instanceof Date ? 'a Date — pass date.toISOString() (or the stored form)' : typeof value}.`, + ) + } let segments: string[] try { segments = parseSelectorSegments(path) diff --git a/packages/stack-supabase/src/types.ts b/packages/stack-supabase/src/types.ts index 60f77c4ea..c29165cd8 100644 --- a/packages/stack-supabase/src/types.ts +++ b/packages/stack-supabase/src/types.ts @@ -216,11 +216,14 @@ export type EncryptedJsonContainsValue = | readonly unknown[] /** - * The scalar leaf value the selector methods compare at a JSONPath. Objects and - * arrays are rejected (that shape is `contains()`); see the adapters' shared - * `unsupportedLeafReason`. + * The scalar leaf value the selector methods compare at a JSONPath: exactly the + * JSON scalar kinds a stored {@link import('@cipherstash/stack/eql/v3').JsonValue} + * can carry. `Date` and `bigint` are deliberately absent — a `JsonDocument` + * cannot contain them, so such a needle could never match a stored leaf + * (serialize to the stored form first, e.g. `date.toISOString()`). Objects and + * arrays are rejected too (that shape is `contains()`). */ -export type SelectorLeafValue = string | number | bigint | boolean | Date +export type SelectorLeafValue = string | number | boolean /** * The operand `contains()` accepts on a PLAINTEXT column, mirroring From 109bad36c65d0818b19935287ec5ca1e3a644654 Mon Sep 17 00:00:00 2001 From: Dan Draper Date: Wed, 15 Jul 2026 23:48:05 +1000 Subject: [PATCH 4/4] docs(skills): un-stale the v3 ORDER BY bullet in stash-supabase MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The bullet still said order() is rejected on every encrypted v3 column — the supabase-v3-order-by-ope-term change made that false: OPE-backed ordering columns sort via order=col->op (ORE and term-less columns stay rejected). Caught while auditing ordering coverage; the skill ships to customers, so the stale claim was steering users away from a capability they have. Claude-Session: https://claude.ai/code/session_01MxTTPaPP16m6br7Hoab94w --- skills/stash-supabase/SKILL.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/skills/stash-supabase/SKILL.md b/skills/stash-supabase/SKILL.md index 36db369f2..ae3c33e98 100644 --- a/skills/stash-supabase/SKILL.md +++ b/skills/stash-supabase/SKILL.md @@ -557,12 +557,18 @@ All envelopes (stored payloads and filter operands) are versioned `v: 3`. in GET query strings — so these envelopes can land in URL logs, intermediate proxies, and Supabase request logs. The remaining gap is PostgREST operand casting; an adapter-side fix is tracked. -- **No `ORDER BY` on encrypted v3 columns** — including the range-capable - ones. PostgREST cannot emit `ORDER BY eql_v3.ord_term(col)`, and a bare - `ORDER BY` would silently sort the raw ciphertext envelope, so the builder - rejects `order()` on any encrypted column with a clear error. Range - *filtering* (`gte`/`lte`/…) works. Order by a plaintext column, or sort - application-side after decrypting. +- **`order()` works on OPE-backed encrypted ordering columns** (every plain + `*_ord` domain, plus `text_ord` and `text_search`). PostgREST cannot emit + `ORDER BY eql_v3.ord_term(col)`, and a bare `ORDER BY` would silently sort + the raw ciphertext envelope — so the builder instead emits `order=col->op`, + sorting by the OPE term inside the envelope, which reproduces plaintext + order (the term is fixed-width lowercase hex, so string comparison agrees + with the bytea btree; pinned by `ope-term.integration.test.ts`). ORE-flavour + columns (`*_ord_ore`) are rejected at compile time and runtime — their `ob` + term needs the superuser-only operator class no jsonb path can reach — and + columns with no ordering term (storage-only, equality-only, match-only) + reject `order()` with a clear error. For those, order by a plaintext column + or sort application-side after decrypting. - **Storage-only domains are not filterable** (e.g. `types.Boolean`, `types.Text`): a filter (including `.match()`) on one is a type error on a declared table, and always a clear runtime error. `.is(column, null)`