fix(content-drive): align keyword search with Content Search index query #36688

[ihoffmann-dot]

Problem

Content Drive's toolbar keyword/title search returned inconsistent, slow, and irrelevant results (issue #36688). Tested with client data, Content Drive diverged badly from the Search portlet: results took a long time and often had nothing to do with the query.

Root cause

Content Drive built its own Elasticsearch text query (BrowserAPIImpl.buildBaseESQuery) that diverged from the Content Search portlet's query:

• Broad leading wildcard catchall:*kw* (matches the term anywhere inside any term, across body text → unrelated matches; also a slow, non-indexed scan) instead of Content Search's selective prefix +catchall:kw*.
• Untokenized term and a mixed +/OR boolean group.

Content Search (LuceneQueryBuilder + GlobalSearchAttributeStrategy) does not have these problems.

An earlier revision of this PR routed the keyword to the DB. Per team discussion that was rejected: ADR-0018 keeps text search on the index, and an unindexed ILIKE over contentlet_as_json on this endpoint is a DB-performance risk. This PR now keeps text on the index and instead makes Content Drive consistent with Content Search.

Fix

buildBaseESQuery now reuses GlobalSearchAttributeStrategy (the same strategy the Content Search portlet uses) to build the free-text clause — a selective +catchall:kw* prefix plus tokenized, escaped title boosts. Content Drive keyword search is now consistent with Content Search.

• No routing change (text stays on the index, ADR-0018 compliant).
• No DB schema, no ES mapping, no reindex → rollback-safe.
• /v1/drive/search is @Hidden, so no openapi.yaml change.

Behavior (validated by ContentDriveKeywordSearchTest)

Seeding IMG_1004.jpeg and searching:

Term	Finds the file
IMG, img, Img, IMG_1004, jpeg	✅ yes — a single, relevant result
1004, 1004.jpeg, IMG_1004.jpeg	❌ no

The reported case (IMG) now returns exactly the file (previously: unrelated results + the file missing).

Note (consistency, not a regression introduced here): matching is prefix-based, so mid-token terms (1004), boundary-spanning terms (1004.jpeg), and even the exact full filename (IMG_1004.jpeg — the . splits it into img_1004 + jpeg tokens) do not match. This is identical to the Content Search portlet using the same strategy. Worth a QA confirmation (cc perf/relevance testing).

Out of scope / follow-ups

• Read-your-writes / index lag (a just-uploaded item not yet in ES): inherent to index-based text search (ADR-0018), long-standing; not addressed here.
• Relevance ordering: Content Drive still re-sorts the page by sortBy; full score-ordering parity with Content Search is a separate change to the hybrid flow.
• Performance of the hybrid DB-candidate + permissions path: separate, pre-existing concern.

Closes #36688

🤖 Generated with Claude Code

[claude]

Claude finished @ihoffmann-dot's task in 6m 58s —— View job

---

PR Review — Content Drive keyword search #36688

Note: This PR pivoted since @fabrizzio-dotCMS's review. Commit 964e7a28 reverted the DB-routing approach (removed the resolveTextInDb flag, restored BrowserQuery/ContentDriveHelper to their pre-PR state) and instead reuses GlobalSearchAttributeStrategy inside the existing ES query. The net effective diff is now just 2 files: an 18-line change to BrowserAPIImpl.buildBaseESQuery and the new test. Most of the prior review's blocking findings (performance regression, precision/contentlet_as_json::text noise, unescaped LIKE wildcards) no longer apply — they were about DB code that's been removed. My review reflects the current state.

New Issues

• 🟠 High: dotcms-integration/.../ContentDriveKeywordSearchTest.java:55 — Test is still not registered in any suite, so it never runs in CI. This was prior-review Blocking update with latest SVN #1 and remains unfixed: the diff doesn't touch MainSuite3a.java, whose siblings (ContentDriveFieldFilterTest, ContentDriveWorkflowFilterTest, etc.) are registered at MainSuite3a.java:73-76. A *Test in dotcms-integration only runs via @SuiteClasses, so the regression guard for [BUG] Content Drive: keyword/title search returns inconsistent or incoherent results #36688 is inert. One-line fix: add ContentDriveKeywordSearchTest.class to MainSuite3a (+ import). Fix this →

• 🟡 Medium: dotcms-integration/.../ContentDriveKeywordSearchTest.java:156 — The originally-failing term is no longer asserted. The issue's screencast failure was searching 1004; those terms (1004, 1004.jpeg, IMG_1004.jpeg) are now in the characterize list — logged, never asserted (lines 158-162). Since matching is prefix-based (+catchall:kw*) a mid-token term like 1004 will not match IMG_1004.jpeg, which is presumably why they were downgraded to characterization. That's a legitimate behavior choice, but it means the test asserts only prefix cases that already work and does not guard the exact input from the report. Worth confirming with the issue reporter that "search 1004 finds IMG_1004" is not an expected behavior before merging — otherwise the fix doesn't close the report.

• 🟡 Medium: dotcms-integration/.../ContentDriveKeywordSearchTest.java:89 — Read-your-writes is no longer covered. The PR's stated original purpose was the just-uploaded-file-not-visible (index lag) scenario. The current fix stays on Elasticsearch, and every fixture is seeded with IndexPolicy.WAIT_FOR (lines 89, 106), so the test can never observe index lag. The prior version had a de-index-then-still-find test that genuinely proved the DB path; it's been removed. If index lag was the real root cause of the screencast, this change (query-shape only) does not address it and nothing tests it. Please confirm the root cause is query precision (this PR's premise) and not lag.

• 🟡 Medium: dotCMS/.../BrowserAPIImpl.java:1189 — Assumption: the reused GlobalSearchAttributeStrategy emits +catchall:IMG* and title:IMG* without lowercasing the term (GlobalSearchAttributeStrategy.java:27,38), and query_string prefix/wildcard terms are not analyzed by default. The catchall/title fields use my_analyzer (lowercasing) at index time (es-content-mapping.json:104). What to verify: that uppercase IMG/Img actually match at ES runtime — i.e. that the keywordSearch_uppercaseIMG_findsFile assertion passes against a real index, not just that the query string is well-formed. This is exactly the case-sensitivity symptom from [BUG] Content Drive: keyword/title search returns inconsistent or incoherent results #36688 and from the prior review's ES-root-cause hypothesis (New Issue #4). Because the test isn't wired into CI (finding above), this has not been demonstrated green in this PR's build. Note: this mirrors the Search portlet's existing behavior, so if it's wrong it's a pre-existing ES bug — but this PR makes Content Drive depend on it.

• 🟡 Medium: PR description — stale. It describes the reverted DB approach (resolveTextInDb, ILIKE over JSON/tags, "route to the database", the removed read-your-writes test). The current code does the opposite (keeps text on ES). Please rewrite the description and the class javadoc at ContentDriveKeywordSearchTest.java:42-51 so reviewers/rollback-analysis aren't misled.

Resolved

• ✅ BrowserAPIImpl.java — prior perf regression (full-site ILIKE scan per keystroke) removed; DB text-predicate path reverted.
• ✅ BrowserAPIImpl.java — prior contentlet_as_json::text precision noise removed.
• ✅ BrowserAPIImpl.java:2085 — prior unescaped-LIKE-wildcard concern no longer reachable via this path.
• ✅ ContentDriveHelper.java:68 — javadoc "Uses Elasticsearch for text filtering" is accurate again after the pivot back to ES.
• ✅ BrowserQuery.java — resolveTextInDb flag removed; two-flags-for-one-decision concern gone.

What's good

• The pivot correctly reuses the shared GlobalSearchAttributeStrategy, so Content Drive and the Search portlet now produce identical text queries — a good consistency win and the right layer for the concern.
• Blast radius stays contained: only buildBaseESQuery's text group changed; fileName/field-filter/DB paths are untouched.
• The composition test asserts exclusion when the field filter doesn't match (:209), not just inclusion. 👍

Recommendation

Fix the suite registration (🟠 High) regardless of direction — the test is the whole regression guard and currently doesn't run. Then confirm the two open questions: (1) is uppercase IMG actually matching at ES runtime, and (2) was the root cause query-precision (fixed here) or index lag (not addressed here)? Update the stale description either way.

[claude]

Claude finished @ihoffmann-dot's task in 7m 16s —— View job

---

Rollback-Safety Analysis

• Read docs/core/ROLLBACK_UNSAFE_CATEGORIES.md
• Diff BrowserAPIImpl.java, BrowserQuery.java, ContentDriveHelper.java, and the new integration test
• Check against every C/H/M category
• Verdict posted

Verdict: ✅ Safe to Rollback — no category matched.

Why:

• No runonce migration task, no DDL (ALTER/DROP/RENAME), no PK change — rules out C-1, C-3, C-4, H-1, H-2, H-3, H-7.
• No putMapping() call, no ESMappingAPIImpl/ESMappingUtilHelper/ESMappingConstants touch, no reindex trigger — rules out C-2.
• No new ContentType field type, no storage-provider change, no DROP PROCEDURE/FUNCTION — rules out H-4, H-5, H-6.
• No ViewTool implementation or object returned to VTL templates was touched — rules out H-8.
• /v1/drive/search is @Hidden per the PR description (confirmed no openapi.yaml diff in the changeset) and the JSON response shape is untouched — only the internal data-source routing for the free-text term changed. Rules out M-3.
• No push-publishing bundle XML or OSGi-exported interface touched — rules out M-2, M-4.

What actually changed: a new BrowserQuery.resolveTextInDb flag (default false, so /v1/browser legacy path is byte-for-byte unchanged), a routing branch in BrowserAPIImpl.selectQuery/buildBaseESQuery/isUseElasticSearchForFiltering, and ContentDriveHelper setting the new flag instead of forcing useElasticsearchFiltering. All in-memory query-building logic — no persisted data shape, index mapping, or wire contract changes.

On rollback to N-1: the toolbar keyword search simply reverts to routing through Elasticsearch again (re-introducing the original read-your-writes bug this PR fixes) — a functional regression, not data loss, startup failure, or a broken contract for a surviving consumer. N-1 boots normally and reads/writes exactly as before.

Label AI: Safe To Rollback has been applied.

[fabrizzio-dotCMS]

Code review

Thanks for the detailed write-up and for shipping a regression test with the fix. I agree the read-your-writes gap is real, but I don't think free-text search should be resolved in the database: I measured the query this PR introduces against two production-scale datasets and the cost is significant. I also don't think the "incoherent results" half of #36688 is fixed by this change.

---

🔴 Blocking

1. ContentDriveKeywordSearchTest is not registered in any suite, so it never runs in CI

It lives in dotcms-integration with a *Test name, so it only runs through @SuiteClasses. Its four siblings are registered (MainSuite3a.java:73-76: ContentDriveFieldFilterTest, ContentDriveHelperContentletAPIComparisonTest, ContentDriveWorkflowArchiveStepTest, ContentDriveWorkflowFilterTest), and the diff doesn't touch MainSuite3a. The "Integration Tests - MainSuite 3a" job on this PR passed in 27 minutes without executing a single new assertion — so the regression guard for #36688 is currently inert. One-line fix: add it to MainSuite3a.

2. Measured performance regression on the keyword search path

I rebuilt the exact SQL this PR produces — selectQuery base with skipFolder(true) (Content Drive at site root, so no parent_path predicate), the appendFilterQuery predicate (BrowserAPIImpl.java:2084-2104), order by c.mod_date desc LIMIT 900 — and ran EXPLAIN (ANALYZE, BUFFERS) on two anonymized production-scale datasets:

Dataset	items in site scope	contentlet table	pre-fix (no text predicate in SQL)	post-fix, common term	post-fix, no-match term
A	~391,000	1.5 GB	0.49 – 2.0 s	4.7 s	8.2 s
B	~324,000 (+129k SYSTEM_HOST)	9.2 GB	0.37 s	—	37.6 s

Warm cache, local SSD, a single connection, no concurrency; EXPLAIN ANALYZE doesn't transfer rows, so these numbers are, if anything, optimistic.

Two aggravating factors that aren't visible in the diff:

• order by c.mod_date desc forces the predicate to be evaluated over the whole candidate set before the LIMIT 900 can cut. There is no early exit for rare terms — and in a toolbar the user types progressive prefixes, so most keystrokes land in exactly the rare/no-match case (the 8–37 s column).
• The existing safety valve stops bounding the work. BROWSER_DB_MAX_SCAN_ROWS = 50_000 (BrowserAPIImpl.java:725) caps rows returned per chunk, not rows examined. Previously the ILIKE was not in the SQL, so the DB scan stopped at 50k rows and ES narrowed each chunk. Now the predicate is inside the SQL, so Postgres sweeps the entire site (390k+ rows) evaluating contentlet_as_json::text, which forces a detoast + lz4 decompression of every row's JSON (postgres.sql:779). No trigram/GIN index exists that could serve ILIKE '%x%', and postgres.sql:2389 only has an expression index on the template field.

This also reverses a deliberate decision: the ES text path was introduced for performance in #33416 / 6a024b8 ("significant performance improvements for text-based content searches"). Reversing it is fine if warranted, but it needs a measurement, and the measurement doesn't support it.

3. The "results unrelated to the typed text" half of #36688 is not fixed — the noise source just moves

appendFilterQuery matches contentlet_as_json::text, i.e. the whole serialized JSON document, not field values. That document contains (see com.dotcms.content.model.Contentlet:41-49, FieldValue's @JsonTypeInfo(property = "type"), and the ->'fields'->...->>'value' shape used in FolderIntegrityChecker:182):

• identifier, inode, contentType ids → any hex substring (1004, e5f) matches unrelated content
• modDate as ISO-8601 (ContentletJsonHelper:42 sets WRITE_DATES_AS_TIMESTAMPS=false) → searching 2026 returns everything modified this year
• JSON keys and type discriminators: "fields", "value", "type":"Binary", "Image", "Text", "TextArea" → searching image returns every content that has an Image field; searching value or type returns everything in scope

None of that was ever in the ES catchall, so for these terms precision gets worse, not better. The new test only asserts presence of the target and never exclusion of irrelevant items, which is why it passes. If the DB path stays in any form, the predicate should be scoped to field values (contentlet_as_json->'fields', e.g. jsonb_path_query_array(..., '$.fields.*.value')) and the test should include a negative assertion for a term like 2026 or Text.

---

Suggested alternatives — close the lag on the write side, not on every read

The lag is real and it's the default: INDEX_POLICY_SINGLE_CONTENT is DEFER (IndexPolicyProvider.java:35), so a save returns before the index refreshes. But that can be closed where it originates:

1. IndexPolicy.WAIT_FOR on Content Drive's own write operations (upload, create, publish, rename, move). The write waits one refresh (≤ refresh_interval, typically 1 s) and every subsequent search sees the item. That's ~1 s once per write instead of 4.7–37 s per search. Worth noting: the new test seeds its fixtures with setPolicy(IndexPolicy.WAIT_FOR) — the primitive that solves this is already being used, in the right place.
2. Optimistic insert on the front end — zero backend cost. The client just created the item and holds its payload; it can prepend it to the list and reconcile on the next refresh. This is what drive-style UIs do, and it fully covers the "I just uploaded IMG_1004.jpeg and can't see it" report.
3. Recency-bounded overlay, if a server-side guarantee is wanted: keep ES for the search and union a second DB query restricted to recent writes (and c.mod_date > now() - interval '5 minutes' plus the same scope). Same predicate as this PR, but bounded so the ILIKE evaluates over tens of rows instead of ~400k.
4. Fix the ES query, which is most likely the actual root cause. buildBaseESQuery emits +(title:X* OR title:'X'^15 OR title_dotraw:*X*^5 OR +catchall:*X*^10). The + on the last clause makes catchall mandatory in query_string, and wildcard/prefix terms don't go through the analyzer — so an uppercase term like IMG cannot match the lowercased index terms (→ "finds nothing"), while lowercase img matches any body containing img (→ "unrelated results"). Those are precisely the two symptoms reported in #36688. Lowercasing the term, dropping the inner +, and targeting title / title_dotraw / metadata.name instead of catchall likely fixes the report without moving anything to the database. If the ES text branch is kept for external callers, this bug stays latent either way and should be fixed.

---

🟠 Medium

4. Unescaped LIKE wildcards (BrowserAPIImpl.java:2085, 2102-2104) — pre-existing, but this PR promotes the path to Content Drive's primary keyword search. Searching 50% becomes ILIKE '%50%%' (matches "50" followed by anything); a_b matches "axb". Cheap fix: escape %, _, \ and add ESCAPE '\'.

5. The PR description overstates the DB predicate. Only the JSON clause is tokenized. asset_name, the binary asset name and the tag sub-select all use the full filter string, so the generated SQL is (json~tok1 AND json~tok2) OR asset_name~"full string" OR binName~"full string" OR tag~"full string". IMG 1004 works only because the file name is embedded in the JSON. Worth correcting in the description and the new javadoc.

🔵 Minor

6. Two flags now encode one decision. useElasticsearchFiltering is set only by ContentDriveHelper, and only when INDEX field criteria exist, so the text group in buildBaseESQuery / buildPureESQuery is unreachable from any in-product caller (external OSGi callers aside). Either drop it or mark it explicitly as external-caller-only — along with the +catchall bug above.
7. ContentDriveHelper's class javadoc still says "Uses Elasticsearch for text filtering while maintaining database reliability" — stale after this change.
8. keywordSearch_uppercaseIMG_findsFile and keywordSearch_digits1004_findsFile are strict subsets of keywordSearch_findsFileAsset_caseInsensitive_anySubstring. The class already runs on DataProviderWeldRunner, so a @DataProvider over the terms would report each one separately instead of collapsing them into one fail().
9. Test: languageId(1) is hardcoded while the search filters by the system default language — better derived from APILocator.getLanguageAPI().getDefaultLanguage().getId(). There's also no @AfterClass cleanup (site, content type, temp dirs leak), and removeContentFromIndex leaves the index inconsistent for the rest of the suite.
10. No unit-level coverage for the two new branches. BrowserAPITest.test_buildBaseESQuery_withDifferentFilterCombinations (already in MainSuite3a) is the natural home for two cases: resolveTextInDb=true producing no text group, and the isUseElasticSearchForFiltering truth table.

---

What's good

• Blast radius is correctly contained: useElasticsearchFiltering is only ever set by ContentDriveHelper, so /v1/browser and the legacy JSP paths are byte-identical, and the false default guarantees it.
• The read-your-writes test (de-index, then still find it) genuinely proves the behavior change — it would have failed before the fix.
• The composition test asserts exclusion when the field filter doesn't match, not just inclusion. 👍
• No DB schema, no ES mapping, no reindex, and /v1/drive/search is @Hidden so no openapi.yaml churn — the rollback-safe claim holds.

Recommendation

As it stands the PR fixes a write-side problem (index lag) with a read-side regression (a full site scan per keystroke), and it doesn't close the precision half of the issue. I'd suggest (4) + (1) or (2) from the alternatives above, with (3) if a server-side guarantee is required, and I wouldn't leave resolveTextInDb as Content Drive's default. Item 1 (suite registration) should be fixed regardless of which direction the fix takes.