Skip to content

Support batch vector query API and shared flat KNN scan #6821

Description

@BubbleCal

Parent: #6820
Linear: https://linear.app/lancedb/issue/OSS-923/support-batch-vector-query-api-and-shared-flat-knn-scan
Linear parent: https://linear.app/lancedb/issue/OSS-922/vector-search-performance-optimization-roadmap
Related historical issue: #876

Goal

Allow one vector search request to contain multiple query vectors and make the first implementation immediately useful for flat KNN by sharing scan/load work across the query batch.

Scope

  • Add or finalize the public API shape for batch query vectors in Rust and Python.
  • Return results with an explicit per-query discriminator, for example query_index, so callers can split top-k results by query vector.
  • Apply limit/top-k per query, not globally across all query vectors.
  • For flat KNN, load each data batch once and evaluate all query vectors against that loaded batch.
  • It is acceptable for the first version to keep ANN/indexed search out of scope, as long as the behavior is explicit and covered by tests.

Acceptance criteria

  • Rust tests compare batch query results against repeated single-query results.
  • Python tests cover 2-D query input and validate output grouping by query id.
  • Docs/docstrings explain the batch input shape and result contract.
  • A small flat KNN benchmark demonstrates lower tail latency versus issuing repeated independent queries, especially when I/O or decode cost dominates.

Metadata

Metadata

Labels

A-indexVector index, linalg, tokenizerA-pythonPython bindingsenhancementNew feature or requestperformancerustRust related tasks

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions