Skip to content

Add filtering to vector index #21916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 26 commits into
base: devel
Choose a base branch
from
Draft

Conversation

jbajic
Copy link
Contributor

@jbajic jbajic commented Aug 19, 2025

Scope & Purpose

  • Fix tests
  • Avoid multiple materializations, once in RocksDBIndex iterator and one after EnumeraterNearVectorNode in MaterializeNode Move evaluation in callback

Introduces filtering to the vector index. This way, we will push the filtering expression to the vector index iterator and evaluate it there. A continuation of this would be an investigation into whether the batching of documents would be beneficial, and if so, implementing it.
Now queries like this should work, and the FilterNode will be merged with the EnumerateNearVectorIndexNode:

FOR d IN col
FILTER d.val > 3
LET dist = APPROX_NEAR_L2(d.vector, @q)
SORT dist LIMIT 3 RETURN d

This implementation still does not support storedValues usage and compound indexes.

  • 💩 Bugfix
  • 🍕 New feature
  • 🔥 Performance improvement
  • 🔨 Refactoring/simplification

Checklist

  • Tests
    • Regression tests
    • C++ Unit tests
    • integration tests
    • resilience tests
  • 📖 CHANGELOG entry made
  • 📚 documentation written (release notes, API changes, ...)
  • Backports
    • Backport for 3.12.0: (Please link PR)
    • Backport for 3.11: (Please link PR)
    • Backport for 3.10: (Please link PR)

Related Information

(Please reference tickets / specification / other PRs etc)

  • Docs PR:
  • Enterprise PR:
  • GitHub issue / Jira ticket:
  • Design document:

@jbajic jbajic self-assigned this Aug 19, 2025
@cla-bot cla-bot bot added the cla-signed label Aug 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant