skillZs
★ LIVE SKILL TAGS ★
>>> LIVE SKILLS INDEX <<<
* OPEN SOURCE *
NO LOGIN, NO TRACKING
※ REAL INSTALL DATA ※
← back to all skills
scylladb/agent-skills60 installs

scylladb-vector-search

Guides users through implementing and optimizing ScyllaDB Vector Search and full-text search for semantic similarity, keyword search, RAG, and similar use cases. Use this skill when users need to store and query embeddings, build vector indexes (HNSW), run approximate nearest neighbour (ANN) queries, apply filtering (global/local secondary indexes), configure quantization, integrate with LLM frameworks (e.g. LangChain), create full-text indexes and BM25 queries, combine full-text and vector results (hybrid search), or run Vector Search locally with Docker for testing. Schema design outside the vector and text columns goes to scylladb-data-modeling. Triggers on "vector", "embeddings", "similarity search", "ANN", "nearest neighbor", "RAG", "semantic search", "recommendation system", "vector index", "HNSW", "vector-store", "full-text search", "BM25", "keyword search", "hybrid search", "fulltext_index".

How do I install this agent skill?

npx skills add https://github.com/scylladb/agent-skills --skill scylladb-vector-search
view source ↗

Is this agent skill safe to install?

  • Gen Agent Trust Hubpass

    The skill provides comprehensive documentation for implementing ScyllaDB Vector Search. It utilizes official ScyllaDB resources for additional configuration and emphasizes secure development practices, such as using prepared statements and placeholders for credentials.

  • Socketpass

    No alerts

  • Snykpass

    Risk: LOW · No issues

  • ZeroLeakspass

    Score: 93/100 · 2 sections analyzed

What does this agent skill do?

ScyllaDB Vector Search and Full-Text Search

If the task goes beyond this skill, use the parent scylladb skill for a ScyllaDB overview and the list of other ScyllaDB skills. If it is not installed, fetch it from https://raw.githubusercontent.com/scylladb/agent-skills/main/skills/scylladb/SKILL.md.

Vector Search stores embeddings in a vector<float, N> column and answers approximate nearest neighbor (ANN) queries from an HNSW vector index. Full-text search ranks rows by BM25 from a fulltext_index on a text column (ScyllaDB 2026.3.0+). Both index types run on the same indexing nodes, so the deployment rules below apply to both. For production they are available only on ScyllaDB Cloud. The indexes are maintained outside the table, so TRUNCATE does not clear them.

When to Use This Skill

  • The user wants to store embeddings in ScyllaDB and run similarity search
  • The user is building semantic search, RAG, or a recommendation system on ScyllaDB
  • The user is creating a vector index or tuning HNSW parameters
  • The user needs to combine ANN queries with filters on other columns
  • The user wants to reduce vector index memory with quantization
  • The user needs a driver version that supports the vector CQL type
  • The user wants to try Vector Search locally or in CI with Docker
  • The user needs keyword or full-text search (BM25) over text columns
  • The user wants hybrid search: full-text and vector results for one query

For adjacent topics, use the sibling skill instead:

  • Partition keys and table design for the non-vector parts of the schema: scylladb-data-modeling
  • Connecting to the Cloud cluster and choosing a driver: scylladb-cloud-setup
  • Similarity search through the DynamoDB API instead of CQL: scylladb-alternator

Where Vector Search and Full-Text Search Run

  • Production: ScyllaDB Cloud only. Vector Search runs on dedicated indexing nodes that ScyllaDB Cloud manages and syncs from the storage nodes through CDC. Self-hosted and on-prem deployments are not supported. If the user plans one of these for production, tell them that Vector Search requires a standard ScyllaDB Cloud cluster with Vector and Text Search enabled. Full-text search uses the same indexing nodes and also needs ScyllaDB 2026.3.0 or later.
  • Development and testing: local Docker. Run a ScyllaDB container plus a separate scylladb/vector-store container (the indexing service) connected to it. Full-text search also needs this container. See local-docker.

Core Principles

  1. Understand before building — Validate the use case to ensure Vector Search is the right solution
  2. Inspect schema first — Check existing tables and indexes before making recommendations
  3. Explain before executing — Describe what indexes will be created and confirm before proceeding
  4. Start with defaults — Use default HNSW parameters and no quantization; tune only when needed

Workflow

1. Discovery Phase

Understand the use case:

  • What type of data are they searching? (text, images, audio, structured data)
  • What embedding model are they using or planning to use?
  • How many vectors will they store? (affects quantization decision)
  • Do they need filtering alongside similarity search? (affects index type: global vs. local)
  • What latency/throughput requirements do they have?

Common use cases:

  • Semantic search — Find documents/passages matching the meaning of a query
  • RAG (Retrieval-Augmented Generation) — Provide relevant context to an LLM
  • Recommendation systems — Find items similar to those a user interacted with
  • Image/audio search — Find visually or acoustically similar media
  • Anomaly detection — Identify outliers far from clusters in vector space
  • Deduplication — Find near-duplicate records

2. Determine Requirements

Before creating tables and indexes, establish:

ParameterHow to Determine
DimensionsFrom the embedding model (e.g., 384, 768, 1536)
Similarity functionFrom the embedding model docs (COSINE is default and safe for most)
Need filtering?Does the query combine similarity with metadata constraints?
Dataset size< 1M vectors → no quantization; 1M-10M → consider i8; > 10M → consider b1

3. Consult Reference Files

Always consult the matching file in the Quick Reference before recommending indexes or queries.

4. Implementation

Typical implementation order:

  1. Create a keyspace (tablets are enabled by default)
  2. Create a table with a vector<float, N> column
  3. Insert vectors (from your embedding pipeline)
  4. Create a vector index (CREATE CUSTOM INDEX ... USING 'vector_index')
  5. Run ANN queries (ORDER BY vec_col ANN OF [...] LIMIT k)
  6. Add filtering if needed (local index preferred for performance)
  7. Add quantization if needed for memory savings

5. Validation

After setup, verify:

  • The ANN query returns results (wait ~1 second after inserting vectors to account for write-to-query latency)
  • Similarity scores are in the expected range (0-1, higher = more similar)
  • Filtered queries return correctly filtered results

Anti-Patterns to Avoid

NEVER use ALLOW FILTERING with vector search when a local index would work: Global vector indexes require ALLOW FILTERING when adding a WHERE clause and are always much slower than local indexes. Design the schema so filter columns are part of the partition key and use a local vector index.

NEVER mix embedding models: Vectors from different embedding models live in incompatible vector spaces. If you change the model, you must re-embed and re-index all data.

NEVER skip the LIMIT clause: ANN queries require a LIMIT — ScyllaDB will reject the query without one.

Handling Edge Cases

User doesn't have an embedding model yet:

  • Recommend starting with all-MiniLM-L6-v2 (384 dims, open weights, good general-purpose)
  • For production: suggest evaluating OpenAI or Cohere embedding models

User's cluster doesn't have Vector Search enabled (queries fail with Vector Store is disabled):

  • Direct them to the ScyllaDB Cloud UI → cluster details → Enable Vector and Text Search
  • See Vector and Text Search Deployments for setup
  • On a local Docker setup, check that the vector-store container is running and ScyllaDB has --vector-store-primary-uri set (see local-docker)

Query returns no results:

  • Check write-to-query latency (~1 second delay after insert)
  • Verify the query vector dimensions match the index dimensions
  • Verify the index was created successfully
  • Ensure the embedding model is the same for indexing and querying

TTL is needed:

  • Use per-row TTL (ScyllaDB 2026.2.0+). Cell-level TTL (USING TTL, default_time_to_live) is accepted, but expired rows stay in the vector or full-text index and use slots against LIMIT.

User wants to clear all vectors (reset the table):

  • TRUNCATE does not clear the vector index. The vector store updates its index only through CDC, and TRUNCATE is not propagated to CDC — the CUSTOM INDEX remains, and previously indexed vectors stay in memory.
  • The only correct solution is to drop and recreate both the table and the custom index. See vector-type-and-indexing for the exact DDL.

Quick Reference

  • vector-type-and-indexing — The vector type, the tablets requirement, CREATE CUSTOM INDEX ... USING 'vector_index', similarity functions, HNSW parameters, and why TRUNCATE does not clear the index. Consult before creating a vector table or index, or when the user wants to reset vector data.
  • ann-queries — ORDER BY ... ANN OF ... LIMIT syntax, similarity scores, write-to-query latency, and CQL features that ANN queries do not support. Consult when writing or debugging an ANN query.
  • filtering — Global vs. local vector indexes, schema design for filtered and multi-tenant search, and their limitations. Consult when an ANN query needs a WHERE clause.
  • quantization — Quantization levels (f16, bf16, i8, b1), oversampling, rescoring, and why index options cannot change after creation. Consult when the dataset is over about 1M vectors or index memory is a concern.
  • driver-integration — Minimum driver versions for the vector type and insert/query examples in Python, Java, Go, and Node.js. Consult when writing application code that reads or writes vectors.
  • local-docker — Running ScyllaDB and the Vector Store service together in Docker, the flags and environment variables that connect them, and why this setup is not for production. Consult when the user wants Vector Search or full-text search locally, in CI, or in tests, or has no ScyllaDB Cloud cluster.
  • full-text-search — Requirements, BM25() query rules, and hybrid search with Reciprocal Rank Fusion. Consult before creating a fulltext_index, writing a BM25() query, or combining keyword and semantic results.

Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.

<a href="https://skillzs.dev/skills/scylladb/agent-skills/scylladb-vector-search">View scylladb-vector-search on skillZs</a>