Vector Search

Pylon ships vector search as three pieces that compose:

field.vector(dims): an embedding column on an entity.
ctx.llm.embed(texts): batch embeddings from OpenAI or Voyage.
ctx.db.vectorSearch(entity, query): exact k-nearest-neighbor over the stored vectors.

Pylon needs no extension, no sidecar, and no separate vector database. The embedding is a regular column: a packed float array stored as BLOB on SQLite and BYTEA on Postgres. The search itself is an exact scan, scored in Rust.

Declare the field

import { entity, field } from "@pylonsync/sdk";

export const Doc = entity("Doc", {
  title: field.string(),
  body: field.string(),
  status: field.string(),
  embedding: field.vector(1536).optional(),
});

field.vector(dims) fields are always server-only. A 1536-dimension embedding is over 6KB per row, so it never travels over the sync stream, never appears in HTTP entity reads, and is stripped from search hits. Server functions read it via ctx.db.get. Writes validate the shape: the value must be a number array of exactly dims finite elements (or null to clear an optional field). Anything else fails with VECTOR_INVALID.

Write embeddings

ctx.llm.embed is available in mutations and actions. A mutation can embed and store in one transactional scope. The provider call holds the write transaction open for its duration, so for high-write apps, prefer the action shape: embed outside the transaction, then store with a small mutation:

// functions/indexDoc.ts — action: embed (external I/O), store via a mutation
import { action, v } from "@pylonsync/functions";

export default action({
  args: { docId: v.string() },
  handler: async (ctx, { docId }) => {
    const doc = await ctx.runQuery("getDoc", { docId });
    if (!doc) return { ok: false };
    const [embedding] = await ctx.llm.embed([`${doc.title}\n\n${doc.body}`]);
    await ctx.runMutation("saveEmbedding", { docId, embedding });
    return { ok: true };
  },
});

// functions/saveEmbedding.ts — mutation: the transactional write
import { mutation, v } from "@pylonsync/functions";

export default mutation({
  args: { docId: v.string(), embedding: v.array(v.number()) },
  handler: (ctx, { docId, embedding }) =>
    ctx.db.update("Doc", docId, { embedding }),
});

(Actions have no ctx.db. Reads and writes from an action go through ctx.runQuery or ctx.runMutation.) ctx.llm.embed resolves its provider independently of chat:

Env	Effect
`OPENAI_API_KEY`	Default: OpenAI `text-embedding-3-small` (1536 dims), even when chat runs Anthropic
`PYLON_EMBEDDINGS_PROVIDER=voyage` + `VOYAGE_API_KEY`	Voyage `voyage-3.5` (1024 dims)
`PYLON_EMBEDDINGS_MODEL`	Model override
`PYLON_EMBEDDINGS_API_KEY`	Key override (either provider)
`PYLON_EMBEDDINGS_BASE_URL`	OpenAI-compatible endpoint override

Match field.vector(dims) to the model: 1536 for text-embedding-3-small, 3072 for text-embedding-3-large, 1024 for voyage-3.5. embed is not available in queries. A reactive query re-runs on every dependency change and would re-bill the provider each time. Embed in a mutation or action, then store the vector.

Search

ctx.db.vectorSearch lives on ctx.db, available from queries and mutations. The typical flow embeds the search text in an action, then searches from a query:

// functions/searchDocs.ts — action: embed the text, delegate the search
import { action, v } from "@pylonsync/functions";

export default action({
  args: { q: v.string() },
  handler: async (ctx, { q }) => {
    const [vector] = await ctx.llm.embed([q]);
    return ctx.runQuery("findSimilar", { vector });
  },
});

// functions/findSimilar.ts — query: read-only vector search
import { query, v } from "@pylonsync/functions";

export default query({
  args: { vector: v.array(v.number()) },
  handler: async (ctx, { vector }) => {
    const { hits } = await ctx.db.vectorSearch("Doc", {
      field: "embedding",
      vector,
      limit: 5,
      filter: { status: "published" },
    });
    return hits.map((h) => ({ id: h.id, score: h.score, title: h.doc.title }));
  },
});

The query shape:

field: which vector(dims) field to search (an entity may have several).
vector: the query embedding. Its length must match the declared dims.
limit: the maximum number of hits, default 10, capped at 200.
metric: "cosine" (default, higher means closer), "dot", or "l2" (Euclidean distance, lower means closer).
filter: an equality pre-filter applied in SQL before scoring. A plain value means equality, an array means IN, and null means IS NULL.

Hits come back best-first as { id, score, doc }. doc is the full row with vector fields stripped. Re-fetch by id if you need the embedding itself. The same query works over HTTP: POST /api/vector-search/Doc with the query as the JSON body. The route enforces read policies. Entities whose read policy depends on row data are refused (SEARCH_REQUIRES_ROW_INDEPENDENT_POLICY), because top-k ranking over every row would leak the proximity of rows the caller cannot read. This is the same rule faceted search aggregates follow.

Scaling limits

The scan is an exact k-NN search, not an approximate index. Every non-NULL embedding is decoded and scored on each search. That is the right trade-off until your table grows large: exact search gives perfect recall, needs no index maintenance, and adds no extra infrastructure. Rough guidance at 1536 dimensions: 10,000 rows score in a few milliseconds, and 100,000 rows score in the low hundreds of milliseconds. Past that point, use filter to shrink the candidate set (by tenant, status, or collection), or move that entity’s retrieval to a dedicated vector store. An approximate nearest-neighbor (ANN) index can land behind this same API later, without changing your code.

RAG in one action

export default action({
  args: { question: v.string() },
  handler: async (ctx, { question }) => {
    const [vector] = await ctx.llm.embed([question]);
    // findSimilar is the query from the Search section above.
    const hits = await ctx.runQuery("findSimilar", { vector });
    const context = hits.map((h) => h.body).join("\n---\n");
    const res = await ctx.llm.complete({
      messages: [{ role: "user", content: question }],
      system: `Answer from this context only:\n${context}`,
    });
    return { answer: res.content, sources: hits.map((h) => h.id) };
  },
});

Get started

Core concepts

SSR

Auth

Plugins

Clients

Operations

Compare

Migrate

Vector Search

Declare the field

Write embeddings

Search

Scaling limits

RAG in one action

​Declare the field

​Write embeddings

​Search

​Scaling limits

​RAG in one action

Declare the field

Write embeddings

Search

Scaling limits

RAG in one action