Skip to content

Semantic index

The conversation module's semantic index is a local, owner-invoked build step: it walks your Obsidian vault, chunks each note, embeds the chunks with a local model (no cloud call), and stores the vectors in a sidecar sqlite-vec database. Everything else in the module — search_vault, the chat tab, and RagClient.ask — reads from that store.

It is dormant: there's no scheduled job for it. You run it by hand, whenever you want the index to catch up with the vault.

Install the embedding dependency

pip install -r modules/conversation/agent/requirements-embed.txt

This installs fastembed, the local ONNX embedding runtime. It's not part of the base install and not present in CI — CI runs the indexer against a fake embedding provider instead (see Embedding models below).

Build the index

JARVIS_VAULT_DIR=~/vault \
  PYTHONPATH="agent-runtime/src:modules/conversation/agent" \
  python -m conversation_agent.agent index

JARVIS_VAULT_DIR is required — the command exits immediately with Missing required env var: JARVIS_VAULT_DIR if it's unset. On success it prints a one line summary:

conversation index: IndexSummary(notes_seen=..., chunks_added=..., chunks_updated=..., chunks_pruned=..., chunks_unchanged=...)

What it does

  1. Walk. Every *.md file under JARVIS_VAULT_DIR, at any depth, is a candidate note.
  2. Exclude. Three globs are excluded by default: docs/**, .obsidian/**, .trash/**. Add more with JARVIS_INDEX_EXCLUDE, a :-separated list of extra fnmatch globs (for example JARVIS_INDEX_EXCLUDE="drafts/**:private/**").
  3. Chunk. Each note's frontmatter is stripped, the body is split on markdown headings, and each section is packed into token-budgeted pieces (512 tokens, 64-token overlap between consecutive pieces of the same section). Every chunk gets a stable chunk_id (derived from the source path, heading anchor, and position) and a content_hash (derived from the chunk's text).
  4. Re-embed only what changed. The indexer compares each chunk's content_hash against what's already stored. A chunk whose hash matches is left alone — zero embedding calls. A chunk that's new or whose text changed gets embedded (as a passage) and upserted.
  5. Prune. After the walk, any previously-indexed chunk that no longer comes out of a still-present note, or that belonged to a note deleted from the vault entirely, is removed from the store.

Idempotent by design

Running index twice in a row against an unchanged vault performs no embeddings and no writes — the second run's summary is all chunks_unchanged, zeros everywhere else. This makes it safe to re-run the command as often as you like, e.g. after every vault-editing session.

Embedding models

Setting Model Dim Notes
Default sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 384 Multilingual (FR/EN), runs on CPU via fastembed/ONNX — no torch dependency.
JARVIS_EMBED_MODEL=intfloat/multilingual-e5-large intfloat/multilingual-e5-large 1024 Heavier e5-family alternative. e5 models require asymmetric query: / passage: text prefixes, which are applied automatically only when the model name contains e5.
JARVIS_EMBED_FAKE=1 — 8 A deterministic hash-based fake provider — no model download, no network. Used by CI and for offline runs; it ignores kind (query and passage embed identically), which is fine for tests but not meant for real retrieval quality.

Set JARVIS_EMBED_MODEL to override the default model id. The vector store records its own dimensionality on creation, so switching models against an existing store requires rebuilding it (see below).

Where the index lives

The vectors go into a sidecar sqlite-vec file, resolved in this order:

  1. JARVIS_VECTOR_DB, if set — used as-is.
  2. Otherwise, vectors.db under JARVIS_PB_DATA (or pb_data if that's unset too).

This file is never written inside PocketBase's own data.db — it's a separate, independent SQLite database (with a vec0 virtual table via the sqlite-vec extension) that only the conversation module reads and writes.

Rebuildable

The vector store holds nothing that isn't re-derivable from the vault. If you ever want a clean rebuild (e.g. after switching embedding models), stop anything using it, delete the file at the resolved path, and re-run index — it will treat every note as new and re-embed the whole vault.

See Configuration for where JARVIS_VECTOR_DB, JARVIS_EMBED_MODEL, JARVIS_INDEX_EXCLUDE, and JARVIS_PB_DATA are set, and Chat & RAG for how the index gets queried once it's built.