3.3. Retrieval
The assistant is expected to answer questions about specific code bases — SO3, BitBake-based build systems, LVGL, U-Boot — that no general model has memorised. Retrieval is what closes that gap.
3.3.1. Corpus
The vector store lives in client/chromadb/. It is entirely
rebuildable: nothing in it is authoritative, it is a derived index of source
trees that live elsewhere.
Two ingestion entry points:
index_corpus.pyFull corpus build across the configured project set.
index_dir.pyIncremental ingestion of a single directory, for when one tree has moved on and a full rebuild would be wasteful.
Both honour SPEAR_DB_PATH (--db-path), so a test — or a container —
indexes into its own store rather than the one the assistant queries. Both
answer --help with their usage and index nothing.
3.3.1.1. Which tree, and which collection
/reindex rebuilds the corpus’s index — its tree and its collection — and
neither is the current directory whenever the session runs outside its corpus,
which is the normal case after a workspace split. Registered corpora that sit
under that tree are excluded from it: a tree owning an index is never part of
another one, and re-walking them would spend the file cap on trees indexed
elsewhere.
The registry itself is one implementation with two front doors: /corpus
(list/add/rm/scan) inside a session and the spear-corpus
CLI, both calling handle_corpus_command.
3.3.1.2. A truncated index is a refusal, not a warning
index_dir.py walks until it hits --max-files (60000 by default).
Reaching that cap is an error: exit 2, nothing embedded, nothing committed,
and the previous index left intact — the staged collection is only swapped in
at the very end.
The reason is that a partial index is not a sample. It is whichever
directories os.walk reached first, and it answers confidently from that
fraction: a build-system tree that vendors QEMU and U-Boot can yield an index
of vendored source with none of the build system it exists for, and
nothing fails. The refusal names the directories that consumed the budget, so
the fix — usually --exclude on a vendored tree that belongs in its own
corpus — is visible in the message. --allow-partial still builds it, for
whoever has a reason.
Cross-cutting notes are registered as a separate "shared": true corpus.
They are indexed once and attached at retrieval time, so changing a note does
not require rebuilding every project collection.
Which trees are corpora, and how a path resolves, is the registry’s business (Projects and corpora).
3.3.3. Prompt assembly
On the normative runtime, a turn’s context is composed by ContextEngine
from explicit layers (an implementation turn on the coding core builds its own
request from the project’s rules, workspace knowledge, skills and admitted
external capabilities, and reads the tree through its tools instead of from
retrieved chunks):
system and tool-use guidance;
matching project rules;
the current
WorkingStateprojection;compacted and recent conversation;
the skills the workspace’s scope admits, and retrieved corpus chunks;
bounded tool evidence.
Every item retains provenance, priority and a token estimate. Stable rules do not contain current plan, failure or verification state. Those dynamic facts come from the WorkingState layer.
3.3.4. Workspace knowledge, rules and skills
What a workspace keeps across sessions is its workspace knowledge (Workspace knowledge), selected by exact workspace and lifecycle, never by resemblance. Rules, skills and orientation maps are selected the same way, deterministically (What a turn is shown).
3.3.5. Embedding on another machine
Corpus embedding is the one stage worth moving. Measured on identical real chunks (median 1533 characters): 28 chunks/s on the laptop’s RTX 4060 against 279 on an RTX PRO 6000. Reading and chunking 984 files takes 0.17 s, so indexing is embedding. Only the compute moves — chunking and the Chroma writes stay local, so there is no source tree to mirror and no index to copy back.
Queries are never offloaded: one sentence per turn embeds in 44 ms here against 58 ms there plus the round trip, so the laptop wins.
3.3.5.1. The client sends the semantics; the worker executes
The client and the worker speak a versioned protocol, defined in
server/embed/protocol.py. Every field that decides what a vector means
travels in the request:
Decision |
Decided by |
Why |
|---|---|---|
model, prefix, sequence cap, normalisation, batch size |
client |
they define the collection, and the client owns the collection |
device, which card, dtype |
server |
properties of the machine, which a laptop cannot know |
The worker has no model registry, no default prefix and no per-model branch. That is the correction this protocol exists to make: the worker it replaces imported the client’s module on the GPU host and read the client’s registry from whichever copy was installed there, so the document prefix, the cap and the normalisation were decided by a file nobody was comparing against the client’s. A collection filled with two different prefixes is inconsistent in a way no later query reports — results merely get worse.
spear_embed_protocol is carried and checked in both directions. A
mismatch is refused, naming both numbers, and is never negotiated down: two
sides that still parse each other’s bytes while disagreeing about who applies
the prefix would produce vectors that are subtly wrong and perfectly
well-formed.
server/embed/README.md is the deployment contract — what to copy, how to
configure it, and how to verify it without indexing anything.
3.3.5.2. A configured destination is required, never preferred
Configuration |
Behaviour |
|---|---|
no destination |
local embedding, a valid deployment |
destination configured |
remote, or an error. Never a local substitute. |
Falling back to local compute when the remote embedder fails looks like a
kindness — a slower index beats a failed one. It is not. The two devices
load the same weights and do not produce the same vectors, so a collection
half-filled from each is quietly inconsistent. RemoteEmbeddingError is
raised and never swallowed.
Before a deployment switches workers, client/tests/test_embed_equivalence.py
runs both on real weights and requires the vectors to be bit-identical —
not merely close. Measured on BAAI/bge-m3, max absolute difference 0.0
for documents and for queries, on CPU/fp32 and on CUDA/fp16 alike.
The tolerance question is not hand-waved. The same texts embedded singly and in one batch differ by padding alone: ~2e-07 on CPU/fp32 and ~4.9e-04 on CUDA/fp16. A tolerance loose enough to cover the GPU figure would be large enough to hide a genuinely different encoding, so bit-exactness is the criterion — and it is available precisely because both paths reduce to the same call.
That second number is worth knowing operationally too: changing the batch size changes the vectors, on fp16 well above float32 noise, though at cosine 0.999999 it is far below anything retrieval can notice.
3.3.6. Retrieval budget
Retrieval competes with the conversation for the context window (asked of the server at startup, 32 768 tokens when it cannot say). Two mechanisms keep it bounded:
the number and size of retrieved chunks are capped at assembly time;
tool output is truncated before it reaches the model — 10 000 characters on the normative runtime, 50 000 on the coding core.
The second one matters more than it looks. A single make on a failing
build can emit hundreds of kilobytes; without truncation one tool call would
evict the entire conversation and the assistant would lose the thread of what
it was doing.