7.3. Directory layout
The code lives in the checkout — ~/spear in the examples; the tree can sit
anywhere. Two directories outside it matter as well: the state directory,
which may sit outside the checkout (Persistent state), and, on an inference
host, the runtime root that scripts/bootstrap-runtime.sh populates
(~/spear-runtime in the examples, Rebuilding the runtime). The
checkout itself mixes three quite different kinds of content — application
code, multi-gigabyte model weights, and persistent state — so it is worth
knowing which is which before running a find over it.
Fig. 7.2 The checkout at a glance: application, models, state.
7.3.1. Top level
Path |
Content |
|---|---|
|
The application and its Python virtualenv. |
|
The generic inference server component (see The server component). |
|
Release, configuration and image scripts ( |
|
The container image: |
|
Model weights, when a model is served locally. Not tracked. |
|
The |
|
The fine-tuning working area: LoRA trainers, GPU-host scripts, corpus builders and the load preflight. The governed training path lives in the application instead (Training and fine-tuning). |
|
Small trees indexed as their own corpora ( |
|
|
|
This documentation (Documentation). |
|
The only CI workflow: it builds this documentation and publishes it. |
Note
client/ being both the application directory and the virtualenv root
is why bin/, lib/ and include/ sit next to the packages.
It is unusual but deliberate: the app and its exact dependency set travel
together.
7.3.2. The application
The code is one package per concern, imported from client/ as their common
root (from evidence import completion). rag_chat and the other
commands are run as files (client/bin/python client/cli/rag_chat.py); the
launchers and the image do that for you.
Package |
Role |
|---|---|
|
The commands. The chat client’s entry point is |
|
The coding core: its loop, tool dispatch, tools and prompt. It imports
nothing of SPEAR and talks to a |
|
The execution harness: |
|
The provider-neutral model/tool loop of the normative and general
runtime ( |
|
The evidence plane: canonical evidence, source epochs and the
implementation verdict ( |
|
What a turn is given: deterministic context selection, the workspace
context, workspace knowledge ( |
|
The MIXED pipeline ( |
|
The normative store: ingestion, structure, semantic records, retrieval, review and the standard’s tools. |
|
Provider-neutral model turns ( |
|
Embedding and corpus ingestion into Chroma ( |
|
The fine-tuning subsystem: capture, curation, governance, readiness,
frozen bundles and the operator control plane, none of it
model-visible (Training and fine-tuning); |
|
The test suite; see Testing. |
|
Evaluation harnesses that call no model, the retrieval one first
( |
|
The real-model benchmark runner and its ablations
( |
|
Installing the application on a machine: |
|
Two standalone notes kept beside the code, outside this manual. |
Beside the packages, client/ holds the launchers spear-chat.sh and
spear-corpus.sh (Entry points), tool-guide.md (the tool usage guide
handed to the model) and TRAINING_DATA.md (the operator’s notes
on the training-data workflow, Training and fine-tuning).
7.3.3. The server component
Path |
Content |
|---|---|
|
|
|
|
|
The embedding worker ( |
|
|
|
|
|
The server’s own suite; it needs no GPU (Testing). |
7.3.4. Configuration
These files sit in client/. The ones marked ignored are git-ignored:
each machine writes its own, usually from the .example file beside it.
File |
In git |
Meaning |
|---|---|---|
|
ignored |
Absolute path of the GGUF that |
|
tracked |
LoRA adapter path, or the literal |
|
tracked |
The embedding model that indexing and retrieval use.
|
|
ignored |
The command that runs the embedding worker on a remote host.
|
|
ignored |
Connection details for the remote vLLM pod used by
|
|
ignored |
Host, port and model of a shared GPU server used by
|
|
ignored |
Settings specific to THIS machine: the |
|
ignored |
Named workspaces the chat can open, mapping a short name to an absolute path and a project kind. |
|
ignored |
External capability providers registered per workspace (MCP servers
over stdio). |
|
tracked |
The tool usage guide handed to the model. |
|
tracked |
Prompt fragments. Each one applies where the header it opens with
scopes it, and a rule without a header applies nowhere
(Projects and corpora). |
|
tracked |
Task recipes, each optionally opening with a |
active-backend.conf is state, not configuration: see Persistent state.
7.3.5. Content the deployment owns
rules.d/, skills/ and benches/ are content, not code — and a
deployment’s own rules, its learned skills and the bench that rates it are
exactly the material that does not belong in a public tree. Each therefore
answers to an environment variable:
Variable |
Default |
What it holds |
|---|---|---|
|
|
the rules, and |
|
|
the skill library, read and written — |
|
|
the acceptance benches a project declares by bare name |
|
unset |
further content trees, colon-separated, each holding |
The default is the in-tree directory; no default points outside the checkout, which is what keeps a private path out of public source. An empty value counts as unset rather than naming the filesystem root.
A directory that is not there reads as empty, without a warning: a variable
that points at the wrong place gives a session with no rules and no skills.
The same three variables are read by scripts/docker/build.sh when it bakes
an image (The optional inputs), so one setting covers both.
7.3.6. Persistent state
What a session accumulates goes to the state directory:
SPEAR_STATE_DIR (--state-dir), or client/ when it is not set.
spear-configure writes SPEAR_STATE_DIR into machine.env, by default
~/.local/state/spear; the container sets it to its mounted state volume
(Container).
Not everything follows that rule. The vector index and the local server’s log
stay in client/ whatever the state directory is. The knowledge store,
and the standard store as the standard review commands open it, fall back to
~/.local/state/spear rather than client/ when SPEAR_STATE_DIR is
unset. Setting SPEAR_STATE_DIR therefore puts everything but the first
two rows below in one place.
Path |
Where, when nothing is set |
Content |
|---|---|---|
|
|
The vector store. |
|
|
Log of the local |
|
|
Workspace knowledge ( |
|
|
The normative-standard store. |
|
state directory |
Append-only audit trail of mutating tool attempts. Metadata only: assignments that look like secrets are redacted before the record is written. |
|
state directory |
The runtime trace, written only with |
|
state directory |
Resumable task snapshots, referenced large results, and pre-mutation rollback bytes. Active sessions may reference the latter two. |
|
state directory |
Canonical training episodes, the source a bundle freezes from
(Training and fine-tuning). |
|
state directory |
Rules taught at runtime with |
|
state directory |
Conversation transcripts, one per corpus, and their archive. |
|
state directory |
Legacy remembered notes and their metadata. Not shown to a turn;
their content reaches one only once migrated into workspace knowledge.
|
|
state directory |
Rated agent trajectories and the samples |
|
state directory |
The cached result of probing a project’s build and test commands. |
|
state directory |
The training host the operator control plane launches on; an example
ships as |
|
state directory |
The backend chosen last, preselected by the startup picker. |
|
state directory |
The prompt’s line-editing history. |
<tag> is derived from the corpus root, so history follows the tree rather
than the directory a session was launched from.
7.3.7. Entry points
The repository provides these scripts; linking them onto PATH under the
names below — a one-line wrapper in ~/.local/bin each — is the usual
arrangement, and the rest of this documentation uses the names:
Name |
Script |
What it does |
|---|---|---|
|
|
the assistant |
|
|
the corpus registry ( |
|
|
index any tree |
|
|
rebuild a build-system corpus with the curated walk |
|
|
|
|
|
the assistant in a container, this machine’s trees mounted (Container) |
|
|
a published image on a machine that has docker and nothing else (Running the reds image) |
spear-chat --help lists every flag — permissions, model, corpus — and is
answered before any server is started or any corpus resolved, so it costs
nothing. The launcher invokes the venv interpreter by path rather than
sourcing bin/activate: a virtualenv hardcodes its own absolute path, so a
relocated tree activates into a directory that no longer exists and
python3 silently falls through to the system interpreter.
spear-corpus is a wrapper around
corpus_registry.handle_corpus_command, which is also what /corpus calls
inside the chat: one registry, one implementation, and one environment — it
reads machine.env exactly as spear-chat does.