6. The model and its weights
Where the answers come from: which model is served, how it is brought up, how it is trained on the harness’s own usage, and — the page that exists because the others would otherwise repeat its lessons — what the earlier attempts measured before this one was chosen.
- 6.1. Model backends
- 6.1.1. Selecting a backend
- 6.1.2. Ports and endpoints
- 6.1.3. Pointing at an endpoint directly
- 6.1.4. The context window
- 6.1.5. Sampling
- 6.1.6. Recording and replaying
- 6.1.7. The provider interface
- 6.1.8. Prompt caching on the Anthropic path
- 6.1.9. Authenticating against Anthropic
- 6.1.10. The
stop_reasoncontract - 6.1.11. Smoke test
- 6.2. Model serving
- 6.3. Setting up an inference host
- 6.3.1. What you are building
- 6.3.2. 1. Host prerequisites
- 6.3.3. 2. The checkout
- 6.3.4. 3. The runtime
- 6.3.5. 4. The configuration
- 6.3.6. 5. First run, in the foreground
- 6.3.7. 6. Supervision
- 6.3.8. 7. The embedding worker
- 6.3.9. 8. Client access
- 6.3.10. 9. Verifying the whole
- 6.3.11. 10. Updating and rolling back
- 6.3.12. If the harness itself also runs on this host
- 6.4. Rebuilding the runtime
- 6.5. Training and fine-tuning
- 6.5.1. The five stages
- 6.5.2. Capture is wider than judging
- 6.5.3. Governance: where a sample may come from
- 6.5.4. Readiness is evidence, not a feeling
- 6.5.5. The frozen bundle
- 6.5.6. The MoE lesson
- 6.5.7. Will it fit? Measure it
- 6.5.8. The operator surface
- 6.5.9. Where it runs today
- 6.5.10. Related material
- 6.6. Model history and what it cost
Model backends is the client side: the endpoints and ports the client can talk to, how it is pointed at one, and what it sends. Model serving covers the serving side: the quantised weights, the GPU/CPU split and the flags that decide throughput. Setting up an inference host is the procedure: an empty GPU machine to a supervised server with clients attached. Rebuilding the runtime is what has to be true before the first token, and what the harness does when it is not.
Training and fine-tuning is the fine-tuning side, and it stands on its own: how the harness turns its own usage into a governed dataset, and what it takes to run a job on the card that is also serving the model.
Model history and what it cost is the record of what was tried and what it cost — including the experiment that rented a B200 to learn something a smaller run would have shown. It is kept because a choice whose alternatives are forgotten gets re-litigated every year.