size_opt
Right-sized models
Smallest model that hits quality and latency — often not the largest API card.
run_id=ft-2026 · eval=pass
// research → prodYour domain. Your model stack.
Base selection, fine-tunes, eval harnesses, and serving — including private local GPU when data cannot leave. Pragmatic over theatrical.
golden_set
1,240
regression
gate:on
privacy
local_gpu
router
multi-model
accuracy · golden 91.4%
latency · p95 210ms
val loss · gate 0.142
regression · suite pass
Why the stack has to fit
Fine-tunes and serving without eval gates is just expensive noise. We right-size models, measure quality, and ship private paths when required.
eval_first
Eval-first
Golden sets and regression gates before any deploy.
private_path
Private training
Local/high-end GPU options for sensitive corpora.
serve_ops
Serving that survives
Batching, fallbacks, observability, cost caps. modern model APIs routing when public APIs fit.
Pipeline modules
// 01 audit
Model audit
Baseline quality, cost, and risk of your current stack.
// 02 finetune
Fine-tune program
Dataset hygiene, training loops, and acceptance tests.
// 03 serve
Private serving
On-prem or VPC inference with monitoring.
// 04 route
Router design
Multi-model routing for cost vs quality tradeoffs.
Lab notes
When to fine-tune (and when not to)
Try retrieval or prompt changes for missing facts; fine-tune only when repeatable behavior improves on held-out tests.
Eval harness basics for product teams
Build evals from real task cases, edge cases and clear rubrics; compare model changes by failure category.
Private model serving without drama
Choose a serving path by control needs, traffic shape and operating ownership—not by hardware preference alone.