The Cost Problem
Frontier agents can do impressive work. The bill becomes harder to defend when repeated business workflows pay frontier-agent runtime costs on every case.
Model routing is the common first move: keep expensive models on planning and give smaller models the easy steps. That helps, but each case still does too much live planning.
A better answer changes the shape of the computation. A large slice of enterprise work sits between deterministic workflow and open-ended research. It is repetitive procedural work with a little local ambiguity: an insurance claim, a room booking, an appointment change, an airline rebooking, an HR intake. The process is known. The allowed tools are known. The policy gates are known. The inputs still arrive as messy natural language.
This is where specialized harnesses matter. The harness carries the stable structure: state, transitions, tool gates, output contracts, audit events, retries, and failure states. The model handles the local judgment slots where language understanding or fuzzy mapping earns its keep. Because the runtime is narrower, cheaper models can often do the work more accurately than a broad agent solving the whole case live.
Reason Every Time
Strong runtime model, broad tool surface, broad context, improvised plan, expensive per case.
Route Each Step
Better model portfolio, but the case still begins as a live planning and decomposition problem.
Run Cheaper, Score Higher
Spend strong-model effort at build time, freeze the task runtime, then execute cheaply and repeatedly with tighter task performance.
Why A Harness Compiler?
Specialized Harness Engineering argued for the core separation: the model supplies local judgment; the harness supplies structure. harnessc asks the next question: can that harness itself be generated from a task specification?
The compiler analogy is practical. At compile time, a frontier coding agent can read the SOPs, examples, tool schemas, rubrics, counterexamples, and representation notes. It can spend tokens profiling the task, drafting a declarative spec, writing narrow judgment prompts, inspecting traces, and proposing fixes. That cost is paid once per harness version.
At runtime, the frozen harness handles individual cases with a smaller surface area. The workflow becomes a constrained runtime with typed state, deterministic gates, declared model slots, preserved traces, and task-level evals. Smaller models get a better-shaped problem, so the system can be both cheaper and more accurate.
The bet is simple: spend more intelligence while compiling the harness so each runtime case can be cheaper, more accurate, more stable, and easier to audit.
What Went Open Source
The repo is vivekhaldar/harnessc, released under Apache-2.0. It is a working research repo for compiling and evaluating specialized task harnesses from task specifications. It is still a repo workflow rather than a hosted product or polished SDK.
Task Packets
SOPs, policies, tools, fixtures, train/dev/holdout examples, counterexamples, rubrics, and representation notes.
HarnessSpec
A declarative IR for slots, states, transitions, tool gates, judgment slots, context policy, audit events, and claims.
Kernel Runtime
Policy-free execution of specs: state merge, controller transitions, tool gateway, context assembly, model routing, and traces.
Validator Wall
Hard gates for schema, literals, task leakage, purity, affordances, coverage, secrets, and over-broad claims.
Eval Gauntlet
Split evals, genericity perturbations, adversarial sessions, stateful sandboxes, exact tool-argument scoring, and baseline economics.
Climber
A trace-driven mutation loop that proposes scoped fixes and keeps them only if the regression gate measures improvement.
The important design move is that generated task-specific control code is not trusted. Structural behavior lives in a shared kernel and declarative spec. Generated surfaces are constrained to data and typed judgment slots, with the wall and gauntlet acting as enforcement.
The Pipeline
A useful harness starts with a concrete task specification instead of an architecture diagram. The repo workflow turns that specification into a candidate runtime, then tries hard to break it before making any claim.
Source
Task specification: SOPs, tools, examples, rubrics, counterexamples, fixtures, and representation limits.
Compiler Backend
A headless coding agent drafts the HarnessSpec, prompt slots, and pure functions from the packet.
Type System
The validator wall rejects example memorization, undeclared affordances, impure functions, kernel/task boundary leaks, and over-claims.
Target VM
The policy-free kernel runs the spec: state, transitions, gates, tools, model calls, costs, latency, and trace emission.
Test Suite
The gauntlet scores splits, generated perturbations, adversarial sessions, stateful sandboxes, exact tool arguments, and general-agent baselines.
Optimizer
The climber reads failing traces, proposes one scoped mutation, and keeps it only if no split regresses.
Artifact
A frozen harness with a claim, manifest, traces, costs, and known residuals that stay visible rather than being swept away.
Evidence So Far
The first numbers are local dev-split comparisons. They use the same task packets, fixture tools, rubrics, and strict output metrics for the compiled harness and a headless general-agent baseline. The pattern is the one the project is trying to prove out: lower runtime cost with higher exact-contract performance on narrow tasks.
| Task | Harness | Harness Cost | General Agent | Agent Cost | Runtime Advantage |
|---|---|---|---|---|---|
| dangerous_goods | 6/6 | $0.0000 | 4/6 | $0.0778 | Deterministic / no model cost |
| fnol_auto_insurance | 3/3 | $0.0067 | 0/3 | $0.1366 | ~20.4x cheaper |
| eva_airline_rebook | 3/3 | $0.0108 | 0/3 | $0.0982 | ~9.1x cheaper |
| eva_medical_hr_fmla | 3/3 | $0.0113 | 1/3 | $0.1185 | ~10.5x cheaper |
Read these as local harness evidence, not a universal benchmark. The baseline is intentionally strict: if a general agent misses the exact state, tool argument, gate contract, or output normalization, it fails. That exactness is part of the product claim, because enterprise workflows care about the contract as much as the prose.
The repo also preserves failures. A simulator-found confirmation hole in the airline harness remains visible as a counterexample. The climber can already fix single-artifact clusters and reject regressions, but coordinated spec-plus-prompt mutations are still the main research gap.
Where It Fits
harnessc is aimed at tasks with a recognizable skeleton and messy language at the edges. It is a good fit when the organization has an SOP, policy, rubric, or allowed tool sequence; the input is still natural language; some actions are irreversible or audit-sensitive; and the desired output contract is strict.
Open-ended research, one-off strategy work, and unstable processes still belong with general-purpose agents. Repeated procedural work deserves a narrower runtime. The point is to stop paying for generality when the task does not need it.
- Use general agents when the problem is exploratory, broad, or genuinely novel.
- Use model routing when the task is broad but has separable subtasks with different model needs.
- Use compiled harnesses when the task family repeats, has policy gates, and rewards exact contracts.
Try It
The fastest way to understand the repo is to read the README, run the existing checks, then inspect one task packet and one generated harness side by side. The core question to bring is practical: what repetitive task in your world deserves a frozen runtime instead of a fresh general-agent plan every time?
Compile Your Specialized Harness
Bring a real task specification: SOPs, tools, examples, rubrics, and counterexamples. HarnessC is an attempt to turn that into a runtime that is cheaper, narrower, and better at the job.