Jev Harness / Agent experiments
One task. Two tool menus.
Compare Codex with and without Jev routing. Keep the evidence and watch performance over time.
Read src/sum.ts in the synthetic workspace. Explain what sum([2, 3]) returns, citing the file.
Ready to compare.
The full tool catalog
Codex gets every fixture tool.
A selected tool menu
Jev routes first. The host keeps the reading tools needed for a grounded proposal.
Run an example to compare the answers, calls, input tokens and timing. Each result is saved locally for your next visit.
Saved in this browser
Performance over time
Read a tiny module · 0 saved runs
Your next run starts the timeline
Run the same example again to compare both agents over time. Results stay here after refresh.
From demo to your codebase
Give your coding agent a clear brief.
Keep your runtime. Add Jev where the host chooses which tools enter context, then measure the result.
This changes the brief only. It does not change Arena settings or start a provider request.
Keep improving
A repeatable loop, with evidence.
Freeze the baseline
Keep the task, dataset and host/model settings fixed. Baseline makes no Jev call; shadow measures selection without reducing the menu.
Check the answers
Inspect source evidence and required tool coverage. Record Meets task or Needs work for each lane; missing review remains unreviewed.
Compare repeated runs
Count CLI and Jev input, output, cache reuse and time. Keep failures visible. Compare matching setups and check variation, not just the fastest run.
Change one thing
Use a lesson to form a hypothesis. Version the descriptor, catalog or policy change; evaluate held-out tasks, then keep or revert it.
Start with the working offline adapter
The repository includes a typed example using the same public APIs your host will call.
pnpm exec tsx examples/integration/host.tsUse createCatalog and prepareToolContext; inject your server-side ToolRouter. Keep permissions, argument validation, freshness checks and execution in your host. Cancellation discards the active context in both modes.
Inspect the adapter example ↗ · Official Choice wire contract ↗
Independent community source, not a published SDK or production runtime. The proposal validator/runner extraction and context scoring are separate work; this guide uses the implemented routing boundary.