# FluxGit Git context token benchmark

Public, path-sanitized artifacts for the FluxGit `repo.brief` benchmark reproduced on 2026-09-25.

## Verified headline result

On the deterministic 44-commit, 40-file synthetic repository:

- 10-command CLI orientation sweep: 6,861 bytes, 1,790 `o200k_base` tokens, 10 calls.
- `repo.brief`: 1,918 payload bytes, 561 `o200k_base` tokens, 1 call.
- Hand-tuned minimal CLI subset: 1,892 bytes, 492 tokens, 5 calls.

The wide-sweep comparison is 3.2x fewer output tokens and one call instead of ten. The minimal CLI subset is the counterexample: it uses 14% fewer output tokens than `repo.brief`.

## Reproduce from a FluxGit source checkout

The directory is placed five levels below the repository root so `run-benchmark.sh` can find `app/core/Cargo.toml` and build the sidecar.

```sh
cd product/web/research/git-context-token-benchmark/benchmark
./run-benchmark.sh
shasum -a 256 -c SHA256SUMS
```

Set `SKIP_SIDECAR_BUILD=1` only when deliberately reusing an existing release binary. Set `SIDECAR_BIN` if that binary is at a different path.

## Files

- `results.json`: byte, token and call counts.
- `manifest.json`: source revision, environment and repository invariants.
- `SHA256SUMS`: checksums for every captured CLI/MCP output plus the results and manifest.
- `run-benchmark.sh`: end-to-end runner and invariant checks.
- `make-repo.sh`: deterministic synthetic repository builder.
- `mcp-drive.py`: MCP stdio JSON-RPC driver.
- `count.mjs`, `package.json`, `package-lock.json`: pinned `o200k_base` counting step.
- `cli-outputs/`: normalized Git command output.
- `mcp-outputs/`: normalized raw MCP responses.

All published captures replace the absolute benchmark directory with the literal `$BENCH_ROOT`. The runner also fails if it finds the benchmark root or home directory in a captured artifact.

## Measurement boundary

CLI counts use normalized stdout. MCP counts use the normalized `content[0].text` payload, excluding the JSON-RPC envelope. The manifest preserves raw wire byte sizes separately. Requests, model reasoning, latency and the cost of re-reading context are outside the measurement.