Skip to main content

Measure and retrospect

autodev derives metrics from planning Git history and fleet events. It does not maintain a hand-edited metrics store. Missing evidence remains unknown, and aggregates include their denominator.

Planning flow and forecast​

autodev metrics stories
autodev metrics stories --epic upload-safety
autodev metrics epics
autodev metrics forecast
  • stories derives time in each state from commits that touched each story file.
  • epics aggregates throughput, median cycle time, and open versus total stories.
  • forecast estimates backlog drain time from measured delivery rate. With insufficient history, it reports unknown rather than a zero-duration forecast.

Uncommitted status edits are reported as a difference between working-tree state and the last committed observation. Keep planning history committed when lifecycle timing matters.

Fleet performance and proof​

autodev metrics waves
autodev metrics fleet
autodev metrics orientation
autodev metrics predictions
  • waves reports phase durations, attempts, and park reasons from events.
  • fleet reports tick cadence, active-versus-quiet ratio, and evidence/judge pass rates.
  • orientation counts tool calls before a worker's first write, grouped by recorded harness and model.
  • predictions scores predicted filesets against delivered write-sets over time.

Prediction data may be absent when no analyst role is configured. That absence does not mean predictions were perfect or zero-cost.

Usage and cost​

autodev metrics usage
autodev metrics usage --by story
autodev metrics usage --by wave

The default grouping is model. Token usage reported by a harness is recorded per model; an unreported value remains unknown. Cost uses a harness receipt when one exists and otherwise estimates from the built-in price table plus repository overrides.

Override a model price only with rates you have verified against your billing source:

.autodev/fleet.toml
[pricing."my-model"]
input_per_mtok = 3.0
output_per_mtok = 15.0
cache_read_per_mtok = 0.3
cache_creation_per_mtok = 3.75

A repository entry replaces the built-in entry for that model as a whole. Omitted rates remain unpriced, not zero. Reports label table-derived cost as estimated.

Mine friction with retro​

autodev retro
autodev retro --output json

retro reads stored transcripts, turn receipts, and events and emits typed friction findings. It is read-only and may be run while the fleet operates. Findings do not make the command exit 1; the report is diagnostic, not a gate.

Use retrospectives to identify repeated failure classes, then inspect the relevant event and transcript evidence before changing configuration or filing corrective work. A finding is a signal for interpretation, not an automatic root-cause verdict.

Automate without losing meaning​

Every metrics command accepts --root and --output json. Preserve these distinctions in dashboards:

  • unknown versus observed zero;
  • estimated cost versus harness-reported cost;
  • the numerator and denominator of every rate;
  • a filtered epic report versus the whole board;
  • current working-tree status versus committed planning history.

Flattening those fields into one number removes the provenance the reports are designed to retain.