Skip to content

Ticket 083: Calibration Profiles and Parameter Fitting

Goal

Tune model parameters from observed flights without rewriting core formulas.

Current Gap

There is no calibration data format and no fitter for turning observed flights into updated profile parameters.

Scope

  • Add calibration profile data format separate from base vehicle profile.
  • Fit a narrow first parameter set:
  • cruise speed
  • climb rate
  • descent rate
  • station-keep wind authority
  • phase energy coefficients when observed power is available
  • Record for each fitted parameter:
  • fitted value
  • confidence range or spread
  • sample count
  • calibration dataset version
  • applicable conditions
  • Keep calibration artifacts versioned and reproducible.

Integration Requirements

  • Calibration profiles must layer on top of existing vehicle YAML rather than replacing vehicle profiles.
  • Calibration artifacts should be selectable by estimate, scenario, API, and validation workflows through explicit configuration.
  • Add examples that pair base vehicle YAML, calibration artifacts, mission YAML, terrain, wind, and validation data.
  • Keep fitted values traceable to source logs, normalized traces, validation reports, and tool versions.
  • Preserve deterministic behavior for a fixed calibration artifact.

Acceptance Criteria

  • A calibration dataset can produce a versioned calibration artifact without changing core estimator logic.
  • Fitted parameters are separated from raw manufacturer/default profile values.
  • Calibrated runs compose with existing mission, vehicle, terrain, wind, geofence, landing-zone, energy, and scenario behavior.

Out of Scope

  • Online auto-tuning.
  • Black-box model replacement.

Implementation

Status: implemented

New files

File Purpose
schemas/calibration.py Calibration artifact models (calibration-profile.v2) plus the non-serializable VerifiedCalibration runtime proof
adapters/calibration/fitter.py fit_calibration_profile, CalibrationInput, and the canonical fitted-parameter digest
adapters/calibration/apply.py Verified operational apply and the separate untrusted validation-candidate apply seam
adapters/calibration/io.py Profile I/O, source-trace verification, deterministic refitting, and verified application
adapters/calibration/__init__.py Public package
adapters/calibration_markdown.py render_calibration_markdown — Markdown report renderer
adapters/commands/calibrate.py calibrate CLI command
examples/calibration/quadplane_v1_calibration.json Deterministically generated example artifact
examples/calibration/README.md Example pairing base vehicle + trace + artifact
tests/test_calibration.py Unit, adversarial, determinism, I/O, and CLI coverage

adapters/cli.py registers the calibrate command. estimate and scenario accept --calibration PATH only with one repeatable --calibration-traces TRACE per source trace. validate --calibration deliberately treats the profile as an untrusted candidate and never grants operational calibration status.

Calibration artifact

A calibration-profile.v2 artifact layers on a base vehicle: it carries base_vehicle_id (never a full vehicle profile) and a list of fitted parameter records. Each record holds the fitted value, the observed sample range (confidence_low/confidence_high), the population spread, sample_count, the deterministic calibration_dataset_version, the applicable_conditions envelope, and a human-readable derivation. Provenance carries tool_version, the sorted source_trace_ids, and any linked validation_report_ids. All models use extra="forbid".

The fitting approach

fit_calibration_profile is pure and deterministic — the same base vehicle and the same ordered trace/segmentation pairs always produce byte-identical canonical JSON. It reuses Ticket 081 segmentation as the phase bridge and touches no core estimator formula:

  • cruise_speed_mps — mean groundspeed over transit-phase trace records.
  • climb_rate_mps / descent_rate_mps — mean vertical rate over records whose finite-difference rate clears the segmenter's climb/descent threshold; descent stored as a positive magnitude.
  • max_station_keep_wind_mps — maximum wind speed observed while holding position during loiter_dwell segments (the demonstrated authority).

Parameters with no supporting samples are reported in notes, never fabricated. Phase records with measured power also fit the supported energy coefficients.

The apply seam

Operational application first reloads and segments every supplied normalized trace, recomputes the dataset and source IDs, reruns the deterministic fitter, and compares the canonical digest of all fitted records. Only that verifier can mint the runtime-only proof accepted by apply_calibration; serialized log_calibrated metadata is a claim, not proof that the check ran. The applied vehicle is re-validated, and a mismatched base_vehicle_id or broken vehicle invariant is invalid input. apply_calibration_candidate exists separately for held-out validation and never grants operational trust.

CLI usage

# Fit a calibration profile from a base vehicle + one or more traces
bvlos-sim calibrate VEHICLE.yaml TRACE.json [TRACE2.json ...]          # Markdown
bvlos-sim calibrate VEHICLE.yaml TRACE.json --format json -o cal.json  # envelope

# Reproduce the fit, then run with operational calibration credit
bvlos-sim estimate MISSION.yaml VEHICLE.yaml \
  --calibration cal.json --calibration-traces TRACE.json
bvlos-sim scenario SCENARIO.yaml \
  --calibration cal.json --calibration-traces TRACE.json

# Compare an untrusted candidate against an independent held-out trace
bvlos-sim validate MISSION.yaml VEHICLE.yaml TRACE.json --calibration cal.json

Error handling mirrors validate: InputLoadError → exit 11, output-write failure → exit 13.

Out of scope (kept for later)

Online auto-tuning and black-box model replacement are not part of this ticket. Held-out validation reporting is Ticket 084. Trace hashes establish integrity and reproducibility; they do not prove the normalized trace describes a flight that occurred, nor verify its asserted raw-log digest against raw bytes.