Skip to content

Contract Evaluation

Practical guide to turning on contract-aware gating. For the mental model (what a "contract" is, the three modes, why this can't hide a break), see Contract-Aware Compatibility. For the exact field vocabulary and precedence, see Compatibility Evaluation Config.

Turning it on

abicheck compare old.so new.so -H include/ --contract-evaluation

With no --contract, the domain follows the legacy --scope-public-headers/--no-scope-public-headers flag (default: public headers). To select a domain explicitly:

abicheck compare old.so new.so -H include/ \
  --contract-evaluation --contract public   # or: exports | all

--contract requires --contract-evaluation — passing it alone is a usage error (exit 64), since without the flag nothing consumes the selected domain.

scan --against accepts the identical pair of flags, for the same reason:

abicheck scan build/libfoo.so --against baseline.json \
  --contract-evaluation --contract exports

Reading the result

abicheck compare old.so new.so -H include/ \
  --contract-evaluation --format json -o report.json

Two things to check in the JSON:

  • Per finding: contract_relevance, compatibility_evaluation_status, compatibility_decision (null for anything NOT_EVALUATED).
  • Run level: contract_coverage_failures (a list, empty when evidence was complete) and contract_coverage_exit_contribution (0/1).

Markdown/text output includes a "Not evaluated (contract)" count in the headline summary, and SARIF annotates an excluded finding with a contractRelevance property (severity note, not error) rather than dropping it.

Accepting incomplete evidence deliberately

If a domain's evidence is known to be incomplete for some legs of your matrix (e.g. one lane never gets header input), accept that explicitly rather than leaving CI red or turning the feature off:

# packs/accept-unresolved.yml
id: accept_unresolved
version: 1
kind: contract
assignments:
  contract.unresolved: warn

abicheck compare old.so new.so -H include/ \
  --contract-evaluation --contract exports \
  --pack packs/accept-unresolved.yml

This zeroes the contract-coverage exit contribution only — the failures stay listed in contract_coverage_failures, and it changes nothing about per-finding compatibility decisions. contract.unresolved requires --contract-evaluation; setting it in a pack without the flag is a usage error naming the field and the reason.

Consumer/entrypoint evidence outranks header/export inference

compare --used-by APP or --required-symbol(s) layered on top of --contract-evaluation promotes a finding to IN_CONTRACT whenever it matches the app's actual imports or the plugin host's required entrypoints — stronger evidence than anything a header/export scan alone can infer, per ADR-049 §4.3:

abicheck compare old.so new.so --used-by ./myapp \
  --contract-evaluation --contract public

This promotion only ever raises a finding toward IN_CONTRACT — it never demotes one, and it recomputes the scoped verdict/gate afterward if it changed anything. See Application Compatibility → Why does this consumer depend on the changed declaration? for the deeper "why", not just "whether."

CI recipe: gate only the declared contract, don't hide the rest

- name: ABI contract gate
  run: |
    abicheck compare baseline.json build/libfoo.so \
      -H include/ \
      --contract-evaluation --contract public \
      --pack packs/accept-unresolved.yml \
      --severity-preset default \
      --format json -o report.json

Read report.json's verdict for the gated compatibility result and contract_coverage_exit_contribution separately if your pipeline needs to distinguish "a real break" from "we couldn't prove enough" in its own messaging — the process exit code already folds both, but a human-facing summary usually wants to say which one fired.

Common mistakes

  • Expecting --contract alone to do anything. It needs --contract-evaluation.
  • Suppressing to fix a coverage gap. --suppress cannot reach a CoverageFailure — give the evaluator the missing evidence (headers, build info) or accept the gap explicitly with contract.unresolved: warn.
  • Reading compatibility_decision: null as "compatible." It means policy never scored the finding — check contract_relevance for why.
  • Assuming exports mode never benefits from headers. It does — roots come from the export table alone, but the closure that resolves from those roots still needs declaration data. See Contract-Aware Compatibility → Three contract modes for the full explanation.

See also