ADR-065: Comparison Scope, Member Selection, and Input Completeness¶

Amendment (2026-09-06, ADR-068). This ADR's model is depended on, not changed: unselected, expected but not produced, failed and deliberately retired stay four states, a partial local build still never reads as a removal, and a run that completed zero comparisons still never reads as a pass. Two additions follow from retiring scan:

One new acquisition state, declared_absent — the OLD side of an abicheck compare --no-baseline NEW run, which replaces scan's audit-only mode. It is explicitly not not_supplied: the user declared there is no prior surface, so the evolution axis is reported not_evaluated and contributes no compatibility verdict, rather than the scope being reported incomplete.

scan --artifact-set's multi-library members become this ADR's members. ADR-056 is superseded, so its parallel set model does not survive alongside the acquisition and selection model here; S3's package component inventories are the prerequisite for that fold.

Sequenced as Phase 1 of plans/one-comparison-product.md.

Date: 2026-09-05 Status: Proposed — S1, S2, S3 and S4 implemented.

S2 (2026-09-05): the acquisition record (abicheck/model/scope_acquisition.py), the completeness axis on RunOutcome/ExitDecision (abicheck/policy/outcome.py, abicheck/policy/exit_decision.py, abicheck/policy/scope_completeness.py), the release fan-out's per-member record and no comparison completed outcome (abicheck/workflows/release_scope.py, abicheck/cli_compare_release.py), the degraded stranded-library marker (BundleFacts.degraded_members), and the comparison_scope report section (abicheck/report/comparison_scope.py), with --on-incomplete-scope warn|block as the D6 setting; exit 8 now requires D2's completeness proof (migration note in docs/reference/exit-codes.md). S1 also implemented (2026-09-06, landed after S2): abicheck/model/release_selection.py's ReleaseSelection, --select/--select-required on compare's directory/package fan-out, and the --dry-run "Comparison plan" preview (abicheck/workflows/release_plan.py, abicheck/frontends/cli/release_dry_run.py) — identity/coordinate member selection with a plan preview, through the typed API and CLI. bundle_variants_config.py was deleted outright in this slice rather than given a consumer (see the plan's own A1.6 note).

S3 (2026-09-06): package component inventories and support-promise findings. abicheck/model/package_inventory.py is the declared component set with its own completeness flag; abicheck/package.py's package_component_inventory builds one from an archive the extractor unpacked in full (ExtractResult.container_complete), which makes a live package operand a D2 completeness proof for the first time — a directory operand still proves nothing. EXPECTED_NOT_PRODUCED, reserved by S2 and unproduced until now, gets its producer (a declared component the tree cannot reach). abicheck/policy/support_promise.py is D1's fifth concept under the --support-promise off|declared contract-policy field, emitting support_promise_component_retired/_introduced from proven inventory changes only; abicheck/workflows/release_support_promise.py wires it into the release fan-out's own results.

S4 (2026-09-06): the deletion gate. _match_release_keys's old-minus-new / new-minus-old set difference is gone (abicheck/cli_compare_release_helpers.py) — its last two consumers, the JSON unmatched_old/unmatched_new keys and the fan-out's stderr notices, read the acquisition record instead (abicheck/report/comparison_scope.py's release_scope_warnings). The Action/aggregate parity this slice also names landed with S2 (the SCOPE_INCOMPLETE verdict tier in action/run.sh, aggregate report schema 1.8's scope_completeness axis). Scalar-versus-bundle operand convergence stays with plans/cli-cleanup-phase-two.md's open PR I rather than being done twice — see that plan's own row. The silent canonical fallback in compare_product_directories (D3's ambiguous diagnostic) is also left for a later slice, since turning it into a refusal-to-compare is a behaviour change with its own migration note.

S0 (executable scenario table) remains open. Design record for the vision's "partial matrices" and "scope-sensitive analysis" decisions (vision.md). Implementation is sequenced in plans/vision-api-abi-evolution.md (workstream "Comparison scope and completeness"), which also carries the existing-versus-missing assessment this ADR's Context summarizes. Decision maker: maintainer (product decision recorded in vision.md); technical sign-off pending review of this document.

Context¶

abicheck already compares one artifact against another, a release directory against another, a package against another, a candidate against a multi-profile baseline set, and folds a CI matrix into one gate. Each of those paths grew its own answer to three questions this ADR separates:

What is the analysis boundary? One artifact, a package's component set, a declared release matrix.
Which members were expected, which were actually supplied, and which were compared? The difference between unselected, expected but not produced, failed, out of scope, and deliberately retired.
Did the project's support promise change? A retired platform or a component confirmed absent from a complete inventory is a contract change; an artifact a partial run never supplied is not.

The current code answers these unevenly (file references are from the 2026-09-05 assessment; see the plan for the full table):

The release fan-out pairs libraries by filename stem and defines removed = old_keys - new_keys (abicheck/cli_compare_release_helpers.py, _match_release_keys), then reports that set under the JSON key unmatched_old and feeds it to --fail-on-removed-library's exit 8. A name-normalization miss, a SONAME bump, a failed extraction, and a genuine deletion are one state. (Closed: S2 took exit 8, the verdict bump and the Markdown/PR-comment sections off it; S4 deleted the set difference itself, so the JSON key and the stderr notices read the acquisition record.)
compare_product_directories (abicheck/product_baseline.py) has richer pairing (exact path, then an ambiguity-guarded SONAME/case-folded fallback), but an ambiguous group is silently left unpaired and then surfaces as BUNDLE_LIBRARY_REMOVED plus BUNDLE_LIBRARY_ADDED — an ambiguity reported as a removal.
A release run with zero matched pairs appends a warning and keeps worst_verdict = "NO_CHANGE", exit 0 (cli_compare_release_pairwise.py).
Multibuild variant pairing is exact-fingerprint-only and deliberately never a union (abicheck/bundle_multibuild.py, G38 Phase 3), with same-side collisions detected — but every capture path stamps the default fingerprint, bundle_variants: (abicheck/bundle_variants_config.py) has no production caller, and a declared-but-never-captured required variant is invisible.
The aggregate workflow is the one place the model this ADR wants already exists end to end: ExpectedTargets with required/optional members, synthesized TargetReports for expected-but-missing cells, OnMissingRequired/OnUnexpectedTarget gates, and a finding_matrix with an explicit undetermined third state (abicheck/workflows/aggregate/). It is aggregate-only; no per-library or per-variant scope consumes it.
The Action's baseline resolution has a typed outcome vocabulary (resolved / not_found / ambiguous / wrong_profile / stale_schema / incompatible_evidence / wrong_project_ref / stale_generation / new_target, abicheck/buildsource/baseline_set.py) that stops at the Action boundary: a new_target or dry-run "baseline unavailable" run is a green check with no comparison performed.
No typed request (CompareRequest, BundleCompareRequest, ScanRequest, AnalysisPlan) carries an expected inventory or a member/variant selection; no ExitReason/RunOutcome axis expresses "the inventory I was asked to cover was not fully covered".
Package extraction (abicheck/package.py) returns directories, never a declared component inventory, so a component the extractor failed to unpack and a component the package no longer ships are indistinguishable. (Closed by S3: package_component_inventory returns a declared component set whose complete flag is the extractor's own "this container was unpacked in full" statement, and a declared component the tree cannot reach is EXPECTED_NOT_PRODUCED rather than absent.)
A stranded old-side library in the release fan-out degrades to an ELF-only AbiSnapshot written into the baseline with only a stderr line (cli_compare_release.py, _resolve_stranded_library) — a degraded capture persisted as if complete.

The consequence the vision names directly: a developer who builds one Linux/GCC variant locally and compares it against a twelve-variant baseline can be told eleven platforms were removed; a CI matrix whose macOS job failed can be read as a retired macOS promise; and a release whose one library failed to extract can pass.

Decision¶

D1 — Four concepts, four representations¶

The following are distinct and are never collapsed onto one field:

Concept	Meaning	Where it lives
Analysis boundary	What the user asked to analyze: one artifact, a package/component set, or a declared release matrix	The typed request (existing `CompareRequest`/bundle/scan request types, extended — never a new request family)
Selection	Which members/variants the user or the resolved plan chose for this run, and any explicitly authorized cross-profile pairing	The resolved plan (`AnalysisPlan`/run plan), with the selector and its provenance recorded
Expected inventory	Which members this run should have produced/consumed, with provenance (a resolved project plan, a package's own manifest, a trusted complete inventory)	The resolved plan; absent for a bare artifact comparison
Acquisition state	Per expected member: `available`, `expected_not_produced`, `failed`, `not_supplied`, `unsupported`, `out_of_scope`	The typed result, per member, separate from any policy verdict

A support-promise change (a retired platform, a component confirmed absent from a complete inventory) is a fifth thing: a contract change with evidence, emitted as a finding under a configurable policy, never inferred from acquisition state alone.

D2 — Unmatched is not removed¶

A member present on one side and absent on the other is unmatched with a recorded reason. It becomes a removal finding only when the selected domain's completeness is proven on the side that lacks it: the new side's inventory is complete for the boundary (a full package inventory, a matrix whose expected members all resolved, or an explicit user statement that the new side is complete). The rule is symmetric: a new-only member becomes an addition finding only when the old side's inventory is proven complete, since a partial old input cannot prove the member was absent from the old release. out_of_scope is reserved for members the run did not select: in a partial or selected run an unselected baseline member is out_of_scope and contributes nothing to the verdict, whereas a selected, expected member whose counterpart was not produced or whose extraction failed keeps its expected_not_produced or failed acquisition state and flows into D6's incompleteness outcome or D8's operational error — it is never demoted to out_of_scope, and never promoted to a removal. The existing BUNDLE_LIBRARY_REMOVED/_ADDED kinds and --fail-on-removed-library keep their meaning and become consumers of the proven-complete state of the relevant side rather than of a raw set difference.

Clarification (2026-09): completeness evidence is a concrete receipt, not an implicit claim. A component-removal finding produced under D2's completeness rule should carry the receipt that justified it, not merely a boolean "completeness was proven" — otherwise a reviewer (or a later re-evaluation, ADR-066) has no way to check what established completeness without re-deriving it from the raw inputs. The receipt names, per finding: the side the completeness claim is about (old or new — D2's rule is symmetric, but each finding's own completeness evidence is one-sided); the selected boundary the claim covers (the analysis boundary from D1 — completeness for "this package" is not completeness for "this release matrix" containing it); the inventory source (a package manifest, a resolved project plan's expected-member list, or an explicit user completeness assertion — D1's three provenance kinds); the inventory identity (a stable identifier/digest for the specific inventory document consulted, so two runs claiming completeness from two different, silently inconsistent manifests are distinguishable); the acquisition coverage (which members that inventory named were actually resolved to an available acquisition state, versus expected_not_produced/failed — completeness is never proven while a named member's own acquisition status is unknown); and, when applicable, the explicit user completeness assertion itself (a caller can assert "this side's package is complete" without a machine-readable manifest — D1's third inventory-source kind — and that assertion is itself part of the receipt, not indistinguishable from a manifest-derived proof). No schema is fixed by this clarification; the shape above states what a future implementation's per-finding completeness data must be able to answer, not a JSON field list.

D3 — Ambiguity is a diagnostic, never a guess¶

When a candidate could pair with more than one baseline member (or vice versa), the run reports an actionable ambiguous selection diagnostic naming the candidates and the coordinates that would disambiguate, and does not compare. Neither "first", nor "latest", nor an arbitrary canonical fallback is chosen silently. This generalizes the aggregate/baseline-set ambiguous outcomes and the multibuild same-side-collision error to every pairing site, including compare_product_directories' canonical fallback.

D4 — Pairing is by identity and coordinates¶

Members pair on a stable target identity plus variant coordinates (the existing VariantRef declared/captured coordinates, variant_fingerprint, profile fingerprint, and the baseline-set channel × target × profile tuple), not on filenames alone. Filename-based matching remains a last-resort tier that is reported as such. A cross-profile pairing (old GCC, new Clang; old x86-64, new AArch64) is allowed only when explicitly requested, and is then analyzed dimension by dimension per the comparability contract (ADR-050, extended by the evidence-adequacy workstream) — never merged into one profile to manufacture coverage.

D5 — Many baseline members, one candidate selects¶

"Many baseline members, one candidate" means select the matching member. It is not a Cartesian product and not a union. An explicitly requested check of one candidate against several supported baselines (a support window) is a different task that produces independently attributed comparisons, each with its own selection record, never merged evidence.

D6 — Completeness is a run outcome with a default of warn¶

Every selected, expected member that did not reach a completed comparison is an incompleteness signal on the run, whatever the reason: expected_not_produced, not_supplied, unsupported (an artifact this build cannot analyze), failed (which is also an operational error, D8), or an ambiguous selection (D3). The signal is a warning by default, configurable to block through the existing outcome/exit machinery (a new axis on RunOutcome/ExitDecision beside compatibility, assurance, operational, and coverage — ADR-064's precedence, extended, not a second gate scheme), and it is raised independently of how many other selected members compared cleanly: a matrix with one clean pair and one unsupported or ambiguous selected member is an incompletely checked scope, never a clean pass. It never becomes an ABI finding.

The two settings are defined exactly like ADR-049 Phase 7's coverage axis (contract_coverage_exit.py), so there is one fold shape, not two:

Setting	`RunOutcome`	`ExitDecision` contribution	Exit status	Report
`warn` (default)	The scope axis reads `incomplete` (members and reasons listed); the outcome is not "scope fully checked"	`0`	Unchanged: `0` when the compared members are clean, the gate's own `2`/`4` otherwise	The compact and full views state the unchecked members and that the scope was incompletely checked; the top-level decision wording never says "compatible" for the whole scope, only for the compared members
`block`	Same axis value	`1`, folded with `max`	A clean `0` becomes `1`; a `2`/`4` is never lowered	Same wording plus the reason the gate failed

warn therefore permits a successful exit for the compared members while the outcome and the report remain honest about the scope; it never silently upgrades an incomplete scope into a clean one, and the axis is persisted in the typed result so a downstream consumer can gate on it even when the process exit was 0. A retired support promise is configured separately (a contract-policy field), so a project can say "macOS is no longer supported" and have that evaluated as a contract change, while a missing macOS job stays an incomplete run.

D7 — Zero completed comparisons is never success¶

A run whose selected scope produced no valid comparison reports no comparison completed as its operational outcome, whatever the completeness policy says. A permissive completeness setting can downgrade missing members to a warning; it cannot turn nothing compared into "compatibility checked". This applies to the release fan-out, the Action's new_target/dry-run baseline-unavailable paths, and any future selected comparison alike.

Clarification (2026-09): "no comparison completed" is scoped to the requested operation, not to "zero binary-pair diffs ran" specifically — a documentation review found the acceptance-test wording below ("a run with zero valid comparisons reports no comparison completed") reads, out of context, as if it could apply to a case D2 already resolves the other way. Consider: an old package has three libraries, the new package intentionally has zero — every one of the three is unmatched, and D2's symmetric rule resolves each to a genuine removal finding once the new side's inventory is proven complete (an empty package is a completely-inventoried package, same as a full one). Zero binary pairs ran, but a real, evidence-backed package-inventory comparison completed and established three removals — that is success at the requested question ("did this package's component set stay the same"), never "no comparison completed". D7's own trigger condition is narrower and orthogonal to this: it fires when the run could not establish any answer at all for the requested scope — no complete inventory on either side to reason from, every selected member ambiguous or failed, or (the dry-run/baseline-unavailable case) no baseline to compare against in the first place. State each of the three outcomes' reporting independently, since none is a variant of another:

Successful planning — a --dry-run plan view enumerated the expected set. It never claims compatibility, and it never reports no comparison completed either, because no comparison was attempted at all — D7's trigger condition presupposes an attempted comparison, and a pure planning view is not one.
Baseline initialization — a new_target, first-ever-baseline run. This does report no comparison completed, exactly as D7's base text already requires (unchanged by this clarification): it is an honest, expected first-run state, not a failure, but it is not exempted from that label — there is genuinely no prior baseline to compare against.
Successful compatibility analysis with zero binary-level diffs — the three-removals case above, or a matrix where every selected member matched with no findings. This is a real, complete answer, so it must not report no comparison completed.

The acceptance test above should be read with this distinction already applied: "zero valid comparisons" there means zero established answers of any kind for the requested scope, not merely zero binary pairs — it does not relax D7's existing new_target/dry-run reporting requirement in any way.

D8 — Failed extraction is a failed member, persisted as such¶

A member whose extraction failed is failed in the acquisition record and an operational error in the outcome. It is never written into a baseline or BundleFacts document as a degraded snapshot without an in-band marker; the storage document carries the member's status (storage v2's explicit fact availability, ADR-062) so the next run reads missing data, not an impoverished old side.

D9 — Scope inference is conservative¶

The OLD input being a project snapshot or baseline set does not by itself imply a full-release task. The run infers a narrow current-artifact task when the selection is unambiguous (one candidate, one matching member) and requires a minimal selector when it is not. Every run exposes a plan view (the existing --dry-run shape, ADR-043/054) listing each expected member, its match or exclusion, and the reason.

Consequences¶

Scalar comparisons are untouched: with no inventory and no selector, the expected set is the one pair, acquisition is available, and no new field is populated. The one-member package path and the scalar path must produce the same applicable findings (an executable invariant).
The release fan-out's unmatched_old JSON key becomes what its name says; the removal finding and exit 8 require D2's completeness proof. This is a behavior change for a partial release directory that today exits 8; it is a correction, sequenced with a migration note.
bundle_variants:'s required: field, currently unread in production, gets its consumer through D6, or is deleted — the plan decides which, and the deletion gate is explicit.
Report projections (ADR-036/061) gain a scope/selection section and a per-member acquisition table; the compact and one-line views must carry the incompleteness and no-comparison notices (the reporting workstream's rule).
The Action's typed baseline outcomes propagate to the run outcome instead of ending at the composite step.

Relationship to existing decisions¶

Extends, does not replace: ADR-002/006/023 (release/package/bundle comparison), ADR-047/054 (project lifecycle and project plan), ADR-050 (comparability), ADR-055 (typed requests), ADR-062 (storage v2's fact availability), ADR-063 (AnalysisPlan, RunOutcome), ADR-064 (exit precedence). G30/G34/G38 own the profile/variant machinery this reuses. Nothing here revives the retired baseline registry (ADR-022/043 D4) or a settable exit-code scheme (ADR-064).

Implementation slices¶

See the plan for status. S0: this ADR plus an executable scenario table. S1 (landed): identity-and-coordinates member selection with a plan preview, through the typed API and CLI. S2 (landed): expected/observed inventory, acquisition states, and the completeness axis on RunOutcome/ExitDecision. S3 (landed): package component inventories and support-promise findings under contract policy. S4 (landed): Action/project/aggregate parity and deletion of the replaced set-difference path; scalar-versus-bundle operand convergence stays a slice of plans/cli-cleanup-phase-two.md's PR I, and compare_product_directories' canonical fallback (D3) is not yet an ambiguous diagnostic — both are recorded as open in the plan rather than claimed here.

Acceptance tests (contract)¶

Exact member partitions per run (available, expected_not_produced, failed, not_supplied, unsupported, out_of_scope) are pairwise disjoint and sum to the expected set.
One candidate against a twelve-variant baseline yields one comparison and eleven out_of_scope members, zero removals.
Adding an unrelated baseline variant cannot change a selected comparison's findings; input order cannot change pairing.
Two plausible baseline members produce an ambiguous diagnostic and no comparison.
A complete old/new package with a library absent from the new inventory yields a component-removal finding with inventory evidence, without a sibling consumer.
A partial old input against a complete new package yields no addition finding for a new-only member (old-side completeness unproven); the same pair with a complete old inventory does.
A declared matrix with a failed expected member yields incompleteness (warn by default, block when configured) and no invented API deletion.
A mixed matrix — one selected member compared cleanly, another selected member unsupported (or ambiguously matched) — reports the clean pair's findings and an incomplete-scope outcome. Under warn the exit status is 0, the RunOutcome scope axis reads incomplete, and every view names the unchecked member; under block the same run exits 1, and a run whose compared members already exit 4 still exits 4 under either setting.
A run with zero valid comparisons reports no comparison completed under every completeness policy.
Replacing unavailable evidence with empty evidence fails a test.
Binary/binary, binary/snapshot, snapshot/binary, snapshot/snapshot, and the package counterparts all go through the public paths (CLI, typed API, Action) and agree.