ADR-038: Working With Sources — Full-Scan and Two Build-Injection Flows, and the Clang Plugin Specification¶
Date: 2026-07-01
Status: Accepted — implemented. Formalizes and extends ADR-035 D5 (Flow 1 /
Flow 2) into a complete, three-flow producer contract and pins the Clang-plugin
specification. Full source scan (no injection) and Wrapper injection (the
abicheck-cc wrapper) already ship; Plugin injection (the Clang plugin,
contrib/abicheck-clang-plugin/) now implements this spec —
functions/mangled-name rule/signatures/default args,
typedefs, constexpr, records/enums/templates including the AST subtree hashes
(type_hash/body_hash), macros (in-compile PPCallbacks), and visibility. The
subtree hashes are produced by serializing the in-memory AST with clang's own
JSON dumper (Decl::dump(…, ADOF_JSON)) and porting clang.py's
canonicalization onto it, so parity holds by construction for a given clang
version. The C.6 differential-conformance gate runs green in CI across LLVM/Clang
16, 17, and 18 (.github/workflows/clang-plugin.yml) — the plugin's public
surface is entity-equivalent to the clang backend on each. The one documented
residual is a floating-point literal's textual value inside a hashed subtree, plus
the pragmatic visibility-classifier edge cases (C.7).
Decision maker: Nikolay Petrov (@napetrov)
Context¶
Every source-aware check in abicheck rests on one expensive step: turning a
library's C/C++ headers and sources into abicheck's normalized source-ABI
model. Concretely that means running a C++ front end (castxml or
clang -ast-dump=json) over translation units and folding the result into the
L4/L5 evidence layers:
- L0/L1/L2 — artifact-authoritative binary / debug-info / header-AST scan
(
dumper.py,elf|pe|macho_metadata.py,dwarf_*.py). Always the source of truth for a shipped-ABI verdict. - L3 — build/toolchain context from a compile DB, CMake, Ninja, Bazel, or
Make (
buildsource/adapters/*,build_evidence.py). - L4 — scoped per-TU source-ABI replay: parse each TU under its real
build flags → a normalized
SourceAbiTu(source_extractors/*,source_replay.py,source_abi.py). - L5 — the source/implementation graph folded from L3+L4
(
source_graph.py).
L4 is the cost centre. For template-heavy C++ a single TU's clang JSON AST can
be multi-GiB — the reason source_replay.py carries RAM-aware worker caps,
AST-spill-to-tempfile, and an opt-in process pool. The strategic lever is where
that parse happens and who pays for it.
ADR-035 D5 named two answers — Flow 1 (abicheck runs the replay) and Flow 2
(the build emits normalized facts) — and shipped the abicheck_inputs/ artifact
protocol plus the abicheck-cc wrapper. It left two things underspecified:
- The "work with sources" story is not written down end-to-end. A user has to reverse-engineer, from flags and module docs, that there is a full-scan path and two injection paths, that they interoperate, and how to move between them.
- The plugin has no build-to specification. ADR-035 D5 calls it a "performance optimization" but never states what it must emit, how its records must match the wrapper's byte-for-byte, how it is built/versioned, or how it is validated.
This ADR closes both: it documents the three flows as one interchangeable family and gives the plugin a complete, testable contract.
Decision¶
Support three flows for producing source evidence, all converging on one
normalized SourceAbiTu contract and one merge/compare consumer. They differ
only in where the parse happens and how much the product build must change.
| Flow | Name | Build change | Extra parse | Who parses | Consumer path | Status |
|---|---|---|---|---|---|---|
| A | Full source scan (dump --sources / collect) |
None | 1, post-build | abicheck | inline, or collect → dump --build-info |
✅ Shipped (Flow 1) |
| B | Wrapper injection (abicheck-cc) |
Set CC/CXX |
1 companion, in-build | The build (as a companion action) | merge an abicheck_inputs/ pack |
✅ Shipped (Flow 2) |
| C | Plugin injection (-fplugin) |
Add a clang flag | 0 — rides the compile's AST | The compile itself | merge an abicheck_inputs/ pack |
⚙️ Reference — implemented (incl. subtree hashes); C.6 gate in CI matrix |
D0 — the shared-contract invariant. A comparison is always old-vs-new
produced by the same producer, and every producer is deterministic and
diff-stable (same TU → same records). Producers are not required to be
byte-identical to each other: the castxml and clang backends already hash the
same declaration differently — castxml builds a ret(params) signature string
(base.py), clang hashes its type.qualType and uses an alpha-normalized AST
subtree hash for types (clang.py). Cross-producer equivalence is required only
between the plugin and the specific backend it substitutes: because the
plugin reads the clang AST, its reference is the clang extractor (clang.py),
and the C.6 conformance gate compares the plugin against a clang-backed wrapper
(not castxml). The linker folds by SourceEntity.identity() — the mangled name,
else qualified_name#signature_hash — so mangled decls fold consistently even
across producers, while unmangled-decl hashes are guaranteed stable only within
one producer. The binary dump (L0–L2) stays artifact-authoritative for
shipped-ABI verdicts; source evidence only explains, localizes, scopes, or adds
source-level (API_BREAK/RISK) findings and never deletes an artifact-proven
break (ADR-028 D3 authority rule). This is what makes the three flows a
migration path, not a lock-in: a project adopts Full source scan with zero
build changes and moves to Wrapper injection or Plugin injection only when
parse cost demands it — with no change to the compare side. The one operational rule this implies: produce the old and
new baselines of a comparison the same way — the only sanctioned cross-producer
pair is the plugin and the clang backend it is conformance-tested against (C.6),
which agree on the whole surface including macros; an arbitrary producer mix
(e.g. castxml vs clang) is reliable only on the mangled surface.
The shared contract: SourceAbiTu and SourceEntity¶
All three flows emit the same abicheck-owned normalized schema
(buildsource/source_abi.py, SOURCE_ABI_VERSION). Raw front-end output
(castxml XML, clang AST JSON, Android .lsdump) is provenance only and is never
compared (ADR-028 D4).
SourceAbiTu — one per translation unit:
| Field | Meaning |
|---|---|
schema_version |
SOURCE_ABI_VERSION |
tu_id |
cu://<source>#cfg:<hash> — stable per-TU id |
target_id |
target://<library> |
extractor |
{"name", "version"} producer id |
compile_context_hash |
sha256: over standard/triple/sysroot/defines/includes (D8 cache key) |
source |
the TU source path |
public_header_roots |
configured public-header roots (ADR-015) |
functions/types/variables/macros/templates/inline_bodies/constexpr_values/declarations |
SourceEntity[] buckets |
source_edges |
optional intra-TU decl edges (→ L5) |
read_files |
every file the parse actually read (cache invalidation) |
diagnostics |
non-fatal producer notes |
SourceEntity — one per public declaration:
| Field | Meaning |
|---|---|
id |
content hash; the primary key within a TU |
kind |
function/record/enum/typedef/union/variable/macro/template/inline/constexpr |
qualified_name |
fully-qualified source name |
mangled_name |
C++ ABI symbol, or "" when indistinct (see the mangled-name rule) |
signature_hash |
type-level signature (params/return + cv/ref) — stable across default-arg edits |
body_hash |
inline/template body fingerprint |
type_hash |
record/enum/typedef structural hash |
value |
macro/constexpr value, or the function's default-argument string |
source_location |
{path, line, origin} where origin ∈ PUBLIC_HEADER/PRIVATE_HEADER/SYSTEM_HEADER/GENERATED/SOURCE/UNKNOWN |
visibility |
public_header/private_header/system_header/generated/unknown |
api_relevant |
on the callable public surface? |
confidence |
LayerConfidence |
SourceEntity.identity() — the key the linker/diff fold on — is the
mangled_name when present, else qualified_name#signature_hash, else the bare
qualified_name. Because folding is by identity(), entity ordering within a
TU does not affect the verdict; the contract is per-entity equality, not raw
file-byte equality.
Full source scan (no injection)¶
The straight way: change nothing in the build. abicheck reads the build's
existing compile_commands.json (or infers one) post-build and replays the
in-scope TUs itself. Nothing is injected into the compile; the build is untouched
and unaware. This is the default and the recommended starting point.
A1 — Inline (dump --sources)¶
One command materializes a baseline with L3/L4/L5 folded in:
# Binary L0–L2 from the .so + L3/L4/L5 replayed from ./src, in-process.
# No wrapper, no plugin, no build edit. (L4 needs public-header roots — see the
# note below; the inline path takes them from config, not a dump flag.)
abicheck dump libfoo.so --sources ./src -o libfoo.baseline.json
abicheck compare libfoo.old.baseline.json libfoo.new.baseline.json
Compile-DB resolution is zero-config (buildsource/inline.py +
build_query.py, ADR-032 amended): explicit --build-info → a trusted
--build-query command → a build.compile_db glob → an auto-discovered
compile_commands.json → an inferred, abicheck-authored build-system query
(cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON / bazel aquery / make -B -n -k -w).
So --sources ./src alone yields L3 with no flag and no manual compile-DB step.
An arbitrary build.query from an auto-discovered (untrusted) .abicheck.yml
is never auto-run — it needs an explicit --config.
Public-header roots are a separate input from the compile DB — and the inline
dump --sources path takes them from config, not a CLI flag. L4 provenance is
opt-in (provenance.classify_origin): with no public-header set, every
declaration classifies UNKNOWN, link_source_abi drops it, and the L4 surface
is empty even though the TUs parsed. A plain compile_commands.json carries no
public-header metadata, and dump's --public-header/--public-header-dir flags
feed only the L2 header-AST provenance — embed_build_source does not forward
them to the inline L4 collection. So give the inline L4 surface its roots via
.abicheck.yml sources.public_headers or a CMake File API build dir (whose
fileSets populate target.public_headers). For roots specified on the command
line, use the A2 collect -H/--header path (which does feed L4).
A2 — Split producer/consumer (collect → dump --build-info)¶
To parse on a build host and compare elsewhere, materialize a BuildSourcePack
with collect, then attach it to the binary dump with dump --build-info:
# build host: L3 build evidence + L4 source-ABI replay → an evidence pack.
# --binary relinks the L4 surface against the library's exports (the source-decl
# ↔ binary-symbol map); --header gives the public-header roots that classify
# which decls are on the public surface — without them the extractor marks decls
# UNKNOWN and the linker drops them. dump --build-info only embeds the
# pre-captured pack; it does not relink.
abicheck collect --binary libfoo.so --header include/ --compile-db build/compile_commands.json --source-abi -o libfoo.evidence/
# analysis host: attach the pre-captured pack to the binary dump (no re-parse)
abicheck dump libfoo.so --build-info libfoo.evidence/ -o libfoo.baseline.json
collect produces a BuildSourcePack consumed via dump --build-info/--sources
— not an abicheck_inputs/ pack via merge (that protocol belongs to
Wrapper injection and Plugin injection).
Scope, cost, and when to use¶
- Backend: on inline
dump --sourcesthe knob is--ast-frontend auto|clang|castxml(ADR-037 D8; the same dial drives the L2 header AST);collectspells it--source-abi-extractor auto|clang|castxml|android.clangadds inline/template/constexpr body fingerprints + default args;castxmlgives declarations/types/const values only. Oncollecta requestedclangnot on PATH falls back to castxml; the inlinedump --sourcespath instead disables source-only checks when the selected frontend is unavailable (it records "source-only checks disabled" rather than switching backends), so on a clang-less host pass--ast-frontend castxmlexplicitly. - Scope: on inline
dump --sourcesuse the--depth/--maxdial (ADR-037 D5;binary|headers|build|source|full) to bound how deep to collect. The fine-grained replay scopes (--source-abi-scope off|headers-only|changed| target|full+--changed-path) arecollect-only; inlinedump --sourceshas no--changed-pathand defaults to a broadersource-targetcollection. Changed-only replay is therefore acollect --source-abi-scope changed --changed-path …(orscan --since) capability — reach for it on PR jobs; the inlinedumpdefault is not changed-scoped. - Cost: one parse per in-scope TU, paid by abicheck, not the build. The
measured cost cliff is at L4 for template-heavy C++ (
scan_levels.pycost model); scope + the per-TU content-addressed cache (ABICHECK_L4_CACHE_DIR) are the levers that keep it bounded. - Use it when: you can't or won't alter the build; open-source consumers; first adoption. Requires a compatible front end on the analysis host.
Wrapper injection (abicheck-cc)¶
Prefix the real compiler with abicheck-cc (abicheck/cc_wrapper.py). The
wrapper runs the real compile pass-through (preserving its exit code), then
best-effort extracts one SourceAbiTu per source TU using that TU's exact
flags/macros and appends it to an abicheck_inputs/ pack.
export ABICHECK_INPUTS_DIR=abicheck_inputs
export ABICHECK_CC_HEADERS=include # public-header roots
export ABICHECK_CC_LIBRARY=libfoo
make CC='abicheck-cc gcc' CXX='abicheck-cc g++' # CMake: -DCMAKE_CXX_COMPILER_LAUNCHER=abicheck-cc
# merge folds pre-existing .abi.json dumps + packs; dump the binary side first.
abicheck dump libfoo.so -o libfoo.so.json
abicheck merge libfoo.so.json ./abicheck_inputs/ -o libfoo.baseline.json
- Best-effort authority (ADR-028 D3): extraction is skipped on a failed
compile and any extraction error is downgraded to a warning — it never fails
the build. A preprocess-/dependency-only invocation (
-E,-M/-MM,/E /P /EP) is detected and skipped so no non-shipping TU pollutes the pack. A multi-source compile (g++ -c a.cpp b.cpp) contributes both objects' facts, per-TU isolated. - Exact per-TU context: flags, macros, includes, sysroot, and target triple
are captured from the real argv, so
compile_context_hashmatches the build. - Cost: one companion parse per TU, inside the build. More than Plugin injection, but no version-pinned artifact and it wraps any compiler.
- Config (argv-transparent, all env):
ABICHECK_INPUTS_DIR,ABICHECK_CC_EXTRACTOR,ABICHECK_CC_HEADERS,ABICHECK_CC_LIBRARY,ABICHECK_CC_VERSION,ABICHECK_CC_DISABLE. - Use it when: you control the build invocation, want exact-build-context
facts, and can't/won't pin a Clang-plugin
.soto your toolchain.
Plugin injection: the specification¶
Load an abicheck Clang plugin during the normal compile
(contrib/abicheck-clang-plugin/). It emits the same source_facts from the
AST Clang already built for the real compile — zero extra front-end pass.
This is the fastest producer and the strategic answer for large/template-heavy
builds, because the fact stream falls out of a compile the project was already
running.
clang++ -std=c++17 -Iinclude \
-fplugin=./libabicheck-facts.so \
-Xclang -plugin-arg-abicheck-facts -Xclang out=abicheck_inputs \
-Xclang -plugin-arg-abicheck-facts -Xclang public-roots=include \
-c src/foo.cpp -o foo.o
# real compile only; abicheck_inputs/source_facts/<tu>.jsonl appended
# dump the binary side first, then fold the emitted facts in (no re-parse).
abicheck dump libfoo.so -o libfoo.so.json
abicheck merge libfoo.so.json ./abicheck_inputs/ -o libfoo.baseline.json
Two argument details matter and are part of the spec:
- Use the
-Xclang -plugin-arg-<name> -Xclang <arg>form, not the-fplugin-arg-<name>-<arg>shorthand. The shorthand mis-parses a hyphenated plugin name: clang splits-fplugin-arg-abicheck-facts-out=…at the first hyphen and delivers it to a plugin namedabicheck(verify withclang++ -###), soabicheck-facts'sParseArgsnever seesout=. The-Xclangcc1 form is unambiguous. (Alternatively, register the action under a hyphen-free name so the shorthand works — but the reference skeleton usesabicheck-facts.) public-roots=is mandatory — it is the plugin's equivalent of the wrapper'sABICHECK_CC_HEADERS. L4 provenance is opt-in (provenance.classify_origin): with no public-header roots every declaration classifies non-public andlink_source_abidrops it, so the plugin would emit an empty public surface. Repeatable; may also be sourced from build metadata.
C.1 — Structure and lifecycle¶
The plugin is a clang::PluginASTAction registered as abicheck-facts with
getActionType() == AddAfterMainAction — it runs after the real codegen
action, so it never perturbs the object output. ParseArgs reads out=<dir>
(default abicheck_inputs) and the repeatable public-roots=<path> (see the
invocation note above for the correct -Xclang -plugin-arg-abicheck-facts form).
To capture macros without a second parse (C.2), it registers PPCallbacks on the
CompilerInstance's Preprocessor in CreateASTConsumer/ParseArgs.
HandleTranslationUnit walks the TU with a RecursiveASTVisitor, buffers one
SourceEntity per public declaration, wraps them in a SourceAbiTu envelope,
and appends one JSON object per line to a per-TU file.
C.2 — What it MUST emit (record-equivalence with the clang backend)¶
The plugin reads the clang AST, so its reference is
buildsource/source_extractors/clang.py (source_abi_from_clang_ast) — not
base.py, which is the castxml recipe. The hashing recipe is fixed and part of
the contract:
- Content hash:
_hash(*parts) = "sha256:" + hex(sha256(parts joined by "\x00"))— same construction asbase.py, but the parts below are the clang recipe. Any deviation changesidentity()/*_hashand fails the C.6 gate.
| Kind | id parts |
key hashes / fields |
|---|---|---|
| function | "function", mangled_or_name, sig |
sig = type.qualType (clang's printed function type); signature_hash = _hash("sig", sig); value = _default_arg_repr → p<pos>=<literal-or-subtree_hash> per defaulted param |
| inline body | "inline", mangled_or_name, sig |
emitted when the function has a CompoundStmt body; body_hash = subtree_hash(body, param_ids) |
| record / enum | "type", qualified_name |
type_hash = subtree_hash(node); definitions only (skip forward decls) |
| typedef / alias | "typedef", qualified_name, underlying |
underlying = type.qualType; type_hash = _hash("typedef-target", underlying); value = underlying |
| constexpr var | "constexpr", qualified_name, value |
value = lone-literal value, else subtree_hash(init) |
| template | "template", qualified_name |
body_hash = subtree_hash(node); do not descend into the templated pattern |
| macro | "macro", name, value |
captured in-compile via PPCallbacks (MacroDefined/MacroUndefined) — never a second -E -dD pass (C.3); public-header macros only, include guards dropped, value normalized to match clang.py::macros_from_preprocessor |
subtree_hashwas the hard part and is now implemented:clang.pyhashes an alpha-renamed, commutative-operator-normalized, build-root-stripped canonical form of clang's JSON AST (_canonical/_alpha_rename_map/_subtree_hash). Rather than hand-reproduce clang's JSON, the plugin serializes the relevant subtree with clang's own JSON dumper in-process —Decl::dump(os, false, ADOF_JSON), the exact code path-ast-dump=jsonuses — and ports_alpha_rename_map/_canonical/_subtree_hash(plus_expr_value/_default_arg_repr) onto that JSON. Because the wrapper's clang backend consumes the same clang JSON, the hashes match by construction for a given clang version; cross-version drift is caught by the C.6 CI matrix (C.6). No second parse is added — the dump reads the AST clang already built.- Mangled-name rule: take clang's
mangledName; if it equals the plainname(e.g. some constructors), leavemangled_nameempty soidentity()falls back toqualified_name#signature_hashand keeps unmangled overloads distinct. Copying the bare name verbatim would collapseWidget(int)andWidget(double). - Visibility / api_relevant: classify each decl's declaring file (via
SourceManager, threading clang's stickyloc.file) intoPUBLIC_HEADER/PRIVATE_HEADER/SYSTEM_HEADER/GENERATED, mirroringclang.py::_ClassifyContext. Only public-surface decls are emitted; a private/protected member of a public class is dropped (its whole subtree stays non-public). Public-header roots come from the plugin arg / build. - Determinism: the same TU compiled twice must yield identical records
(
clang.pysorts macros; AST-order for the rest). Folding is order-independent, but determinism keeps the pack diff-stable.
C.3 — What it MUST NOT do¶
- Never fail or slow the real compile abnormally — a fact-emission error is
swallowed (write to
stderrat most), exactly like the wrapper's best-effort rule. A plugin exception must not abort codegen. - Never emit a verdict. It produces evidence, not decisions.
- Never ship raw AST as the comparison format.
raw_ast/is forensic only and is never ingested (ADR-035 D5); the plugin normalizes tosource_factsitself. - No second parse. If a mapping needs data the AST does not cheaply expose, approximate within the visitor — do not re-invoke the front end.
C.4 — Output layout (per-TU, race-free)¶
Append to <out>/source_facts/<stem>.<sha256(source)[:12]>.jsonl, mirroring
inputs_emit.facts_filename(), so parallel -j compiles never race on one file.
The plugin also ensures <out>/manifest.json exists (kind: abicheck_inputs,
created_by: "abicheck-clang-plugin <ver>") — idempotent, atomic write, matching
init_inputs_pack.
C.5 — Build and versioning¶
cmake -S contrib/abicheck-clang-plugin -B build \
-DCMAKE_PREFIX_PATH="$(llvm-config --cmakedir)/.."
cmake --build build # → libabicheck-facts.so
The plugin is a CMake MODULE linked against the loading clang's symbols
(find_package(LLVM/Clang CONFIG), cxx_std_17, no bundled LLVM). It is
therefore ABI-locked to its LLVM major: a plugin built against LLVM N only
loads into that clang. abicheck cannot ship one .so for every LLVM, so the
plugin is not a required gate in the main abicheck CI; a product build has a
pinned toolchain image and builds the plugin once against that image. That
ABI-lock is exactly why validation is per-version: the clang-plugin workflow
(.github/workflows/clang-plugin.yml) builds the plugin against a matrix of
LLVM/Clang majors and runs the C.6 conformance test on each (C.6). It is a
standalone, non-blocking workflow, path-filtered to the plugin and the
clang-recipe modules it mirrors — validation without gating merges, consistent
with "not a required gate".
C.6 — Validation: differential conformance¶
The plugin is correct iff it is a drop-in for the clang backend. The gate
is a differential test (contrib/abicheck-clang-plugin/tests/conformance.py):
compile one fixture TU both ways with the same clang — the plugin (with
public-roots=include), and the wrapper pinned to clang with the same roots
(ABICHECK_CC_EXTRACTOR=clang, not auto, plus ABICHECK_CC_HEADERS=include,
so both sides use the recipe and the public surface the plugin targets) —
ingest both packs, and assert the two surfaces are entity-equivalent: equal
sets keyed by SourceEntity.identity(), with equal
signature_hash/type_hash/body_hash/value/visibility/api_relevant per
entity. Non-macro entities are compared strictly; macro values are compared
leniently (operator-adjacent spacing is the documented soft edge, C.7). The
clang-plugin workflow runs this on a matrix of LLVM/Clang majors (pinning
clang/clang++ on PATH to each matrix version so the plugin and the wrapper's
extractor use the identical clang — the precondition for byte-for-byte parity),
and — because the plugin is a plain LLVM shared module — builds it with both
GCC and Clang as the host compiler (LLVM 18 both ways; 16/17 on the distro
default) to keep it host-toolchain-portable. It runs only where a matching clang
is available and is never a required abicheck-CI gate.
Beyond entity equivalence, each matrix leg also runs an end-to-end scan
validation (tests/scan_flow.py): it compiles the fixture into a shared
library with the plugin active (one build both links the .so and drops
abicheck_inputs/ beside it), then drives the real user pipeline — abicheck
dump the binary (L0/L1), abicheck merge the plugin pack into the baseline
(asserting the L4 source-ABI and L5 graph layers were ingested with a non-empty
entity set, and that the specific DECL_CALLS_DECL edges the fixture's
calls produce are present in the embedded L5 graph — not merely that L4 entity
counts are non-zero), and abicheck compare the merged baseline against itself
(asserting a clean verdict). This proves a plugin-emitted pack is consumable
by the ordinary scan, not merely entity-equivalent to the clang backend.
Side channels beyond entity equivalence. The entity comparison above
proves the L4 entity contract; it says nothing about source_edges,
read_files, fact_set, or per-family coverage — the exact gap a review of
an earlier revision of this ADR's implementation flagged (the "does not cover
the newly mandatory channels" finding). conformance.py now also asserts:
source_edgesparity for a planted overload regression. The fixture (widget.hpp/widget.cpp) declaresoverload(int)/overload(double)and two callers, one per overload — exactly the shape whosereferencedDeclresolution C.6's original scope did not exercise (see C.10). Both producers'(edge, src, dst)triples for these two calls are required to match (an error if either is missing); any other divergence in the edge set (e.g. compiler-implicit builtin-typedef edges the two producers are not guaranteed to walk identically) is reported but not gated — reconciling that is the broader, still-open concern C.10 tracks, not this regression target.read_files— both producers must have read the fixture's own primary source and header (a required subset, not exact equality: the two collection mechanisms are not guaranteed to enumerate transitively-included system headers identically).fact_set— both producers must declare the same canonicalname/version/compiler_family(the fieldscheck_fact_set_compatibility()keys comparison-compatibility on, C.8).coverage— informational: neither producer should reportpartial/failedfor a mandatory family on this clean fixture.
This is still not full parity coverage of every C.6 field (source locations,
names/relations/ownership are not compared), but it closes the specific
gap the review named: the LLVM 16/17/18 green matrix now also exercises the
edge/read-file/fact-set contract the entity comparison alone left unproven.
C.7 — Non-goals / limitations¶
- Compiler coverage: clang only. GCC (
-fdump-lang-class/-fdump-tu) and MSVC remain documented fallbacks via the wrapper; a small normalizer tosource_factsis out of scope here. - AST-subtree hashes:
type_hash(records/enums) andbody_hash(inline/template bodies) are implemented by dumping the subtree with clang's own JSON dumper in-process and portingclang.py's canonicalization (C.2); the C.6 CI matrix proves parity per clang version. The one residual is a floating-point (or fixed-point) literal's textual value inside a hashed subtree: clang's JSON emits an approximate numericvalue(getValueAsApproximateDouble()) whose shortest-round-trip textual form the plugin reproduces only best-effort. This is self-consistent within the producer, so under D0 (both baselines produced the same way) it never yields a false finding; only the cross-producer C.6 gate can surface it. Should a dump fail at runtime, the entity is still emitted without the subtree hash (partial, never wrong) plus a diagnostic. - Macros: macro parity is delivered by in-compile
PPCallbacks(MacroCollectorinAbicheckFactsPlugin.cpp, registered on the preprocessor inCreateASTConsumer/ParseArgs), never a second-E -dDpass — a companion preprocess would reintroduce exactly the extra front-end pass Plugin injection exists to avoid (C.2/C.3). Captured macro values are normalized to matchmacros_from_preprocessorso the plugin stays entity-equivalent to the clang backend it substitutes; the C.6 gate covers macros (leniently — operator-adjacent spacing is the documented soft edge). A project that still hits a macro mismatch runs Full source scan or Wrapper injection for both sides of the comparison rather than mixing producers — the sanctioned plugin↔clang-backend equivalence of D0, not a licence to mix arbitrary producers. - Public-surface classifier (pragmatic): the plugin classifies a decl/macro
as public by matching its declaring file's path segments against the
public-rootsset (with aSourceManager::isInSystemHeaderguard so stdlib headers reached through a coincidental path segment likeincludedo not leak). This is an approximation ofclang.py's include-spelling model (build_public_set/classify_origin), not a byte-port. It agrees with the backend for the common-Iincludelayout (the C.6 gate passes), but two configurations are known to diverge until the full matcher is ported: public headers reached via a system include path (-isystem, CMakeSYSTEM PUBLIC) are dropped by the system-header guard even though they are explicitly public, and exact-file public roots given from a different tree than the compile's are matched only by segment-subsequence. A project hitting either runs Full source scan or Wrapper injection for both sides of the comparison. - Compiler-implicit special members: the plugin does not emit
compiler-implicit (never user-declared) special members — the default/copy/
move constructors, destructor, and assignment operators a class gets for free.
RecursiveASTVisitor::shouldVisitImplicitCode()is left at its default (false), so implicit members are not traversed. The clang backend, walking clang's JSON, does emit those it finds materialized in the TU (e.g. a public API that returns a record by value odr-uses its copy/move ctor), so a header exposing such a record shows a benign cross-producer MISSING on the C.6 gate. Matching it is a deliberate non-goal: it would require visiting all implicit code and then filtering to exactly the set the wrapper's invocation materialized — a set that depends on the capture point (the plugin runs post-codegen,AddAfterMainAction; the wrapper does not), so parity would be fragile rather than exact. Under D0 (same-producer baselines) the implicit surface is identical on both sides, so it never yields a false finding; a project needing implicit-member facts in a cross-producer comparison runs Full source scan or Wrapper injection for both sides.= default-ed members are user-declared (not implicit) and are emitted normally.
C.8 — Canonical fact-set identity and coverage honesty¶
One canonical fact set, explicitly versioned — never a user-selectable
collection mode. The plugin (and the reference clang.py wrapper extractor)
always collects the complete mandatory family list for its declared fact-set
version; there is no --minimal/--types-only/--no-macros/--skip-* flag,
and there will not be one. A build that wants less evidence simply does not
enable the plugin for that target (see "Producer selection" below and the
deployment guidance in the plugin README.md) — the collector itself has one
profile.
Every SourceAbiTu record (and the plugin's manifest.json) carries:
"fact_set": {
"name": "abicheck-clang-canonical",
"version": 1,
"producer": "abicheck-clang-plugin",
"producer_version": "0.5",
"compiler_family": "clang",
"compiler_version": "18.1.3"
}
SOURCE_ABI_FACT_SET_NAME/SOURCE_ABI_FACT_SET_VERSION
(buildsource/source_abi.py) are the single source of truth both producers
stamp from (default_fact_set()); the plugin's kFactSetName/
kFactSetVersion C++ constants are a literal mirror, kept in sync by comment.
fact_set.version describes the semantic contract (the mandatory family
list) — it is bumped only when that list changes, never for a
performance/producer change (those bump producer_version instead).
Each TU also carries per-family coverage — complete /
empty-confirmed / partial / unsupported / failed
(buildsource.source_abi.COVERAGE_STATES), derived by
coverage_state_for_family()'s pure decision table so every producer reports
it the same way:
"coverage": {
"functions": "complete", "variables": "complete", "types": "complete",
"macros": "complete", "templates": "complete", "inline_bodies": "complete",
"constexpr_values": "complete",
"source_edges": "complete", "read_files": "complete"
}
The plugin derives this from its existing per-declaration "JSON dump
failed" diagnostics (no new state threading through the visitor): a family
with such a diagnostic and at least one collected entity is partial; with
the diagnostic and zero entities, failed; otherwise complete (entities
present) or empty-confirmed (none — collection still ran). source_edges
(DECL_CALLS_DECL/DECL_REFERENCES_DECL/DECL_HAS_TYPE/
TYPE_HAS_FIELD_TYPE/TYPE_INHERITS) and read_files are populated by both
producers during the same AST walk/compile the rest of the record comes from
(recommendation P1 #15-18) — the plugin via a small CallRefVisitor sub-walk
per function body plus SourceManager::fileinfo_begin()/end(), the clang.py
wrapper via clang_source_edges.build_source_edges() reusing
call_graph.py's/type_graph.py's existing pure AST parsers on the JSON AST
it already parsed. Edge identity is (kind, src, dst), deduplicated per TU;
each producer's edges are self-consistent. C.6 now compares a targeted,
planted-regression subset of source_edges (an overload-call fixture) rather
than the full set byte-for-byte — see C.6 and C.10 for why full parity is not
yet asserted end-to-end, and C.10 for where these edges go once collected
(folded into the L5 graph, not merely serialized).
buildsource/fact_set.py implements the comparison-compatibility rules over
these fields: rollup_fact_set()/rollup_coverage() fold per-TU records up
to the linked SourceAbiSurface.coverage["fact_set"]/["fact_family_states"]
(source_link.link_source_abi, worst-coverage-wins per family), and
check_fact_set_compatibility() flags a fact_set.version mismatch (error)
or a compiler_family/producer mismatch (warning — opaque body/template
hashes are producer-specific, C.7). source_diff.diff_source_abi calls this
via _diff_fact_coverage(), emitting SOURCE_FACT_COVERAGE_INCOMPLETE
(RISK) when there is something to report — an incompatible fact-set pairing,
or a mandatory family rolled up partial/failed on either side — so an
absent L4 finding for that family is never silently read as "unchanged".
Silent (as before C.8) when neither side has ever populated this metadata,
so existing baselines and hand-built fixtures are unaffected.
abicheck inputs validate <pack> (buildsource/inputs_validate.py) runs the
same checks before an authoritative merge: manifest validity, fact-set
name and version, duplicate TU identities, incomplete mandatory-family
coverage, and empty-public-surface detection — 0 clean / 1 warnings / 2
errors / 64 not a readable pack, so a CI evidence-production job can fail
closed on a mis-collected pack instead of a much-later confusing missing
finding. check_fact_set_compatibility() (buildsource/fact_set.py) checks
name too: two surfaces whose fact_set.version numbers happen to match but
whose name differs are a different canonical contract, not a compatible
pair — flagged the same as a version mismatch (error), not silently passed.
C.9 — Post-build pack ergonomics (recommendation P1 #21-25)¶
A large parallel build emits one source_facts/*.jsonl file per TU (C.4's
race-free-by-construction naming) — correct during the build, but expensive
to transfer/store afterwards for a pack with thousands of TUs. Three
independent, composable, opt-in post-build knobs address this without
touching what gets collected (still C.8's "one canonical fact set, never a
narrower mode") or what a consumer decodes:
- Compaction (
inputs_emit.compact_inputs_pack(), #21) merges every discoveredsource_facts/file into one, re-serializing each TU through the same canonicaljson.dumps(tu.to_dict(), sort_keys=True)form the writers already use. A post-build step only — never run mid-build, since a still-writing parallel compile could race the merge.remove_originals(defaultTrue) deletes the per-TU files once the merge is durable, so a later ingest cannot double-count TUs from both the merged file and its stale sources. - Compression (
inputs_emit.append_source_facts(..., compress=True)/write_inputs_pack(..., compress=True)/compact_inputs_pack(..., compress=True), #22) gzips the facts file (stdlibgzip— no new dependency).inputs_pack.read_source_facts()decompresses.jsonl.gz/.json.gztransparently, so an ingester never needs to know a pack was compressed. - Profiling to a separate channel (
ABICHECK_PLUGIN_PROFILE_LOG=<path>, #24) redirects the plugin's per-TUABICHECK_PLUGIN_PROFILE=1summary line from stderr (the default — unreadable once many parallel compiles interleave, and often swallowed by a build system that only surfaces stderr on failure) to an append-only log file, one process-wide sink shared across the parallel TUs the same way the facts pack itself is.
Execution-policy invariance (#25): none of the three change the decoded
facts a consumer sees — only where/how many bytes sit on disk before ingest,
or where telemetry that isn't a fact at all gets written. Enforced by:
test_inputs_emit.py's compaction/compression tests decode the same TU set
before and after (test_compact_merges_per_tu_files_and_removes_originals
compares the linked surface's declaration names pre/post-compaction), and
test_clang_plugin_profiling.py statically guards that emitProfileLine()
(the profiling sink) never references the facts-output stream and that the
facts file write always precedes the profiling emission textually. The
existing per-TU replay cache (source_replay.SourceAbiCache,
ABICHECK_L4_CACHE_DIR, D8) was already covered by this invariance —
caching serializes/deserializes the same SourceAbiTu records verbatim.
C.10 — source_edges reach the L5 graph, not just the pack¶
C.8 (#15-18) made both producers collect source_edges during their
existing frontend pass. A subsequent review of an earlier revision of this
ADR's implementation found the gap that collection left open: SourceAbiTu.
source_edges was serialized and round-tripped, but SourceAbiSurface — the
linked, per-library object build_source_graph() actually reads — had no
edge field at all, so nothing ever folded them into the L5 graph. The
separate call_graph/type_graph Clang AST replay passes remained the only
producer of DECL_CALLS_DECL/DECL_REFERENCES_DECL/DECL_HAS_TYPE/
TYPE_HAS_FIELD_TYPE/TYPE_INHERITS edges end-to-end — a second, separate
frontend invocation over the same TUs, exactly the cost this feature exists
to remove. This is now closed:
SourceAbiSurface.source_edges(source_abi.py) is a new field in the same{edge, src, dst, provenance, confidence, attrs}shape asSourceAbiTu.source_edges(which itself mirrorsGraphEdge.to_dict()).source_link.link_source_abi()folds every linked TU'ssource_edgesinto it, deduplicating by(edge, src, dst)across TUs that share a public header — the same dedup precedent already applied to entities.source_graph.fold_source_edges()consumesSourceAbiSurface.source_edgesfrom insidebuild_source_graph()(always, not behind a flag — the graph is already built whenever a source surface exists). Endpoint identities are mapped onto the identicaldecl:///type://node-id schemecall_graph.augment_graph_with_calls()/type_graph.augment_graph_with_types()already use for the replay path, so an edge collected this way lands on the same graph node a separate replay pass would have created —add_edge's(src, dst, kind)de-dup key reconciles the two for free, first-writer-wins.- The separate replay passes are still always run — not yet skipped. An
earlier revision of this fix skipped
fold_call_graph()/fold_type_graph()once the rolled-upsource_edgesfact family wascomplete/empty-confirmedfor a full (unnarrowed) L4 replay scope, on the theory that their edges were already present viafold_source_edges. A review of that revision found the flaw: the rawsource_edgeswire format ({edge, src, dst, provenance, confidence, attrs}) carries only bare endpoint identities, never thedst_file/project-file provenancefold_call_graph/fold_type_graphattach viaproject_files(GraphNode.attrs["defined_in_project"]).source_graph. is_internal_dependency_node()(shared withcrosscheck.py'spublic_to_internal_dependency) requires that provenance — or the L4 surface's owndecl_to_filemapping, which only covers public reachable declarations — to classify an unannotated node as project-internal. A private helper/type only ever reached viasource_edges(never itself part of the public surface) would therefore stay unclassified, silently suppressing a realPUBLIC_API_INTERNAL_DEPENDENCY_ADDED/crosscheck finding. The skip is reverted: both replay passes run unconditionally wheneverwith_call_graphis set, exactly as before this ADR's edges-fold work;fold_source_edgesstill runs (always, viabuild_source_graph()) and reconciles with the replay's edges via the same(src, dst, kind)de-dup key, so it remains a net addition (it can supply edges when a narrower/cheaper collection ran without a full replay, e.g. a Plugin injection-only pack with noclang++on the merge host) — it is just not (yet) a substitute for the replay's provenance. Makingsource_edgesitself carry equivalentdst_file/caller_fileprovenance end-to-end (in both the plugin's wire format andclang_source_edges.build_source_edges()) is the natural follow-up that would let this optimization return safely; tracked as future work, not bundled into this fix. - The Flow-2 ingestion path (
inputs_pack.ingest_inputs_pack) has no replay fallback at all, so the provenance gap above is not just an optimization risk there — it is the only edge source. A follow-up review found that a Flow-2-ingested pack's graph never markedextractor_passesforcall_graph/type_graphat all (fold_source_edgesnever touches it), sosource_graph_findings._common_dependency_edge_kinds()treated a genuinely complete-but-zero-edge family as "never examined" and suppressed a real finding as a coverage artifact.source_graph. mark_source_edges_extractor_coverage()closes that half — applied only atingest_inputs_packand the export-relink graph rebuild incli_buildsource_merge.py(the two call sites that never run a replay afterward;inline._build_inline_graph/cli_buildsource_helpersalready track this precisely via the replay itself and must not call it, since a baresource_edgesrollup can't tell full-scope from narrowed). But this only fixes the pass-coverage gate (_dependency_kinds_covered) — it does not fix the separate node-classification gateis_internal_dependency_node()still applies per-node: a Flow-2-ingested node fromfold_source_edgesalone still carries nodefined_in_projectattr and, being genuinely private, is never covered by the L4 surface's owndecl_to_file(which only maps public reachable declarations). Both gates must pass forPUBLIC_API_INTERNAL_DEPENDENCY_ADDEDto fire on such a node, so for a Flow-2-only pack (no compile DB / no replay ever possible), a public-API-to-private-helper dependency collected only viasource_edgesstill cannot be detected end-to-end. Closing this fully requires the same producer-side wire-format change described above (both the plugin andclang_source_edges.build_source_edges()emittingdst_file); until then this is a known, accepted limitation of the ingestion-only path specifically (Codex review).
Identity normalization (the second half of the same review finding).
Even with the fold wired up, the Clang JSON-AST replay's callee identity was
unreliable: call_graph._identity() read referencedDecl.get("mangledName")
or .get("name"), but a real Clang 17/18 -ast-dump=json's compact
referencedDecl stub for an overloaded callee never carries mangledName —
verified against a live compile of int f(int)/double f(double) (both
call sites' stubs are {"kind": "FunctionDecl", "name": "f", "type": {...}},
differing only in id and type.qualType). Keying solely off the stub
therefore collapsed every overload (and, by the same mechanism,
distinguishably-named constructors/destructors) onto one bare-name endpoint —
producer-dependent and silently wrong, since the plugin's own callee
resolution (over the live FunctionDecl*) does not share this limitation.
type_graph.py had already solved the identical problem for its own
DECL_REFERENCES_DECL/DECL_HAS_TYPE edges (_resolve_ref_identity(), an
earlier review) but call_graph.py never received the equivalent fix. It now
does: parse_clang_ast_calls() builds an id_index: dict[str, str] from
every full FunctionDecl/CXXMethodDecl/CXXConstructorDecl/
CXXDestructorDecl/CXXConversionDecl node seen during the same walk (keyed
by clang's own per-node id, present even on an otherwise-incomplete
referencedDecl stub), and _resolve_ref_callee_identity() resolves a call
site's stub through that index before falling back to the stub's own
(name-only) identity. Verified end-to-end against a live Clang 18 compile of
the overload(int)/overload(double) pair now planted in the C.6 fixture
(widget.hpp/widget.cpp) — see C.6.
Still open: a forward reference to a declaration this walk has not yet
seen in full (rare — C/C++ requires a visible declaration before a call) and
constructors/destructors reached through a route the id-index does not cover
remain a documented best-effort limitation, matching type_graph.py's own
documented gap in the same spirit. Fully general resolution needs a stable
scope-qualified identity for every declaration, not just ones an id happens
to disambiguate — tracked as future work, not blocking this fix.
C.11 — Gating opaque-hash comparability on producer/version agreement¶
C.8's check_fact_set_compatibility() already reported — via
SOURCE_FACT_COVERAGE_INCOMPLETE — that a producer/producer-version/
compiler-version mismatch makes opaque body/template hashes unreliable. But
source_diff.diff_source_abi() emitted that finding and then unconditionally
ran _diff_inline_bodies()/_diff_templates()'s hash comparisons anyway, so a
comparison could report both "these hashes may be unreliable" and a concrete
INLINE_BODY_CHANGED/TEMPLATE_BODY_CHANGED computed from the very hashes
just flagged as unreliable — the second finding still feeding reports,
policies, and gates.
buildsource/fact_set.py now exposes a structured verdict,
FactCompatibility (check_fact_compatibility()), wrapping
check_fact_set_compatibility()'s issue list into three booleans:
structured_facts_comparable (false only on a fact_set name/version
mismatch — the mandatory-family contract itself), opaque_hashes_comparable,
and source_edges_comparable (both false on a producer/producer_version/
compiler_version mismatch too, since the canonicalization recipe can change
independently of fact_set.version). diff_source_abi() computes this once
per comparison and threads it into _diff_inline_bodies()/_diff_templates(),
which skip their hash-diff branch (not their existence/removal branch —
INLINE_FUNCTION_REMOVED/UNINSTANTIATED_TEMPLATE_REMOVED are not
hash-derived and are never suppressed) when opaque_hashes_comparable is
false. _diff_fact_coverage()'s SOURCE_FACT_COVERAGE_INCOMPLETE finding now
says explicitly, in that case, that the hash findings were suppressed and why.
hash_recipe_id (fact_set.hash_recipe_id()) is the escape hatch the
review asked for: two fact-sets declaring the same explicit
"hash_recipe_id" are treated as opaque-hash-comparable regardless of a
producer/producer_version/compiler_version mismatch — a differential
conformance run (C.6) proving two differently-named producers emit
byte-comparable hashes is more precise than treating any producer-identity
difference as inherently incompatible. Absent an explicit field, the fallback
recipe id is the producer/producer_version/compiler_version triple
check_fact_set_compatibility() already keys its rule on, so old fact-sets
recorded before this field existed keep comparing exactly as before. A
fact_set.name/version mismatch is never overridable this way — that is
the mandatory-family contract itself, not a hashing-recipe detail.
C.12 — Target/pack isolation¶
The plugin's per-TU fact filename was keyed on source path + compile-context
hash only, and ensureManifest()/the wrapper's init_inputs_pack() were
first-writer-wins: an existing manifest.json was loaded and returned
unchanged with no check that the current invocation's library/version
agreed. Two different libraries built from a shared object file
(common.cpp compiled into both libA/libB) into one shared out=
directory would therefore silently collide — same filename, same manifest,
whichever compile ran last winning, with nothing downstream able to tell.
- Filenames and
tu_idnow include the target/library identity. The plugin's per-TU fact filename folds a hash oflibrary + "\x1f" + source(falling back to source-path-only when nolibrary=is configured) instead of the source path alone;tu_idgains a#target:<library>suffix when a library is configured. Theinputs_emit.facts_filename()/cc_wrapper.pypath (the non-plugin producer) gets the identical fix — the same bug existed independently in both producers' filename schemes. - An existing manifest is validated, not silently trusted. The plugin's
ensureManifest()now reads an existingmanifest.jsonand prints a loud (never build-aborting — this runs inside a real compile invocation) stderr warning when itslibrary/versiondisagree with the current invocation's.inputs_emit.init_inputs_pack()(the batch/wrapper path, which is not embedded in an active compile and can afford to be stricter) raisesValueErroron the same disagreement instead — idempotent only for a repeated call naming the same library/version, never a silently different one. abicheck inputs validaterejects pack-level target inconsistency.inputs_validate.pynow errors when a pack's TU records name more than one distinct non-emptytarget_id, or when a TU'starget_iddisagrees withtarget://<manifest.library>— the exact same-source/two-library collision surfaced as a validation error instead of a silent merge-time ambiguity. It also promotes afact_setname/version mismatch within one pack's TU records fromrollup_fact_set()'s existing soft "TUs disagree" warning to an error (routineproducer/producer_versiondrift across TUs stays a warning; only a hard mandatory-family-contract mismatch is promoted).- A related, narrower fix to
compact_inputs_pack()'s rollback. The compactor's "was this output path pre-existing" check used to be a barePath.exists(), which could not distinguish a genuine prior compaction (manifest.last_compacted) from an operator's--output-filenamecoincidentally colliding with an ordinary pre-existing file — silently letting the merge overwrite that unrelated file, and (on a subsequent manifest-write failure) leaving the clobbered replacement in place because the rollback branch treated "pre-existing" as "leave it alone." The check is now tied to the resolved identity ofmanifest.last_compacted; a collision with anything else is rejected outright (aValueError) before any write happens — simpler and safer than trying to back up and restore an arbitrary file's prior content. - Also closed: a public typedef whose JSON-dump-derived underlying type
could not be determined was silently dropped with no diagnostic, and
typescoverage only watched for the unrelated "record/enum type_hash unavailable" diagnostic — so a batch of failed typedefs alongside one successfully-collected record/enum still reportedtypes: complete. The plugin now records a"typedef facts unavailable"diagnostic on this path, folded into the same family coverage check. - Still open, deliberately not addressed here: the standardized
abicheck_inputs/<target>/<configuration>/<architecture>/directory layout the review sketched as the fuller structural fix. The validation above makes a collision loud and rejected rather than silent, which is the correctness-critical half; standardizing the on-disk layout is a larger, purely-organizational change better done once real deployments show what layout convention they actually need, and is left for a follow-up ADR amendment rather than bundled into this correctness fix.
Performance re-measurement — explicitly not done here. A prior review of
the plugin (README.md's published 143-TU/four-core LLVM benchmark) measured
an earlier, narrower collector; the current collector additionally performs
per-function call/reference sub-walks, type-relationship collection,
read-file enumeration, and richer serialization, none of which that
benchmark's 2.39× figure reflects. Re-running it is explicitly out of scope
for this change: it requires an instrumented LLVM build and controlled
wall-clock/RSS measurement this environment cannot perform, and fabricating a
number would be worse than stating the gap plainly. Treat the published
figure as historical evidence for the pruned parser it measured, not a
verified measurement of the current collector until someone re-runs the
same benchmark end-to-end and updates README.md.
Producer selection¶
Can you change the build at all?
└─ No ───────────────────────────────► Full source scan. Default. Zero integration.
└─ Yes, and it's a large template-heavy
build where a companion parse hurts,
and you own the toolchain image ─────► Plugin injection. Zero extra parse.
└─ Yes, otherwise ───────────────────── ► Wrapper injection. Exact context, portable.
Because all three share the D0 contract, this is a spectrum, not a fork: start at
A, graduate to B or C when parse cost bites, and mixed fleets are fine — different
targets in one release may use different flows and still merge into one
baseline.
The shared consumer: abicheck_inputs/ and merge¶
Wrapper injection and Plugin injection drop a self-describing pack next to the binary; Full source scan feeds the same linker in-process. The pack (ADR-035 D5) is:
abicheck_inputs/
manifest.json # kind: abicheck_inputs, library/version, created_by
binary/… headers/… # shipped artifact + public headers (dumped separately, L0–L2)
build/compile_commands.json # optional → L3 build evidence
source_facts/*.jsonl # THE PAYLOAD — normalized SourceAbiTu, one per line → L4/L5
raw_ast/… pp/… deps/… # optional, forensic only, NEVER ingested
abicheck merge libfoo.so.json ./abicheck_inputs/ auto-detects the pack
(is_inputs_pack() → kind: abicheck_inputs) and routes it to
ingest_inputs_pack(): pure parsing, no compiler. It reads
source_facts/*.jsonl → L4 surface (link_source_abi), the optional compile DB
→ L3, folds the L5 graph, and embeds the result. Third-party packs are guarded
(pack-root path constraint, symlink-escape safe, per-record skip-with-diagnostic).
Consequences¶
Positive. - The full end-to-end story of working with sources is documented as one interchangeable family; the cheapest path (Full source scan, no injection) is explicitly first-class. - The plugin has a complete, testable build-to spec — hashing recipe, mangled-name rule, visibility model, output layout, versioning, and a differential conformance gate — instead of an open TODO.
Negative / costs.
- Wrapper injection and Plugin injection add build-time cost and CI wiring on
the product side; Full source scan moves that cost to the analysis host.
There is no free parse — the ADR only
lets a team choose where to pay it.
- The plugin carries real maintenance burden: rebuilt per LLVM major, and its
differential test must track any change to clang.py's field/hash mapping (the
recipe the plugin mirrors — not base.py, which is castxml). It stays
optional for exactly this reason.
No new ChangeKinds, no schema bump. This ADR governs producers; the
SourceAbiTu/abicheck_inputs contract is unchanged (ADR-035 D5), so old
baselines and readers are unaffected.
Relationship to other ADRs¶
- ADR-035 (D5) — introduced Flow 1/Flow 2 and the
abicheck_inputs/protocol; this ADR expands them into the three-flow family and specifies the plugin. - ADR-028 (D3/D6) — authority rule + non-executing ingest; unchanged.
- ADR-030 —
SourceAbiTu/SourceEntityschema;clang.py(source_abi_from_clang_ast) is the recipe the plugin mirrors,base.pythe castxml recipe. - ADR-032 — extractor action/security model; producers of the same normalized
facts, ingested by the non-executing
inputs_packpath. - ADR-033 — replay scopes, per-TU caching, and CI cost model that bound Full source scan's parse cost to changed scope.
- ADR-037 — the
--depth/--ast-frontendCLI dials that drive Full source scan.