ADR-015: Snapshot Serialization and Schema Versioning¶

Date: 2026-03-18 Status: Accepted — implemented Decision maker: Nikolay Petrov

Context¶

abicheck dump produces a JSON snapshot file (.abi.json) that captures the complete ABI surface of a library. These snapshots serve multiple purposes:

Offline comparison: abicheck compare old.abi.json new.abi.json without needing the original binaries or headers
Baseline storage: Check snapshots into version control as ABI baselines
Cross-mode comparison: A DWARF-derived snapshot can be compared against a castxml-derived snapshot (ADR-003)
CI caching: Generate once, compare many times

The snapshot format is a user-facing contract. Changes to the format can break stored baselines and downstream tooling.

Decision¶

1. `AbiSnapshot` as the canonical interchange model¶

All pipeline stages — dumper, checker, reporter — operate on the same AbiSnapshot dataclass. Serialization converts this dataclass to/from JSON.

@dataclass
class AbiSnapshot:
    library: str
    version: str
    functions: list[Function]
    variables: list[Variable]
    types: list[RecordType]
    enums: list[EnumType]
    typedefs: dict[str, str]        # default: {}
    constants: dict[str, str]       # default: {} (populated from header #defines)
    elf: ElfMetadata | None
    pe: PeMetadata | None
    macho: MachoMetadata | None
    dwarf: DwarfMetadata | None
    dwarf_advanced: AdvancedDwarfMetadata | None
    platform: str | None         # "elf" | "pe" | "macho"
    language_profile: str | None # "c" | "cpp" | "sycl"
    elf_only_mode: bool
    dependency_info: DependencyInfo | None

2. Integer schema versioning¶

SCHEMA_VERSION: int = 10

Version history (abicheck/serialization.py; kept current — check that module's header comment before trusting this table, since it is the source of truth):

Version	Change	PR
1	Initial format (no `schema_version` field)	—
2	`schema_version` field added	PR #89
3	`pe` and `macho` metadata fields added (multi-format support)	—
4	Provenance metadata (`git_commit`, `git_tag`, `created_at`, `build_id`)	—
5	`build_mode` capture (compiler/stdlib/std normalization)	—
6	Declaration provenance: `source_header` + `origin` on functions/variables/types/enums (ADR-015 §3a)	—
7	Optional `evidence_pack` reference (ADR-028; lightweight ref to an out-of-band build/source pack)	—
8	Pack-ref key renamed `evidence_pack` → `build_source_pack`, plus an optional inline-embedded `build_source` payload (single-artifact UX). The rename forces a bump: a v7-only reader knows only the old key, so without the bump a v8 snapshot's renamed provenance would be silently dropped instead of triggering the forward-version warning below.	PR #356
9	CastXML field const/volatile/mutable facts (`TypeField.is_const`/`is_volatile`/`is_mutable`) and full CV-qualifier type spelling became reliably populated. A pre-v9 snapshot has real but WRONG data (permanently `False` booleans) rather than a missing key, so `snapshot_from_dict` marks it `header_cv_facts_reliable=False` instead of relying on the bump alone.	PR #582
10	`--ast-frontend hybrid` (G28 Phase 3): `ast_producer` can be `"hybrid"`, and a new `fact_provenance` map records per-fact producer for a snapshot mixing castxml- and clang-backed declarations. A pre-v10 reader's detectors have no concept of per-fact provenance (they gate on whole-snapshot `from_headers` only), so reading a v10 hybrid snapshot with pre-v10 code can misread a producer coverage gap as a real change — the bump at least surfaces the forward-version warning below.	—

Integer versioning was chosen over semver because:

There is no concept of "minor" or "patch" format changes — a version bump either happened or it didn't; there is no partial-compatibility signal to encode
Monotonic integers are simpler to compare (if version < 3: migrate(...))
The bump is a discipline trigger, not a guarantee every change is backward-incompatible in practice. Most additions (v3–v7) are purely additive — old readers ignore an unrecognized field and keep working. A version is bumped whenever an old reader could silently misinterpret the new data, even if the change is additive on the wire — v8's key rename is the clearest example: it bumped precisely so a v7 reader hits the forward-version warning below instead of quietly ignoring renamed provenance it doesn't recognize.

3. Backward compatibility rules¶

Reading old snapshots: Snapshots without a schema_version field are treated as v1. The deserializer handles missing fields by using dataclass defaults (empty lists, None values).

Reading future snapshots: If schema_version > SCHEMA_VERSION, emit a warning suggesting the user upgrade abicheck. The deserializer attempts to read the snapshot anyway — forward compatibility is best-effort.

Writing: Always writes the current SCHEMA_VERSION. There is no option to write in an older format.

3a. Declaration provenance fields (v6)¶

Function, Variable, RecordType, and EnumType each carry two provenance fields:

source_header — the defining header path, derived from the existing source_location with any trailing :line / :line:col stripped. Always populated when a source location is available; it is descriptive metadata.
origin — a ScopeOrigin classification of source_header against the user-provided public-header set. This is the Origin axis of ADR-024's two-axis Linkage × Origin surface model:

Value	Meaning
`public_header`	Header matches a `--public-header` / `--public-header-dir` input
`private_header`	A project header outside the public set
`system_header`	A toolchain/system header (`/usr/include`, MSVC, Xcode SDK, …)
`generated`	A machine-generated header (`moc_`, `.pb.h`, `generated/`, …)
`export_only`	Exported by the binary but absent from any header (no provenance)
`unknown`	No public set was provided, or no source location was available

Classification is opt-in (decision D4): without --public-header / --public-header-dir, every origin is unknown and downstream behaviour is unchanged. Matching is done on path segments (suffix / basename / directory containment) so absolute build-tree prefixes that never appear on the command line (e.g. /build/abc/src/include/api.h) still resolve against include/api.h (decision D3).

4. Serialization mechanics¶

Serialization (snapshot_to_dict()): 1. dataclasses.asdict() converts the snapshot tree to a plain dict 2. _sets_to_lists() recursively converts sets to sorted lists (JSON has no set type) 3. Enum values are converted to their string representation 4. Internal cache fields (_func_by_mangled, _var_by_mangled, _type_by_name) are reset to None before serialization 5. schema_version is embedded at the top level

Deserialization (snapshot_from_dict()): 1. Inspect schema_version (default to 1 if absent) 2. Reconstruct typed objects: Function, Variable, RecordType, EnumType, etc. 3. Reconstruct enum instances (SymbolBinding, SymbolType, Visibility, etc.) from string values 4. Platform-specific metadata reconstructed via _elf_from_dict(), _pe_from_dict(), _macho_from_dict(), _dwarf_from_dict(), _dwarf_advanced_from_dict()

5. JSON determinism¶

To ensure reproducible snapshots (important for diffing baselines in version control):

Sets are converted to sorted lists
Dict keys are naturally ordered by json.dumps(sort_keys=True)
Floating-point values are avoided in the schema

6. Cross-mode snapshot equivalence¶

A snapshot produced from DWARF data (--dwarf-only) and a snapshot produced from castxml headers produce the same JSON schema. The checker.compare() function treats them identically. This enables:

# Generate snapshots from different sources
abicheck dump lib.so --dwarf-only > dwarf.abi.json
abicheck dump lib.so -H include/  > ast.abi.json

# Cross-compare works
abicheck compare dwarf.abi.json ast.abi.json

Fields that only one source can populate differ in their empty representation based on their type:

Optional fields (T | None, e.g., elf, pe, macho, dwarf_advanced): null in JSON when the source doesn't provide them
Collection fields (dict / list, e.g., constants, typedefs, functions): empty {} or [] when the source doesn't populate them

For example, a DWARF-only snapshot has constants: {} (empty dict — no header parsing to extract #define values), pe: null (wrong platform), and dwarf_advanced populated with DWARF-specific data. Consumers should handle both null and empty-collection cases.

Consequences¶

Positive¶

Offline comparison without original binaries or headers
Baselines can be checked into version control
Cross-mode comparison (DWARF vs castxml) works transparently
Deterministic JSON enables meaningful diffs of snapshot files
Simple integer versioning avoids semver complexity

Negative¶

Schema version bumps break stored baselines (users must regenerate)
Forward compatibility is best-effort — new fields may be silently ignored
dataclasses.asdict() with post-processing is slower than custom serialization (acceptable for file sizes in practice)
No compression — snapshots for large libraries can be several MB

References¶

abicheck/serialization.py — SCHEMA_VERSION, snapshot_to_dict(), snapshot_from_dict()
abicheck/model.py — AbiSnapshot dataclass
ADR-003 — Data source architecture (DWARF vs castxml snapshot equivalence)