Skip to content

ADR-014: Output Format Strategy

Date: 2026-03-18 Status: Accepted — implemented. Amendment (2026-07-14): the "no information loss" / "no format-specific data loss" claims below are too strong and were superseded by ADR-036 (Report view-model and canonical report severity), which documents that formats deliberately diverge on classification axis (SARIF uses a finer per-kind severity, ABICC-compat HTML uses ABICC's own HIGH/MEDIUM/LOW) and, as verified directly against the formatter code for this amendment, on field coverage too: JSON carries the full DiffResult (including detectors[] coverage-gap info and the suppressed_changes[] list — see abicheck/reporter.py); SARIF and HTML carry per-change old_value/new_value/affected_symbols as structured fields (abicheck/sarif.py, abicheck/html_report.py) but no detector list; JUnit XML (abicheck/junit_report.py) embeds old/new values as free-text inside the <failure> body rather than structured attributes, and has no equivalent of the suppressed-changes section at all (verified: no suppress* reference anywhere in junit_report.py). What is still guaranteed across all native channels (JSON, Markdown/text, JUnit) is the breaking-boundary and override-propagation invariant from ADR-036 — read that ADR for the authoritative cross-channel contract instead of the paragraph below. Decision maker: Nikolay Petrov

Context

abicheck results must be consumable by:

  • Humans reading terminal output or CI logs
  • CI systems parsing machine-readable output for gate decisions
  • GitHub Code Scanning ingesting SARIF for PR annotations
  • Web browsers for standalone report viewing

No single format serves all consumers. The output format strategy defines which formats are supported, what contract each format provides, and how format selection works.

Decision

Human and machine output formats

Format Primary consumer CLI flag Default?
Terminal Humans (terminal, CI logs) -o terminal=- Scalar compare
Review Markdown PR/step summaries -o review=- No
Detailed Markdown Navigable archived report -o markdown=- No
JSON Automation, AI agents, scripts --format json No
SARIF 2.1.0 GitHub Code Scanning --format sarif No
HTML Standalone report viewing --format html No
JUnit XML GitLab CI, Jenkins, Azure DevOps --format junit No

Terminal and Markdown

  • terminal is the bounded plain-text scalar default. It puts compatibility, finalized gate/exit, scope, and policy first, then samples canonical review groups and reconciled count populations.
  • review is the bounded Markdown projection for PR comments and step summaries.
  • markdown is the complete navigable human report; it is intentionally an explicit export rather than pages of default log output.
  • Package/release fan-out retains detailed Markdown as its implicit default because it has no single scalar review document.

JSON

  • Machine-readable structured output
  • Top-level fields: library, verdict, summary, changes[], suppressed_changes[], detectors[]
  • Summary includes: breaking_count, source_breaks, risk_count, compatible_additions, total_changes, binary_compatibility_pct, affected_pct
  • Each change includes: kind, symbol, description, old_value, new_value, source_location, affected_symbols
  • Library metadata: path, SHA-256 hash, file size
  • Detector results: name, changes count, enabled status, coverage gaps

JSON output uses the same DiffResult data as the other formats and is the highest-fidelity format (see the amendment note above for the per-format field-coverage differences verified against the formatter code).

SARIF 2.1.0

  • Targets GitHub Code Scanning (upload via github/codeql-action/upload-sarif)
  • SARIF specification: OASIS SARIF v2.1.0
  • Mapping:
  • Each ChangeKind → SARIF rule (rule ID = ChangeKind.value)
  • BREAKING → SARIF level error
  • API_BREAK → SARIF level warning
  • COMPATIBLE_WITH_RISK → SARIF level warning
  • COMPATIBLE → SARIF level note
  • Tool version from importlib.metadata.version("abicheck")
  • Results include source locations (when available from headers)

HTML

  • Self-contained single file — no external CSS, JavaScript, or images
  • ABICC-inspired layout for familiarity (but not format-compatible)
  • Verdict banner with color coding:
  • BREAKING: red (#b71c1c / #ffcdd2)
  • COMPATIBLE_WITH_RISK: orange (#e65100 / #fff3e0)
  • COMPATIBLE: green (#1b5e20 / #c8e6c9)
  • Binary Compatibility % metric (based on old exported symbol count)
  • Sectioned change tables: Removed | Changed | Added
  • Demangled names displayed, mangled names as tooltips
  • Suppressed changes section (if any)

Self-contained HTML was chosen over an external-stylesheet approach to ensure reports can be emailed, archived, or opened offline without broken rendering.

JUnit XML

  • Targets CI systems with JUnit test result dashboards (GitLab CI, Jenkins, Azure DevOps, CircleCI)
  • Mapping:
  • Each library → <testsuite>
  • Each exported symbol/type → <testcase>
  • classname groups: functions, variables, types, enums, metadata
  • BREAKING / API_BREAK → <failure> element
  • COMPATIBLE_WITH_RISK → <failure> only when per-kind severity is error
  • COMPATIBLE → passing test case (no <failure> child)
  • When old snapshot is available, unchanged symbols appear as passing tests for a meaningful pass-rate
  • Uses xml.etree.ElementTree (stdlib) — no external dependency

JUnit was chosen over a proprietary CI-specific format because all major CI platforms support JUnit natively, making it the best single format for broad CI integration.

Format selection

abicheck compare old.so new.so                         # bounded terminal result (default)
abicheck compare old.so new.so -o markdown=-           # detailed Markdown
abicheck compare old.so new.so -o json=report.json     # JSON
abicheck compare old.so new.so -o sarif=report.sarif   # SARIF
abicheck compare old.so new.so -o html=report.html     # HTML
abicheck compare old.so new.so -o junit=results.xml    # JUnit XML

The format and destination are selected together with repeatable -o FORMAT=DESTINATION exports; the file extension never infers a format. With no export the scalar default is terminal=-. See ADR-036's 2026-09-14 amendment for the bounded-human-output migration.

Information preservation

All five formats are generated from the same DiffResult object, and verdict and exit code computation is independent of output format. See the amendment note at the top of this ADR and ADR-036 for the precise, verified cross-channel contract — formats are not byte-for-byte interchangeable projections of identical fields (SARIF/JUnit/HTML each omit or reshape some DiffResult fields relative to JSON).

Consequences

Positive

  • Every consumer has a first-class output format
  • GitHub Code Scanning integration via standard SARIF — no custom tooling
  • Self-contained HTML enables offline report archival
  • Bounded review output works everywhere with zero configuration; detailed Markdown remains an explicit export
  • JSON/SARIF preserve enough structured detail for automation; see the amendment note above and ADR-036 for what each format actually carries (it is not uniform across formats)

Negative

  • Multiple projections to maintain (terminal/review, Markdown, JSON, SARIF, HTML, JUnit)
  • SARIF severity mapping is a compatibility contract with GitHub
  • Self-contained HTML generates larger files than external-CSS approaches
  • JSON schema evolves with the project (see ADR-015 for schema versioning)

References

  • abicheck/reporter.py — Markdown and JSON formatting
  • abicheck/sarif.py — SARIF 2.1.0 output
  • abicheck/html_report.py — HTML report generation
  • abicheck/junit_report.py — JUnit XML output
  • abicheck/cli.py — --format flag and output file handling
  • ADR-036 — Report view-model and canonical report severity; the authoritative cross-channel contract superseding the "no information loss" claims in this ADR