TaxpayerScope
Status
active
Date
September 24, 2026
Methodology version
4

0007: National source archive object key layout

Context

ADR-0005 (population and CPI series selection) and D-08 (.planning/phases/03-orange-county-source-file-to-published-number/03-CONTEXT.md) require every per-resident and inflation-adjusted figure to resolve to a population observation and a CPI observation, each with its own archived, SHA-256-pinned source — the same PIPE-02/PRD §24 integrity guarantee ADR-0006 already gives the 144 Florida EDR files. But Census PEP and BLS CPI are national sources: PIPE-08 requires them to be collected and archived by core code, not Florida adapter code, so a second state’s population/inflation ingestion never has to reinvent this layout. ADR-0006’s raw/florida/edr/{county_slug}/{series}/… layout is explicitly scoped to the Florida EDR prefix and vocabulary by its own text (pipeline/adapters/florida/archive_keys.py’s module docstring: “Core stages must never import this module”) — it is not a fit for a national source, and amending it in place would blur that Florida-only scoping rather than closing the gap. This record fixes the national layout before the collectors that write it exist (03-RESEARCH.md Pattern 2), the same posture ADR-0006 took toward the Florida collector.

Decision

  1. The archive key shape for national sources is:

    raw/census/pep/{vintage}/{sha256_prefix}-{original_filename}
raw/bls/cpi/{series}/{sha256_prefix}-{original_filename}

    where vintage is the Census PEP release vintage (v2020, v2024, …), series is the lowercase BLS series identifier (cuur0000sa0), and sha256_prefix is the first twelve characters of the file’s lowercase hex SHA-256 digest — identical in shape and length to ADR-0006’s sha256_prefix.

  2. pipeline/collectors/archive_keys.py implements this as a pure function, national_archive_key(family, subdir, sha256, original_filename) -> str, where family is census/pep or bls/cpi. It raises ValueError on any malformed input rather than producing a plausible-looking key, mirroring adapters.florida.archive_keys.raw_archive_key exactly. It lives in pipeline/collectors/ — core, national code — and is never imported by, nor does it import, anything under pipeline/adapters/florida/.

  3. The object name — not just its containing path — carries the content hash, for the same reason ADR-0006 states: a re-collected source file whose bytes changed produces a different sha256_prefix and lands as a new object beside the original, never overwriting it in place.

Rationale

Why a new module and a new ADR, rather than amending ADR-0006/raw_archive_key. ADR-0006’s decision text and its implementing module are both explicitly Florida-scoped — widening raw_archive_key’s county_slug/series vocabulary to also accept a Census vintage or a BLS series identifier would either silently blur that scoping or require Florida-specific validation code to grow a national branch, exactly the coupling PIPE-08 exists to prevent. Rejected alternative: adding an optional national=True branch to adapters.florida.archive_keys.raw_archive_key. This was rejected because it would put national-source logic inside a module whose own docstring tells every core stage never to import it — a second state’s adapter would then have no reusable Census/BLS key layout at all, having to re-derive one from a Florida-scoped module it cannot depend on.

Why the content-addressed object-name principle carries over unchanged. ADR-0006’s rationale for hashing the object name, not just its directory, applies identically here: a re-collected Census file or a re-requested BLS window whose bytes changed must never overwrite the evidence a previously published figure traces back to. Nothing about being a national source changes that guarantee.

Why vintage/series replace ADR-0006’s county_slug/series as the addressable subdirectory. Census PEP files are identified by release vintage (v2020, v2024), not by a county or state — a single archived file covers every U.S. county at once, unlike EDR’s one-file-per-county-per-series layout ADR-0006 was built for. BLS CPI files are identified by series (cuur0000sa0) — there is exactly one series this milestone archives, but the subdirectory still names it explicitly rather than hardcoding a flat raw/bls/cpi/ prefix, so a second CPI series (or a future state’s own national series) has an obvious, collision-free place to land.

Why BLS API responses are pinned by their exact bytes, not re-derived from the series. The BLS public API’s own JSON payload carries a responseTime field that varies between otherwise-identical requests — re-collecting the same [startyear, endyear] window a second time yields a byte-different response even though the underlying CPI values are unchanged. This is expected, not a defect: the archived object and its manifest row pin the one response this pipeline actually reads and computes from, exactly as ADR-0006 already commits to pinning “one file, one hash” rather than trusting that a re-fetch is always identical. A later re-collection producing a new object beside the old one (per point 3 above) is the correct, visible, diffable outcome — never a silent in-place replacement of the pinned bytes.

Why the county/series-shaped browsability rationale still applies. ADR-0006’s “a human investigating an incident wants a prefix listing, not a manifest query” reasoning holds here too: raw/census/pep/v2024/ lists every Vintage 2024 Census file ever archived; raw/bls/cpi/cuur0000sa0/ lists every CPI-U window ever archived — both answerable by browsing the bucket directly, with no separate manifest lookup required.

Source Evidence

The two Census PEP source URLs and the BLS CPI series identifier this layout is built from (ADR-0005, “Source Evidence”):

Methodology Version

This decision takes effect in methodology_version: 1. It has no prior version to supersede — this is the first record of the national raw archive’s canonical S3 key layout, closing the gap ADR-0006 deliberately left open for non-Florida sources.

Revision (2026-09-24)

This revision supersedes methodology_version: 1 with methodology_version: 2. The decision recorded above is carried forward unchanged into methodology_version 2; the changes this version-2 coordinated bump makes are recorded in 0001-total-expenditures-definition.md, 0005-population-and-cpi-series.md and 0008-total-revenue-definition.md. This revision recorded by Task 3, 03-06-PLAN.md.

Revision (2026-09-24)

This revision supersedes methodology_version: 2 with methodology_version: 3 (ADR-0009, 04-01-PLAN.md). The decision recorded above is carried forward unchanged into methodology_version 3; the changes this version-3 coordinated bump makes are recorded in 0009-category-and-revenue-classification.md. This revision recorded by Task 3, 04-01-PLAN.md.

Revision (2026-09-25)

This revision supersedes methodology_version: 3 with methodology_version: 4 (ADR-0010, 05-01-PLAN.md). The decision recorded above is carried forward unchanged into methodology_version 4; the changes this version-4 coordinated bump makes are recorded in 0010-statewide-publication-rules.md. This revision recorded by Task 3, 05-01-PLAN.md.