Status: active
Date: September 26, 2026
Methodology version: 4

0011: Public downloads: location, datasets, file and column names, stamps, and build order

Context

DL-01 through DL-05 (a downloads page at /data/, covering county summary, annual expenditures, annual revenues, population, property-tax data and major spending categories, in CSV/JSON with Parquet where useful, each carrying release_version, methodology_version and generated_at, as a re-export of the published /data/v1/ contract with no tolerance) depend on six questions 06-CONTEXT.md D-10 through D-14 leave open: where the download tree lives, the exact dataset list, every file and column name, how the three release stamps are carried, and — because D-12’s build order and D-13’s “SHA-256 listed on /data/” collide with today’s CI chain — where the download step runs relative to the Astro build. Per T-03-26 (applied identically at ADR-0008, ADR-0009 and ADR-0010), a decision that fixes a permanent public path, file name or column name is never agent-chosen; this record puts all six questions in front of the maintainer at once, with numbers measured from the real, currently-published production release, exactly as ADR-0008, ADR-0009 and ADR-0010 did before their own maintainer decisions.

All figures below are measured, read-only, by a scratch script outside the repository (06-01-PLAN.md mode_notes) that fetches production’s own /data/v1/ JSON over HTTPS into a scratch directory, builds each proposed dataset in memory using the pipeline’s own interpreter (uv run --frozen --directory pipeline, so pyarrow 25.0.1 builds the trial Parquet files), and writes CSV/JSON/Parquet copies to scratch only. Task 1 changed nothing under pipeline/, apps/ or .github/; its own verification confirmed active_methodology_version still returns 4 and all three trees were clean at measurement time, before Task 2’s maintainer reply and Task 3’s implementation of the chosen options.

Decision

Every subsection below states the option the maintainer chose, after Task 1 measured every option and Task 2’s checkpoint:decision (gate="blocking-human") recorded the maintainer’s reply (Decision sign-off, below). The rejected alternatives for every question are recorded under “Considered Options” in Rationale, not repeated here as if they were also in effect.

Q1 — Where the download tree lives

Chosen: option a. A separate, versioned downloads tree at the candidate root, downloads/v1/{state}/ (served as /_r/<release>/downloads/v1/florida/...), versioned independently of /data/v1/ so a future CSV column change does not force /data/v2/, and never a subdirectory of data/v1/<state>/ — _iter_government_directories (pipeline/publish/validation.py:328-340) walks every subdirectory of data/v1/<state>/ as a government directory unconditionally, so a downloads directory nested there would either be walked as a phantom government (loudly failing on missing summary.json etc.) or require teaching that walk a reserved name.

Q2 — The dataset list

Chosen: option a. Exactly six datasets, matching D-11’s named list and the fields already measured below: county-summary, annual-expenditures, annual-revenues, population, property-tax, major-categories.

Q3 — File names and formats

Chosen: option a. {state}-{dataset}.{format}, e.g. florida-annual-expenditures.csv, florida-major-categories.parquet. Stable across releases (the release itself lives in the /_r/<release>/ path segment, not the file name), and self-describing once a researcher has downloaded the file onto their own machine, detached from the URL that produced it.

Formats are fixed by D-13 regardless of file-name option: CSV and JSON for all six datasets; Parquet for the five long datasets measured below, not county-summary (67 rows, no benefit from a columnar format at that size). An index.json listing every dataset, format, byte size and SHA-256 lives in the same directory.

Q4 — Column names, types, numeric text and null encoding

Chosen: option a, “accept as measured.” The six column lists, Parquet types, numeric-text rules, sort keys and null encoding exactly as measured in Source Evidence below (built and proven against production’s own real published figures — every dataset actually builds, every row actually writes, in CSV, JSON and, for the five long datasets, Parquet). No column was renamed and the property-tax window columns are kept.

Flattening rule (applied throughout): a {value, source_ids} figure becomes one column named for the field; a change figure ({start_fiscal_year, end_fiscal_year, start_value, end_value, ratio, source_ids}) becomes <name>_start_fiscal_year, <name>_end_fiscal_year, <name>_start_value, <name>_end_value, <name>_ratio; every figure’s source_ids on a row are unioned, sorted and ;-joined into one source_ids column per row; null_reason fields are kept as their own columns.

Column lists (decided, measured):

county-summary (67 rows, one per government, from counties.json): government_id, slug, name, consolidated, first_fiscal_year, last_fiscal_year, dollar_year, latest_spending, ten_year_spending_change_start_fiscal_year, ten_year_spending_change_end_fiscal_year, ten_year_spending_change_start_value, ten_year_spending_change_end_value, ten_year_spending_change_ratio, population, population_change_start_fiscal_year, population_change_end_fiscal_year, population_change_start_value, population_change_end_value, population_change_ratio, spending_per_resident, real_spending_change_start_fiscal_year, real_spending_change_end_fiscal_year, real_spending_change_start_value, real_spending_change_end_value, real_spending_change_ratio, real_spending_per_resident, real_spending_per_resident_start_fiscal_year, real_spending_per_resident_start_value, real_spending_per_resident_ratio, source_ids (30 columns).
annual-expenditures (670 rows, one per government-year, from spending.json years): government_id, fiscal_year, total_expenditures, population, spending_per_resident, cpi_fy_mean, real_expenditures, real_spending_per_resident, dollar_year, cpi_series, null_reason, source_ids (12 columns; dollar_year/cpi_series are spending.json top-level fields, repeated on every row of that government).
annual-revenues (670 rows, from revenue.json years): government_id, fiscal_year, total_revenue, revenue_per_resident, null_reason, source_ids (6 columns).
population (670 rows, from spending.json years[].population): government_id, fiscal_year, population, source_ids (4 columns).
property-tax (670 rows, from revenue.json property_tax.years, plus the government’s first-to-latest window figure repeated on every row): government_id, fiscal_year, property_tax_revenue, property_tax_per_resident, null_reason, property_tax_change_start_fiscal_year, property_tax_change_end_fiscal_year, property_tax_change_ratio, property_tax_change_null_reason, source_ids (10 columns).
major-categories (737 rows, one per government-category, from categories.json categories; expenditure categories only, per 06-CONTEXT.md’s scope — no revenue-group dataset): government_id, category_id, label, is_residual, first_fiscal_year, last_fiscal_year, first_year, latest_year, change, change_ratio, latest_per_resident, source_ids (12 columns).

Parquet types (five long datasets; county-summary gets no Parquet, D-13): government_id, slug, name, category_id, label, cpi_series, null_reason, property_tax_change_null_reason, source_ids → string; every *fiscal_year column → int16; whole-dollar and population/count columns (total_expenditures, population, real_expenditures, total_revenue, property_tax_revenue, first_year, latest_year, change) → int64; per-resident dollar columns (spending_per_resident, real_spending_per_resident, revenue_per_resident, property_tax_per_resident, latest_per_resident) → decimal128(15, 2); ratio columns (*_ratio) → decimal128(12, 6); cpi_fy_mean → decimal128(10, 3); is_residual/consolidated → bool.

Numeric text in CSV/JSON (measured, written exactly this way by the scratch script): whole dollars, counts and fiscal years as integers, no thousands separator, no $; per-resident values to exactly 2 decimal places; ratios to exactly 6 decimal places (the published precision); CPI to exactly 3 decimal places; never a bare str(float); booleans as true/false. Null is an empty, unquoted CSV field; JSON null; Parquet null.

Sort keys (measured, as written): government_id then fiscal_year ascending (annual-expenditures, annual-revenues, population, property-tax); government_id then category_id ascending (major-categories); government_id ascending (county-summary).

Q5 — Release stamps

Chosen: option a. release_version, methodology_version, generated_at as the last three columns of every CSV and Parquet row, and as top-level fields of each JSON dataset file (whose rows array then omits them) — copied verbatim from the release’s own manifest.json fields release, methodology_version and created (D-12), never from the pipeline’s clock.

Q6 — Build order and how `/data/` lists sizes and hashes

Chosen: option a. Move publish.build_manifest ahead of the Astro build — it reads only the registry, the coverage dataset and the decision records, never dist/, so nothing about its own inputs requires it to run after Astro. Write apps/web/public/manifest.json. Run a new publish.build_downloads step immediately after it, reading apps/web/public/data/v1/ (byte-identical to what Astro copies into dist/data/v1/ today) and writing into apps/web/public/<downloads_root>/. Astro then copies both into dist/ at build time and the /data/ page reads the downloads’ own index.json at Astro build time. publish.validation still runs last and re-derives every cell from dist/, unchanged. Keeps D-12’s “after build_manifest, before publish.validation” ordering and leaves run.py untouched; departs from D-12’s literal “writing into the candidate dist/” (writes into apps/web/public/ instead, which Astro’s own build then copies into dist/ — the release gate’s own re-derivation from dist/ is unaffected either way). Plans 06-05 and 06-06 are written for exactly this option and do not need to be re-planned.

Machine-readable lines

The eleven lines below, plus one download_columns_<dataset>: line per dataset, are parsed directly by pipeline/tests/publish/test_downloads_decision.py and pin the values plans 06-05 and 06-06 compare their own code and contracts against.

downloads_root: downloads/v1/{state} download_datasets: county-summary, annual-expenditures, annual-revenues, population, property-tax, major-categories download_file_name_pattern: {state}-{dataset}.{format} download_formats: county-summary=csv,json;annual-expenditures=csv,json,parquet;annual-revenues=csv,json,parquet;population=csv,json,parquet;property-tax=csv,json,parquet;major-categories=csv,json,parquet downloads_index_file: index.json download_stamp_fields: release_version, methodology_version, generated_at download_stamp_placement: csv_parquet_every_row;json_top_level download_null_encoding: csv_empty_unquoted;json_null;parquet_null download_number_text: whole_integer;cents_2dp;ratio_6dp;cpi_3dp;bool_true_false download_sort_keys: county-summary=government_id;annual-expenditures=government_id,fiscal_year;annual-revenues=government_id,fiscal_year;population=government_id,fiscal_year;property-tax=government_id,fiscal_year;major-categories=government_id,category_id download_pipeline_placement: before_astro_build download_columns_county_summary: government_id, slug, name, consolidated, first_fiscal_year, last_fiscal_year, dollar_year, latest_spending, ten_year_spending_change_start_fiscal_year, ten_year_spending_change_end_fiscal_year, ten_year_spending_change_start_value, ten_year_spending_change_end_value, ten_year_spending_change_ratio, population, population_change_start_fiscal_year, population_change_end_fiscal_year, population_change_start_value, population_change_end_value, population_change_ratio, spending_per_resident, real_spending_change_start_fiscal_year, real_spending_change_end_fiscal_year, real_spending_change_start_value, real_spending_change_end_value, real_spending_change_ratio, real_spending_per_resident, real_spending_per_resident_start_fiscal_year, real_spending_per_resident_start_value, real_spending_per_resident_ratio, source_ids download_columns_annual_expenditures: government_id, fiscal_year, total_expenditures, population, spending_per_resident, cpi_fy_mean, real_expenditures, real_spending_per_resident, dollar_year, cpi_series, null_reason, source_ids download_columns_annual_revenues: government_id, fiscal_year, total_revenue, revenue_per_resident, null_reason, source_ids download_columns_population: government_id, fiscal_year, population, source_ids download_columns_property_tax: government_id, fiscal_year, property_tax_revenue, property_tax_per_resident, null_reason, property_tax_change_start_fiscal_year, property_tax_change_end_fiscal_year, property_tax_change_ratio, property_tax_change_null_reason, source_ids download_columns_major_categories: government_id, category_id, label, is_residual, first_fiscal_year, last_fiscal_year, first_year, latest_year, change, change_ratio, latest_per_resident, source_ids

Decision sign-off

Maintainer reply, recorded verbatim: “accept all recommendations”

Context for the record: the maintainer’s first reply to Task 2’s checkpoint was “1 a”, which was ambiguous between option recommended (option id 1 in the rendered list) and an answer to Q1 alone (Q1 a). The orchestrator asked for clarification; the maintainer then replied “accept all recommendations”, resolving the ambiguity.

Interpretation (the option the maintainer selected from Task 2’s checkpoint options): option recommended — accept every recommendation in this record as originally proposed: Q1 a (separate versioned tree at downloads/v1/{state}/); Q2 a (the six D-11 named datasets, no seventh revenue-sources dataset); Q3 a ({state}-{dataset}.{format} file names); Q4 a (the measured column lists, Parquet types, numeric-text rules, sort keys and null encoding exactly as proven against production’s own real published figures, no column edits, property-tax window columns kept); Q5 a (stamps on every row plus JSON top level, copied from manifest.json); Q6 a (publish.build_manifest and the new publish.build_downloads step move ahead of the Astro build; publish.validation unchanged).

Recorded: 2026-09-26T21:09:57Z (Task 2 of 06-01-PLAN.md, checkpoint:decision, gate="blocking-human"). This reply is the sole authority for the downloads location, the six dataset names, every file and column name, the stamp placement and the build order recorded in this Decision section — T-03-26’s mitigation: a permanent public path, file name, column name or build order is never chosen by the agent alone.

Rationale

Q1. Option a is the only one of the three that adds zero new meaning to _iter_government_directories’s existing “every subdirectory of data/v1/<state>/ is a government” invariant (Pitfall 3, 06-RESEARCH.md) while still keeping downloads independently versionable from the /data/v1/ contract itself — a CSV column addition should not force a /data/v2/ bump on every published county page.

Q2. Option a matches D-11’s own named list exactly, and every field CMP-02/DL-02 name is already measured, below, as buildable from the six named datasets with no gaps.

Q3. Option a’s per-state, per-dataset file name survives being detached from its download URL — the single property that matters once a file leaves the site and lives on a researcher’s own disk, in their own scripts, indefinitely.

Q4. The flattening rule and both alternatives were stated in 06-01-PLAN.md’s own <interfaces> before this record existed; option a is “the interfaces exactly as measured” — every column list, type and format rule below was independently built and proven (each dataset actually parses, every row actually writes, in all three formats) against production’s own real release, not merely proposed as an unverified column list.

Q5. Option a reuses the exact stamp fields ADR-0009’s and ADR-0010’s own published contracts already carry at the top level of every JSON document (methodology_version, and manifest.json’s own release/created), so a download’s stamp means the same thing a researcher already sees on every /data/v1/ file.

Q6. Option a is the recommended reading of D-12’s intent (the download step still runs after build_manifest, before publish.validation) that resolves the literal conflict with D-13/the UI-SPEC by moving the manifest step earlier rather than rewriting an already-built page.

Considered Options

Recorded here, not repeated in the Decision section above, since the maintainer’s “accept all recommendations” reply chose option a on every question:

Q1 b — data/downloads/v1/{state}/. A plausible, safe alternative (still a sibling of data/v1/, not a child of it) kept for a maintainer preference that did not materialize; not rejected for any defect.
Q1 c — data/v1/{state}/downloads/. Rejected: the only option that requires changing _iter_government_directories’s “every subdirectory of data/v1/<state>/ is a government” invariant.
Q2 b — the same six datasets plus a seventh revenue-sources dataset. Recorded because Phase 6’s UI research raised it as a considered-and-not-recommended addition, not because a gap was found in the six-dataset set; not adopted.
Q3 b — {dataset}.{format} (no state prefix). Shorter, but collides across a future state expansion (PRD §12) once a researcher has files from two states on the same filesystem; kept as the deliberately narrower alternative, not adopted.
Q4 b — edit named columns (maintainer-supplied old=new pairs) or drop the property-tax window columns. No edit was named and the window columns were kept; recorded as the maintainer’s own edit path, not because a naming or scope defect was found in the measured set.
Q5 b — a sidecar metadata file per dataset. A real alternative for a consumer who wants to diff many releases’ data files without also diffing per-row/per-file stamp text; not adopted, no correctness advantage over option a found.
Q6 b — keep today’s build order; the download step writes into the already-built dist/, then a post-build script fills the /data/ page’s size and hash slots into the already- rendered dist/data/index.html in place. Rejected as the recommended path: the only option in the project’s CI chain that rewrites a page after Astro has already rendered it, adding an HTML-patching maintenance surface for a problem option a solves by reordering two already- independent steps.
Q6 c — keep today’s build order; /data/ lists dataset names and links only, linking a generated SHA256SUMS file instead of listing hashes inline. Rejected as the recommended path: a narrower reading of D-13 than the UI-SPEC’s already-approved Downloads Page Contract (per-format size and truncate/expand SHA-256 display), which would require re-opening that already-checker-approved spec.

Both Q6 b and Q6 c are kept as real, working alternatives, not because option a was found to have a defect.

Source Evidence

Production release measured: 2026.09.26.001 (read from https://taxpayerscope.com/manifest.json, methodology_version 4, created 2026-09-26T12:21:26.187073+00:00). Every county’s summary.json, spending.json, revenue.json and categories.json, plus the statewide counties.json, were fetched from https://taxpayerscope.com/_r/2026.09.26.001/data/v1/florida/... — 269 of 269 requests succeeded (67 governments × 4 per-government files, plus counties.json), zero failures.

Per-dataset, per-format size and null-count evidence (measured, all 67 governments)

Dataset	Rows	Cols	CSV bytes	CSV gzip	JSON bytes	JSON gzip	Parquet bytes	Parquet gzip
`county-summary`	67	30	26,891	6,260	100,755	8,150	— (D-13: no Parquet)	—
`annual-expenditures`	670	12	146,481	20,761	362,064	25,016	20,535	15,461
`annual-revenues`	670	6	70,552	10,335	179,750	11,740	9,759	8,419
`population`	670	4	46,585	5,258	115,685	5,838	4,625	3,959
`property-tax`	670	10	83,270	10,648	318,344	12,607	11,344	8,875
`major-categories`	737	12	126,261	18,753	341,662	21,506	20,208	17,330

CSV total across all six datasets: 500,040 bytes (488 KB). JSON total: 1,418,260 bytes (1.35 MB). Parquet total (five datasets): 66,471 bytes (65 KB). For comparison, the full published /data/v1/florida/ contract these downloads re-export (every fetched summary.json/spending.json/revenue.json/categories.json/counties.json, uncompressed) totals 7,606,639 bytes (7.25 MB) — the flattened downloads are roughly 6.6% (CSV) to 18.6% (JSON) of the full contract’s raw size, because each download omits per-account-code detail (account_codes in categories.json) and non-expenditure revenue-group detail not named in D-11’s six-dataset list.

Per-column null counts (every dataset; a column omitted below had zero nulls across every row):

annual-expenditures (670 rows): total_expenditures 1, spending_per_resident 1, real_expenditures 1, real_spending_per_resident 1, null_reason 669 non-null-reason rows (1 row carries no_data_reported) — the single null-with-reason government-year is Jefferson FY2018 (ADR-0010 Q2), reproduced here by direct measurement of the same published JSON, not copied from ADR-0010.
annual-revenues (670 rows): total_revenue 1, revenue_per_resident 1, null_reason 669 (same Jefferson FY2018 row, revenues series).
property-tax (670 rows): property_tax_revenue 3, property_tax_per_resident 3, null_reason 667 non-null-reason rows, property_tax_change_start_fiscal_year 10, property_tax_change_end_fiscal_year 10, property_tax_change_ratio 10, property_tax_change_null_reason 660 (one government’s entire 10-year window is null; three individual years elsewhere carry a per-year property-tax null).
major-categories (737 rows): change_ratio 147 rows null (a category whose first_year value is zero cannot support a computed ratio — an existing, already-published CategoryRow.change_ratio nullability, not new to this dataset).
county-summary (67 rows) and population (670 rows): zero nulls in every column.

Text-field scan (formula-injection surface, D-14)

Every text-column value across all six datasets was scanned for ASCII-ness and for a leading =, +, -, @, tab or carriage return (the OWASP CSV-injection surface D-14 names — the concrete guard itself is D-14’s/plan 06-05’s responsibility, not minted here): 11,926 text cells scanned across 3,484 total rows (government_id, slug, name, cpi_series, null_reason, property_tax_change_null_reason, category_id, label, and every dataset’s source_ids column). Zero hits — every text value is pure ASCII, and none begins with any of the six guarded characters. (County names such as “Miami-Dade” contain a hyphen mid-string, which is not a leading character and is not flagged.)

Evidence for Q6: current CI step order and each step’s real inputs

Quoted from .github/workflows/build-release.yml (step names, in file order): “Run the pipeline from the archive (D-01)” → “Archive-backed checks” → “Install apps/web dependencies” → “Build the Astro site” → “Built-site checks” → “Carry the validation report into the release (D-05)” → “Build the release manifest (PUB-03)” → “Validate the candidate (PUB-05) — a non-zero exit ends the job here”. The Astro build step sets PUBLIC_RELEASE_ID from the allocated release id (its only release-scoped input); “Build the release manifest (PUB-03)” invokes publish.build_manifest --release ... --registry ... --coverage ... --decisions ../docs/decisions --out ../apps/web/dist/manifest.json — its declared inputs are the registry, the coverage dataset and docs/decisions/ only, never dist/’s own contents. The not-yet-built /data/ page (this phase) needs each download file’s byte size and SHA-256, which exist only once both build_manifest and the new download step have run — today, both run after the Astro build that would need to render /data/.

Methodology Version

No bump was taken by this record. Downloads re-export already-published /data/v1/ figures verbatim (D-10); nothing here derives a new figure or changes how an existing one is computed. This record was written at methodology_version: 4 (the version already active via ADR-0010) as status: proposed, which active_methodology_version ignored entirely while it was a draft. It is now status: active at the same methodology_version: 4, following the maintainer’s recorded Task 2 decision (Decision sign-off, above).

Revision (2026-09-26)

Same-day activation record, for the same visible-diff discipline every other active record in this directory carries: Task 1 wrote this record as status: proposed with all six questions quantified and no option chosen; Task 2 (checkpoint:decision, gate="blocking-human") obtained the maintainer’s reply the same date, after one clarification round over an ambiguous first reply (“1 a”); Task 3 rewrote the Decision section to the chosen options only (option a on every question), moved the rejected alternatives into “Considered Options” under Rationale, added the Machine-readable lines subsection and the Decision sign-off section quoting the reply verbatim, and flipped status to active — methodology_version stays 4 unchanged, since downloads re-export already-published figures and derive nothing new (Methodology Version, above). This revision recorded by Task 3, 06-01-PLAN.md.