PR #340 — Dashboard: window-aware metrics, configurable truth source, generic vintage subset¶

At a glance¶

Author: ai-tommytf
Merged: 2026-04-28
Branch: tl/refactor-app-for-wheat
Net effect: Five changes to the Streamlit dashboard: (1) added "NASS actual (all counties)" column exposing the full-universe truth alongside the model-subset truth; (2) added "WASDE Harvest-time Forecast" column; (3) renamed "Model MAE" to "Treefera MAE" and split into May→Harvest / May→End windows; (4) added Subset / Full tabs to the Vintage Accuracy Table; (5) derived the Subset tab's decision points generically from each commodity's growing season rather than hard-coded wheat dates.
Why this matters: The user spotted a ~1–1.5 bu/ac gap between the dashboard's "NASS Actual" and the USDA Small Grains headline figure; this PR exposes both truth columns side-by-side and makes the comparison auditable across all four commodities.

PR body (faithful extract)¶

At a glance — what changed¶

#	Change	Where it shows up
1	New column: "NASS actual (all counties) (bu/ac)"	Per-Year Accuracy Breakdown
2	New column: "WASDE Harvest-time Forecast (bu/ac)"	Per-Year Accuracy Breakdown
3	Rename: "Model MAE" → "Treefera MAE"	Per-Year Accuracy Breakdown
4	New tabs: "Subset" / "Full"	Vintage Accuracy Table
5	Generic per-commodity decision points	Vintage Subset tab

Change 1 — Two flavours of NASS truth¶

County-level NASS data
├── (a) Subset used by the model
│       (~2,300 counties — those with complete weather + stress feature joins)
│       └─► area-weighted mean ─► written to delivery CSV as `nass_actual`
│           — this is what the dashboard already showed
│
└── (b) Full universe (every county that reports)
        └─► area-weighted mean ─► written to delivery CSV as
            `nass_actual_area_weighted_all`
            — closest to the headline number on page 13 of the USDA Small Grains report

Both columns were already produced in convert.py but only (a) was shown. The fix exposes (b) side-by-side.

Change 2 — WASDE Harvest-time Forecast¶

The natural twin of "Treefera EOS Forecast": pick the row with the latest init_date for the year. This is USDA's last in-season estimate closest to actual harvest.

Per-Year table layout after changes:

│ Year │ NASS actual │ NASS actual    │ Treefera EOS │ WASDE Harvest- │ Treefera MAE │ WASDE MAE │
│      │  (model     │  (all counties │  Forecast    │ time Forecast  │  (May→Hvst,  │  (May→Hvst│
│      │  region)    │  full universe)│              │                │   May→End)   │  May→End) │
│ 2022 │    174.9    │    175.2       │    176.0     │    172.5       │ 1.50, 1.50   │1.91, 1.91 │

Change 5 — Generic per-commodity decision points¶

Why it was broken: the Subset tab used hard-coded wheat display labels. Corn/cotton/soybeans got an empty tab.

Algorithm:

anchors = [
    plant_dt - timedelta(days=30),              # pre-planting
    plant_dt,                                   # planting
    plant_dt + timedelta(days=int(0.20 * season_len)),
    plant_dt + timedelta(days=int(0.40 * season_len)),
    plant_dt + timedelta(days=int(0.60 * season_len)),
    plant_dt + timedelta(days=int(0.80 * season_len)),
    harvest_dt - timedelta(days=14),            # pre-harvest
    harvest_dt,                                 # harvest
]
# Snap each anchor to nearest fold in schedule.fold_order:
def _snap(target: date) -> str:
    return min(schedule.fold_order, key=lambda f: abs((_fold_init_dates[f] - target).days))

Cross-year crops (gs_start > gs_end, e.g. winter wheat) place planting in the prior calendar year.

Resulting folds per commodity:

wheat      (8 folds): Oct 1, Oct 29, Dec 24, Feb 18, Apr 14, Jun 9, Jul 14, Jul 31
corn       (7 folds): Apr 1, May 13, Jun 24, Aug 5, Sep 16, Oct 15, Oct 29
soybeans   (7 folds): May 1, Jun 5, Jul 10, Aug 21, Sep 25, Oct 14, Oct 28
cotton     (7 folds): Apr 1, May 20, Jul 8, Aug 26, Oct 8, Nov 12, Nov 26

Corn/soybeans/cotton have 7 instead of 8 because two anchors collapse to the same nearest fold (pre-planting and planting land on the same fold when the schedule does not run weekly in that month).

Files / lines touched¶

Additions	Deletions	File
+225	-62	`market_insights_models/src/commodity_hindcast/src/app/app.py`
+18	-14	`market_insights_models/src/commodity_hindcast/src/app/charts_evolution.py`

Cross-references¶

Related entity pages: DeliveryRow, CommodityConfig
Related concept pages: dashboard truth sources
Related PR: PR-363 (dashboard startup fix that followed after the reference-data refactor)
Related PR: PR-331 (which added nass_actual_area_weighted_all and weather_correction_bu_ac to the delivery CSV)

Lessons captured¶

The delivery CSV carries two national NASS columns: nass_actual (model-subset counties) and nass_actual_area_weighted_all (full USDA universe). The latter is closest to the USDA Small Grains headline figure.
The Vintage Accuracy Table Subset tab derives its decision points from cfg["gs_start_doy"] / cfg["gs_end_doy"]; it must never hard-code dates for a single commodity.
_snap() snaps each growing-season anchor to the nearest fold by absolute day distance; ties break to the earliest fold in season order.
Cross-year crops (gs_start > gs_end) put planting in season_year - 1; the snippet plant_year = yr - 1 if gs_start > gs_end else yr is the canonical check.
"Model MAE" is ambiguous (WASDE is also a model); use "Treefera MAE" and "WASDE MAE" as column headers.
The May→Harvest window is the apples-to-apples comparison with WASDE because WASDE only forecasts new-crop wheat from May onwards.