Meta-Models (Post-Processing)

Post-processing sits between raw fold predictions and the delivery layer. It has two orthogonal responsibilities: bias correction (shifting national yield estimates to account for uncovered geographies) and conformal calibration (attaching honest prediction intervals from walk-forward residuals). Both are fit once per fold/experiment and persisted to disk so downstream consumers — forecast, delivery, diagnostics — can load without recomputing.

Modules

bias_correction.py (204 lines)

Defines the AbstractBiasCorrector ABC and two concrete subclasses, plus a build_bias_corrector factory.

AbstractBiasCorrector

All correctors implement:

Method / property	Signature	Notes
fit	(nass_panel, included_geo_identifiers, test_year) → None	Must set internal state.
bias_kg_ha	@property → float	Raises if fit not called (concrete guard in CoverageBiasCorrector).
apply_national	(sim_kg_ha: float) → float	Returns sim - bias_kg_ha. bias_correction.py:57
apply_frame	(df, *, sim_col="sim_yield_kg_ha") → pd.DataFrame	Copies df and subtracts bias_kg_ha from sim_col. bias_correction.py:62
save	(path: Path) → None	Pickles self; creates parent dirs. bias_correction.py:76
load	classmethod (path: Path) → AbstractBiasCorrector	Fails fast with FileNotFoundError if missing. bias_correction.py:83

NoBiasCorrector

Degenerate pass-through: bias_kg_ha is always 0.0; fit is a no-op. Used when config.postprocess.bias_corrector.kind == "none".

CoverageBiasCorrector

Accounts for counties outside the model's included_geo_identifiers. For each reference year in [test_year - n_lookback_years, test_year - 1]:

bias_y = (area_out / area_total) × (yld_in − yld_out)

where in = model counties, out = remaining counties, yields are area-weighted. Annual bias_y values are reduced to a scalar by the configured reduction_method ("median"). bias_correction.py:107

Constructor parameters: n_lookback_years: int, reduction_method: ReductionMethod. Unfitted access of bias_kg_ha raises RuntimeError. bias_correction.py:152

build_bias_corrector

Factory reading a BiasCorrectorConfig (kind: "none" | "coverage"). bias_correction.py:195

conformalise.py (716 lines)

The core conformal prediction module. Introduces CalibrationResult — a frozen dataclass carrying fitted half-widths that supports save, load, and predict_interval.

Type aliases (types.py)

Defined in a leaf module to avoid import cycles:

ConformalMethod = Literal["quantile", "split_conformal"]
ResidualMode = Literal[
    "hindcast_oos_per_init_date",
    "hindcast_oos_per_year",
    "hindcast_oos_fully_pooled",
    "in_sample_pooled",
]
PerYearFallback = Literal["mean", "max"]
InitMD = str  # e.g. "04-10"

types.py:16

CalibrationResult

@dataclass(frozen=True) — immutable after construction. conformalise.py:106

Fields:

Field	Type	Purpose
residual_mode	ResidualMode	Which recipe produced this calibration.
method	ConformalMethod	"quantile" or "split_conformal".
experiment_key	str	Links back to the experiment.
levels	tuple[float, ...]	Confidence levels fitted (e.g. (0.5, 0.8, 0.9, 0.95)).
n_residuals	int	Total residuals consumed.
per_init_md	dict[InitMD, dict[float, float]] \| None	Half-widths keyed by "MM-DD". Set only in hindcast_oos_per_init_date mode.
per_year	dict[int, dict[float, float]] \| None	Half-widths keyed by test year. Set only in hindcast_oos_per_year mode.
pooled	dict[float, float] \| None	Single half-width set. Set by the two pooled modes.
per_year_fallback	PerYearFallback	"mean" or "max" — how to reduce across years on a miss. Only consulted in per_year mode; persisted in all modes for round-trip consistency.

Exactly one of per_init_md / per_year / pooled is non-None.

Persistence — save and load:

CalibrationResult has first-class save/load support via a long-format parquet representation (to_frame). This corrects a claim in commodity_hindcast_kb/domain_model/AGGREGATES.md:214 that it is a transient value.

def save(self, path: Path | AnyPath | str) -> None: ...  # conformalise.py:215
@classmethod
def load(cls, path: Path | AnyPath | str) -> CalibrationResult: ...  # conformalise.py:225

save serialises via to_frame() — one row per (key, level) with categorical dtype columns — then calls write_dataframe (S3-safe). conformalise.py:136,215

load reads the parquet, reconstructs per_init_md / per_year / pooled dicts by branching on residual_mode, then calls cls(...). conformalise.py:225

Both accept local paths and S3 URIs via cloudpathlib.AnyPath. The to_frame method raises ValueError (with a diagnostic message) if serialisation would produce zero rows. conformalise.py:187

Disk sidecar layout:

One parquet per configured mode under the experiment run directory:

{run_dir}/conformal/{residual_mode}.parquet

e.g. runs/corn_usa/conformal/hindcast_oos_per_init_date.parquet. Path constructed by calibration_path() in run_meta_models.py:60.

predict_interval:

def predict_interval(
    self,
    sim_yield: float | np.ndarray,
    *,
    fold_year: int | None = None,
    init_md: InitMD | None = None,
) -> dict[float, tuple[np.ndarray, np.ndarray]]: ...

conformalise.py:360

Returns {level: (lower, upper)}. Half-width lookup is dispatched by which field is non-None:

• per_init_md — calls _lookup_per_init_md(init_md): circular calendar interpolation via day-of-year. Exact hits short-circuit; non-exact lookups emit a logger.warning naming the bracket. Requires init_md. conformalise.py:279
• per_year — calls _lookup_per_year(fold_year): exact-hit-or-reduce. On a miss, collapses all calibrated years' half-widths via per_year_fallback (nanmean or nanmax). Linear interpolation is intentionally absent (adjacent fold-years have non-comparable accumulated residual sets). Requires fold_year. conformalise.py:319
• pooled — broadcasts the single half-width set; ignores both keyword args. conformalise.py:395

Four residual modes

Introduced in PR #361. Controlled by ResidualMode (types.py:16). Dispatched via _MODE_DISPATCH dict. conformalise.py:571

"hindcast_oos_per_init_date" (default)

One calibration per "MM-DD" init-date label, pooling walk-forward residuals across CV years. Residuals come from all non-production folds via _walk_forward_long_residuals. Grouped by init_md column (strftime("%m-%d")). Populates per_init_md. Honest about the season-of-year uncertainty gradient — early-season forecasts carry wider intervals than late-season. conformalise.py:470

"hindcast_oos_per_year"

Walk-forward bootstrap: fold k is calibrated using only the end-of-season residuals from folds with year < k. End-of-season residual for each year is taken as the last init_date row after sorting. The production fold year is assigned all accumulated residuals. Populates per_year. Naturally reflects accumulating evidence but requires per-year lookup at apply time. conformalise.py:493

"hindcast_oos_fully_pooled"

Every (year, init_date) walk-forward residual pooled into a single calibration. Treats within-year init-dates as IID — statistically loose but useful as a baseline. Populates pooled. conformalise.py:526

"in_sample_pooled"

Fallback when no CV folds exist. Pools residuals from the production fold's train_preds.parquet (ADM0-aggregated obs - sim). Issues a logger.warning explicitly labelling intervals as biased narrow (in-sample residuals under-estimate true OOS uncertainty). Populates pooled. conformalise.py:546

apply_conformal (polymorphic entry point)

Two overloaded call shapes: conformalise.py:584

1. Residuals primitive — apply_conformal(residuals: Sequence[float] | ndarray, levels, *, method=...) → dict[float, float]. Delegates to _residual_quantiles.
2. Experiment-level — apply_conformal(experiment: ExperimentResult, levels, *, residual_mode=..., method=..., per_year_fallback=...) → CalibrationResult. Dispatches through _MODE_DISPATCH, then dataclasses.replace stamps per_year_fallback onto the result.

method choices: - "quantile" — np.quantile of finite absolute residuals. - "split_conformal" — MAPIE's AbsoluteConformityScore.get_quantile (finite-sample-corrected Bonferroni-type adjustment). conformalise.py:90

MapieConformalRegressor

Sklearn-style wrapper around mapie.regression.SplitConformalRegressor for the case where calibration features (X, y) are in scope. conformalise.py:645

Method	Notes
fit(X, y)	Fits the base estimator. Rejected by MAPIE when prefit=True.
conformalize(X, y)	Calibrates residuals on a held-out set. Sets _is_conformalised.
predict(X)	Point predictions from base estimator.
predict_interval(X)	Returns (y_pred, {level: ndarray shape (n, 2)}). Raises if not conformalised. conformalise.py:704

This wrapper is intended for per-county conformal modelling where X is available; the residuals-only pathway (apply_conformal + CalibrationResult) is used in the main hindcast/forecast pipeline.

residuals.py (60 lines)

Contains build_rows, the per-fold row builder consumed by postprocess_experiment. residuals.py:22

def build_rows(
    national_by_init: pd.DataFrame,
    bias_corrector: AbstractBiasCorrector,
    calibration: CalibrationResult,
    commodity: str,
) -> list[dict[str, object]]: ...

For each (year, init_date) row in the ADM0-aggregated frame, it:

1. Applies bias_corrector.apply_national(sim_raw) to get the bias-corrected sim.
2. Calls calibration.predict_interval(sim_post_bias, fold_year=..., init_md=...), passing fold_year only when calibration.per_year is not None and init_md only when calibration.per_init_md is not None. residuals.py:52
3. Emits lower_{pct} / upper_{pct} columns for each level.

Output columns: year, init_date, sim_yield_kg_ha, obs_yield_kg_ha, bias_correction_kg_ha, crop_type, plus interval columns.

stages/run_meta_models.py (205 lines)

Orchestrates the POSTPROCESS stage. Key public functions:

Function	Purpose
calibration_path(run_dir, residual_mode)	Canonical path {run_dir}/conformal/{mode}.parquet. run_meta_models.py:60
fit_and_save_calibration(result, ci_levels, residual_mode)	Fits one mode via apply_conformal, saves, returns CalibrationResult. run_meta_models.py:70
fit_and_save_all_configured(result, ci_levels)	Fits + saves every mode in config.postprocess.conformalise (order-preserving; first element = primary). run_meta_models.py:85
get_or_fit_calibration(result, ci_levels, residual_mode)	Load-or-fit for forecast/delivery consumers. run_meta_models.py:100
primary_calibration(result, ci_levels)	Returns the calibration for config.forecast.residual_mode. run_meta_models.py:119
postprocess_experiment(run_dir, *, included_geo_identifiers)	Full POSTPROCESS run: fits all calibrations, iterates folds, aggregates to ADM0, writes postprocessed/national.parquet. run_meta_models.py:138

Separation of axes: postprocess.conformalise (a list) controls which sidecar parquets are written during hindcast. forecast.residual_mode (a single value) controls which is applied at runtime. A forecast can request a mode not in postprocess.conformalise — it is fit on demand by get_or_fit_calibration. run_meta_models.py:119

Loop structure in postprocess_experiment: fit_and_save_all_configured runs before the fold loop so calibrations are complete before any build_rows call. Production's walk_forward_preds has NaN obs_yield_kg_ha; this flows through build_rows without affecting calibration. run_meta_models.py:168

Cross-references

• lib/results/run_result.py — ExperimentResult consumed by apply_conformal and postprocess_experiment.
• lib/geo/aggregation.py — aggregate_weighted_frame used in residual extraction and national aggregation.
• config.py — BiasCorrectorConfig, PostprocessConfig.conformalise, ForecastConfig.residual_mode.
• lib/reference_data/nass.py — load_nass_county_panel_yield_area for coverage bias fitting.
• treefera_market_insights.shared.utils.dataframes — read_dataframe / write_dataframe used by CalibrationResult.save/load.

Relationships

• BiasCorrector is fit per fold (one pkl next to each fold's walk_forward_preds); CalibrationResult is fit once per experiment (shared across all folds).
• CalibrationResult is experiment-scoped, not fold-scoped: it pools residuals from all CV folds before calibrating.
• residuals.py:build_rows is the seam between the two: it applies the fold's bias corrector and the experiment's calibration together to produce the final output columns.
• MapieConformalRegressor is a parallel calibration pathway (feature-conditioned) unused in the current main pipeline; apply_conformal + CalibrationResult is the active path.
• domain_model/AGGREGATES.md:214 incorrectly describes CalibrationResult as transient with no persistence. The actual class has save (conformalise.py:215) and load (conformalise.py:225) methods backed by parquet sidecars written to {run_dir}/conformal/{mode}.parquet.