Skip to content

Entity: BiasCorrectorConfig

Definition

BiasCorrectorConfig is the frozen Pydantic model that selects and parameterises the national-scale residual bias corrector fitted during the POSTPROCESS stage. It lives as PostprocessConfig.bias_corrector (1:1). The kind field dispatches between a no-op identity corrector and a coverage-gap-based scalar corrector.

Kind

Pydantic BaseModel, frozen=True. Nested inside PostprocessConfig, which is nested inside ExperimentConfig.

Source of truth

market_insights_models/src/commodity_hindcast/config.py:528

Key attributes

Field Type Default Meaning YAML example
kind Literal["none","coverage"] required Selects the concrete AbstractBiasCorrector implementation. "none" → NoBiasCorrector (identity, zero correction). "coverage" → CoverageBiasCorrector (NASS in/out-county gap scalar) none
n_lookback_years int 10 Number of most-recent years used to compute the coverage gap scalar. Only used when kind="coverage" 10
reduction_method Literal["median"] "median" Aggregation method applied to the per-year gap values. Currently only "median" is supported median

Concrete implementations

kind Class Behaviour
"none" NoBiasCorrector Returns predictions unchanged; fitted and persisted as a no-op sentinel so the bias-corrector pkl always exists when the stage runs
"coverage" CoverageBiasCorrector Estimates the scalar gap between area-weighted yield in the included_geo_identifiers universe and the NASS all-county total over the lookback window; applies the median ratio as a multiplicative correction to the national aggregate

All production commodity configs (corn_usa, soybeans_usa, soybeans_bra, wheat_usa, cotton_usa) set kind: "none". The coverage corrector was introduced for research purposes when the included-geo-identifiers universe (top-90% or top-95% by production) has a systematic gap relative to the full-county NASS total.

Lifecycle

  1. Constructed as part of PostprocessConfig, which is constructed as part of ExperimentConfig.
  2. During POSTPROCESS, build_bias_corrector(cfg.postprocess.bias_corrector) instantiates the concrete class.
  3. The corrector is fitted on per-fold walk-forward predictions vs NASS actuals and persisted as bias_corrector.pkl at postprocessed/{commodity}/{fold_label}/bias_corrector.pkl.
  4. At delivery time, HindcastSlice.has_bias_corrector guards loading; predictions are adjusted before CI application.

Relationships

  • Owned by PostprocessConfig.bias_corrector (1:1).
  • Drives NoBiasCorrector or CoverageBiasCorrector instantiation in the POSTPROCESS stage.
  • Persisted as bias_corrector.pkl per fold; presence guarded by HindcastSlice.has_bias_corrector (lib/results/results_slice.py:40).
  • Consumed by the national aggregation step in stages/run_meta_models.py before CI application.

Concepts and pipelines that touch this entity

PRs and commits

  • PR #360 (PR-360.md) — BiasCorrectorConfig introduced alongside CoverageBiasCorrector; kind="none" added as explicit no-op sentinel.

Open questions

  • reduction_method is a Literal["median"] — only one value is supported. The field exists as a forward hook for future "mean" or "trimmed_mean" options but none are implemented.
  • n_lookback_years has no validation that it is positive or does not exceed the training history length. An out-of-range value would silently produce an empty lookback window and a NaN correction.