Assess prior sensitivity

Distinguish preparing alternative configurations from fitting and comparing them. Stage 05 writes the scenario plan; stage 75 optionally fits the alternatives using the same data and reference sampler settings.

prior_sensitivity:
  enabled: true
  reference: reference
  scenario_policy: conservative_mmm
  fit_scenarios: false
  robustness_tolerance: 0.2

The default fit_scenarios is false. Set it to true deliberately when the additional fits and their computational cost are intended. The conservative policy varies supported prior scales; inspect the generated configurations to see which variations apply to your model. --no-prior-sensitivity and --quick disable planning and fitting stages.

For manual scenarios, use scenario_policy: manual and a scenarios mapping. Each scenario supplies optional description and reason fields plus dotted configuration-path overrides. The named reference cannot have overrides. Structural changes require allow_model_structure_overrides: true; treat those as specification sensitivity rather than solely prior sensitivity.

When fits are enabled, inspect scenario_fit_diagnostics.csv, sensitivity_comparison.csv, channel_robustness.csv and roas_sensitivity.png under 75_prior_sensitivity_fits. Poorly sampled fits cannot support a reliable comparison, and the numerical robustness tolerance is a declared policy choice. Stable estimates across a small set of alternatives do not prove identification.

This analysis differs from model.sensitivity.run_sweep, which changes model inputs while retaining a fitted posterior. An input-response sweep describes a conditional response surface; it does not refit under alternative priors.

Manual prior comparison

This block applies to a configuration with an explicit model.kwargs.model_config.saturation_beta build specification for a HalfNormal prior whose kwargs.sigma is 0.5. It prepares a doubled-scale alternative without fitting it; add the block to that complete configuration.

prior_sensitivity:
  enabled: true
  reference: reference
  scenario_policy: manual
  fit_scenarios: false
  robustness_tolerance: 0.2
  scenarios:
    reference: {}
    wider_media:
      description: Double the prior amplitude scale
      reason: Check dependence on the illustrative media scale prior
      overrides:
        model.kwargs.model_config.saturation_beta.kwargs.sigma: 1.0

Supported prior prefixes are model.kwargs.model_config., model.kwargs.adstock.kwargs.priors. and model.kwargs.saturation.kwargs.priors.. Structural paths are model.kwargs.adstock.class, model.kwargs.adstock.kwargs.l_max and model.kwargs.saturation.class; arbitrary sampler/data paths are not accepted (src/ammm/prior_sensitivity/overrides.py:18).

Inspect each retained config.resolved.yaml before enabling fits. Override paths must resolve through the existing configuration, and structural paths require explicit opt-in (src/ammm/prior_sensitivity/overrides.py:55, src/ammm/prior_sensitivity/config.py:47). A defensible sensitivity set varies assumptions whose uncertainty matters to the decision; it need not include an arbitrary range merely to obtain a preferred result.

When fits are enabled, the comparison records mean contribution share and all-time incremental ROAS with 94% highest-density intervals (HDIs). A row is robust when the absolute relative change in its posterior mean is at most the tolerance; zero/non-finite reference means make that change undefined and the flag false. HDI overlap is reported separately and does not determine the flag (src/ammm/prior_sensitivity/comparison.py:31, src/ammm/prior_sensitivity/comparison.py:74).

Stage 75 retains summary comparisons and fit-health rows, but does not save each alternative posterior. If the decision needs reproducible draw-level comparisons, run and archive each approved alternative as a separate retained run, then compare the same estimand and coordinate keys; do not claim those posteriors are already in stage 75 (src/ammm/pipeline/stages/core.py:584). Review diagnostics for each fit before interpreting stability, and narrow channel claims when credible alternatives remain materially inconsistent.

Implementation reference at 7cb7f20: src/ammm/prior_sensitivity/config.py:47, src/ammm/pipeline/stages/core.py:584.