A modelling workflow
Define the intervention and decision before selecting a model. Specify the outcome, population, time horizon, channel measurement, proposed spend change and comparison policy. A historical decomposition and a future budget decision usually require different quantities.
- Audit dates, units, missingness, target definitions and changes in measurement. Document aggregation and currency or price adjustments.
- State the causal assumptions, likely common causes and possible mediators. Assess which channel contrasts the available variation can distinguish.
- Reserve time-respecting evaluation data. Fit preprocessing and any target-derived features using training data only.
- Choose a supported estimator, transformations, likelihood and priors. Check the implied outcomes with prior predictive simulation before fitting.
- Add experimental calibration only when the supported contrast, units, population and uncertainty match the experiment.
- Fit, inspect computation and predictive discrepancies, and compare plausible specifications. Preserve a separate evaluation set if model selection uses the original holdout repeatedly.
- Interpret contributions and scenarios with posterior uncertainty and explicit assumptions. Review extrapolation and operational constraints before acting.
Bayesian inference combines the stated likelihood and prior; computational methods approximate that posterior to differing degrees. Neither a posterior interval nor a deterministic diagnostic status includes every source of model misspecification or measurement error. Model building, checking and comparison remain part of the analysis, as described in Bayesian Workflow.
Retain data and code versions, resolved configuration, seeds, diagnostics, comparisons and the decision rationale. The runner’s manifests help reconstruct execution, while responsibility for the evidence and decision remains with the analyst and decision owner.
Terms used in the guides
| Term | Meaning for a review |
|---|---|
| Estimand | The quantity defined by the outcome, population, intervention/comparison and horizon |
| Contribution | A specified model component or contrast; state which before interpreting it causally |
| HDI | Highest-density interval at a stated posterior probability |
| ETI | Equal-tailed interval from the corresponding lower and upper posterior quantiles |
| Predictive interval | Interval for replicated/future outcomes, including observation variation under the specified predictive distribution |
| R-hat and ESS | Sampling diagnostics for chain agreement and effective sample size; they do not identify a causal effect |
| Calibration | A likelihood contribution from matched external measurement evidence |
| Completed run | Execution finished according to the manifest; diagnostic or decision acceptance is separate |
Interval labels have concrete consequences in exported tables:
the summary facade’s method depends on retained sample dimensions
(src/ammm/mmm/summary/factory.py:240), whereas runner curves explicitly retain
ETIs (src/ammm/pipeline/stages/_core_metrics.py:139).