GeoLift-SDID: Marketing Incrementality Measurement Framework¶

Core Business Problem¶

Let’s imagine you are the CMO of a hypothetical chain of stores, Carpets-R-Us. As CMO, you need to quantify the precise causal impact of marketing expenditure with statistical validity. Specific objectives:

Measure true incremental lift from advertising in DMAs 501, 502, and 503
Calculate ROI with statistical confidence intervals
Optimize budget allocation based on causal evidence, not correlation

Technical Implementation Workflow¶

1. Data Structure Requirements¶

Required format for matrix operations:

location,date,outcome,treatment
501,2024-01-01,12500,0
501,2024-03-01,13200,1
502,2024-01-01,9800,0
...

Technical specifications:

Pre-period minimum: 60 days (required for matrix stability)
Granularity: Daily time series with consistent intervals
Outcome variable: Revenue, conversions or other KPI (must be numeric)
Treatment encoding: Binary indicator (0=control period, 1=treatment period)
Panel balance: Complete time series for all units (no missing dates)

2. Control Market Optimization¶

python recipes/donor_evaluator.py --config configs/donor_eval_config_multicell.yaml --treatment-units 501 502 503

This executes a multi-metric evaluation algorithm:

Calculates correlation matrices between all potential control/treatment pairs
Computes RMSE (absolute error) and MAPE (percentage error) for level matching
Applies DTW (Dynamic Time Warping) for pattern recognition with temporal flexibility
Generates composite score through weighted normalization of all metrics

Critical validation: Check the weight distribution in weight_concentration.csv - high concentration indicates potential synthetic control instability.

3. Statistical Power Analysis¶

python recipes/power_calculator.py --mode power --config configs/power_analysis_config_multicell.yaml --alpha 0.05 --effect-sizes "0.05 0.10 0.15" --durations "14 28 42"

This performs t-test approximation calculations to determine:

Minimum Detectable Effect (MDE) at 80% power
Power curves for varying effect sizes (5-15%)
Sample size requirements for statistical validity
Type I/II error trade-offs at specified alpha (0.05)

Decision threshold: If MDE > 2× expected campaign ROI, test sensitivity is insufficient for meaningful decision-making.

4. Causal Impact Estimation¶

After intervention period completion:

python recipes/geolift_multi_cell.py \
  --data data/GeoLift_Multicell.csv \
  --treatments 501 502 503 \
  --donor-pool "504 505 506 507 508 509 510" \
  --intervention-date 2024-03-01 \
  --pre-periods 60 \
  --post-periods 30 \
  --output-dir outputs/campaign_results

This executes the SDID algorithm which:

Constructs synthetic control through weighted donor matrix
Calculates treatment effects via double-difference estimation
Computes standard errors through variance-covariance matrix
Generates statistical significance via asymptotic normality

5. ROI Decomposition Framework¶

The causal estimation generates these primary metrics:

Metric	Technical Definition	Business Application
ATT	Average Treatment Effect on Treated	Absolute lift attributable to intervention
ATT%	ATT / Counterfactual Baseline	Percentage lift over synthetic control baseline
SE	Standard Error of ATT	Uncertainty measure for confidence intervals
p-value	Probability of Type I error	Statistical significance threshold (reject null if < 0.05)
Synthetic Control	Weighted combination of donors	Counterfactual outcome without intervention

Financial impact calculation:

Incremental Revenue = ATT × Exposure (Days × Markets)
Statistical Confidence = (p < 0.05) ? “Confirmed” : “Indeterminate”
ROI = (Incremental Revenue × Margin - Cost) ÷ Cost

Decision Framework Based on Statistical Outcomes¶

Statistical Result	Confidence Level	Recommended Action	Budget Implication
ATT > 0, p < 0.05	High Confidence	Expand to additional DMAs	Increase allocation
ATT > 0, p > 0.05	Low Confidence	Extend test duration or increase test size	Maintain allocation
ATT ≈ 0, p > 0.05	Indeterminate	Modify targeting or creative	Reallocate
ATT < 0, p < 0.05	High Confidence	Terminate campaign	Eliminate allocation

Test Market Selection Methodology¶

If markets haven’t yet been selected, use these technical steps for optimal selection:

1. Data Preparation Requirements¶

Full panel dataset of all potential markets (min 90 days history)
Strict stationarity testing (ADF test) for all units
Covariate collection for any exogenous variables affecting outcome
Data completeness verification (no missing observations)

2. Donor Relationship Mapping¶

python recipes/donor_evaluator.py --evaluate-all --config configs/donor_eval_comprehensive.yaml

This systematic analysis:

Generates full N×N correlation matrix across all potential units
Assesses predictive power of each unit for every other unit
Quantifies pre-period fit quality through multiple metrics
Identifies optimal control candidates for any potential treatment unit

3. Market Selection Optimization¶

python recipes/power_calculator.py --mode market_selection --min-power 0.8 --effect-size 0.1 --duration 30 --config configs/power_selection_config.yaml

This optimization process:

Evaluates statistical power for all potential treatment units
Ranks markets by detection sensitivity for specified effect size
Identifies market combinations maximizing overall detection power
Calculates required sample size for reliable causal inference

4. Selection Criteria Hierarchy¶

From the output market_selection_results.csv, apply this decision framework:

Statistical criteria (primary):
- Minimum detectable effect (lower is better)
- Donor pool quality (higher composite scores)
- Pre-period stability (low coefficient of variation)
- Absence of structural breaks in time series
Business criteria (secondary):
- Revenue materiality (significant business impact)
- Media efficiency (lower CPM/CPC)
- Geographical diversity (representative mix)
- Absence of planned business disruptions

Once markets are selected based on this framework, implement the measurement workflow beginning with step 2.