Incrementality Testing Guide

Prove whether your marketing spend is actually driving new revenue — or just taking credit for conversions that would have happened anyway. Follow these six steps to run your first incrementality test.

Define Your Hypothesis

Start with a clear question: "Does Channel X drive incremental revenue, or would these customers have converted anyway?" Write it down. If you can't articulate the hypothesis, you're not ready to test.

Tip: Good hypotheses are specific and falsifiable. "Facebook ads drive incremental purchases" is testable. "Our marketing works" is not.

Choose Your Test Design

Select between a geo-based holdout (pause spend in specific markets), an audience-based holdout (suppress ads for a random segment), or a time-based test (on/off periods). Each has trade-offs in cost, speed, and statistical power.

Tip: Geo tests are the gold standard for most B2B companies. They're harder to contaminate and easier to measure.

Establish Your Baseline

Measure conversion rates, revenue, and key metrics for both test and control groups BEFORE the test starts. You need a clean baseline or your results will be meaningless. Run the baseline for at least 2-4 weeks.

Tip: Watch for seasonality. A baseline captured during a promotional period will skew your results.

Run the Test

Execute the holdout for a statistically significant period — typically 4-8 weeks for B2B. Resist the urge to peek at results early. Premature analysis leads to false conclusions and wasted effort.

Tip: Calculate your required sample size before starting. Running a test that's too short is worse than not testing at all.

Analyze the Results

Compare the test group (exposed to marketing) against the control group (not exposed). The difference in conversion rate or revenue is your incremental lift. Calculate statistical significance before drawing conclusions.

Tip: Incremental lift = (Test conversion rate - Control conversion rate) / Control conversion rate

Make Decisions and Iterate

Use the results to reallocate budget. If a channel shows no incremental lift, reduce spend and redirect to channels that do. Then test the next hypothesis. Incrementality testing is a continuous practice, not a one-time project.

Tip: Document every test — hypothesis, design, results, and decisions made. This institutional knowledge compounds over time.