Difference-in-Differences — Measuring Incrementality of Paid Ads

The Scenario

UrbanCart is a mid-sized U.S. e-commerce platform selling urban lifestyle products — fitness gear, home accessories, and personal care. Three months ago, the marketing team launched a paid digital ad campaign (Google Display + Meta) in five selected cities. Five comparable cities were intentionally left out of the campaign to serve as a comparison group. Both sets of cities had been tracked for weekly revenue for several weeks before the campaign launched. Now, four weeks into the campaign, the CMO wants to know: how much of the revenue increase in the campaign cities was actually caused by the ads — and how much would have happened anyway?

The Statistical Challenge

This is not a simple before-and-after comparison. Revenue tends to grow over time for all cities, and external factors — seasonality, competitor promotions, economic trends — affect all markets simultaneously. A naive comparison of revenue before and after the campaign in the treatment cities will overstate or understate the ad effect unless those background trends are accounted for. Difference-in-Differences (DiD) solves this by using the control cities as a counterfactual: whatever revenue trend the control cities experienced post-campaign, we assume the treatment cities would have followed the same trend if no ads had been run. The "difference" that remains — after subtracting the control trend — is the estimated causal effect of the campaign. You have been brought in as the analyst to run this DiD analysis on weekly revenue data from all ten cities across both periods.

What's at Stake

UrbanCart spent $48,000 on this four-week campaign across the five treatment cities. The marketing team needs to know whether the ads generated enough incremental revenue to justify that spend — and whether scaling the campaign to additional cities next quarter makes financial sense. If the DiD estimate shows significant incremental lift, the team will expand the campaign. If the effect is not statistically distinguishable from zero, the campaign budget will be reallocated toward other channels. A credible causal estimate — not just a trend comparison — is what the CMO needs to walk into the board meeting.