Did the new promotion move the needle or just move product?

The Scenario

GreenLeaf Supermarkets is a regional grocery chain operating 64 stores across the Pacific Northwest. Last quarter, the marketing team launched a new promotional mechanic called "Basket Builder" — a shelf-edge campaign encouraging shoppers to add complementary items to qualify for tiered discounts (e.g., "Buy any 4 items from this aisle, save 15%"). Before rolling the promotion out to all stores, leadership wanted evidence that it actually works. They ran a controlled experiment: 30 stores received the Basket Builder promotion materials (Treatment group), while the remaining 34 stores continued with standard pricing and no promotional signage (Control group). Over a 3-week trial window, the team recorded the average basket size — total number of unique items per transaction — for a random sample of 50 customer transactions per store group per week.

The Statistical Challenge

After the 3-week trial, the data team aggregated the transaction records into two groups: 250 transactions from Control stores and 250 transactions from Treatment stores. The raw averages suggest Treatment customers added slightly more items per basket on average. However, basket sizes are naturally noisy — they vary by time of day, day of week, customer demographics, and store layout differences. The critical question is whether the observed increase in basket size is large enough to be statistically significant, or whether it could simply be the result of random variation between the two groups. As the analyst assigned to this experiment, you will apply a two-sample independent t-test (the core engine of A/B testing) to determine whether the promotion had a genuine, measurable effect.

What's at Stake

The Basket Builder campaign costs GreenLeaf approximately $38,000 per store per year to run (signage printing, staff briefing time, margin given away on discounts). If the promotion genuinely increases basket size, even by 1–2 items per transaction, the incremental revenue across thousands of daily transactions can easily justify the investment. If the difference is statistically insignificant, the company risks spending $2.4M annually chain-wide on a promotion that delivers no real lift. Your statistical conclusion will determine whether GreenLeaf scales, pauses, or redesigns the Basket Builder campaign.