Experimentation is critical to making MMM work.

With the rise of open-source Marketing Mix Model (MMM), one thing is clear: Experimentation is critical to making MMM work. The hard part? Analyzing them correctly.

The impact is huge, as wrong conclusions can shift entire marketing budgets. And yet, analysing experiments does not get much attention. Here is what you should not forget.

🔹 Correlation is key
Running a geo experiment? Your test and control regions need to be strongly correlated. Weak correlation means weak results. A common mistake is simply making the test group bigger. What really matters is finding a valid correlation while keeping the sales impact of pausing marketing as low as possible.

🔹 Fair benchmarking against your models
Marketing mix models assign value to every impression, but conversion lift studies don’t catch everything (think ad blockers, in-store purchases, or cookie consent restrictions). Make sure you’re comparing results fairly. Otherwise, you’ll likely underestimate your true marketing impact.

🔹 Account for contamination
What about customers travelling between test and control regions? This will significantly impact your test results. A customer who lives in the test region but works in the control region still sees ads. If you don’t correct for this, your incremental impact will be undervalued.