The MMM handbook

A guide for developing impactful

marketing mix models (MMM)
 Marketing mix models (MMM) help marketers compare different
investments as they provide answers to questions such as:

• What drove my sales?

• What was my ROI?
• How do I optimise my marketing investment?

There is more to measurement strategies than the science behind the model. Incorporating business
context to shape an MMM is an art — one with implications for the model’s outcomes and financial
recommendations. Advertisers who embrace that art can empower their businesses to make more
strategic measurement-based decisions.

The checklist below summarises key MMM best practises to ensure rigorous model design.

Click on each section to explore related checklists:

1. Start with the right questions

2. Measure the impact on the right KPIs
3. Aim to capture everything
4. The insight is in the details
5. Use best practices for capturing media in MMM
6. What to ask for from an MMM partner
7. Taking MMM to the next level

1
 How to use this guide:

If you are considering an MMM for your organisation for

the first time, refer to this checklist when discussing
methodology with potential partners and internal marketing
science or data science teams. Check off all best practices
that you consider important for your business.

If you have already started your MMM journey, check off

all best practices already included in your models and
consider discussing the remaining ones with your MMM
team and partners.

Start with the right questions

While it can be tempting to start the conversation about

MMM with methodology, it is more beneficial to begin with
business questions and then choose the best approach to
answer them. MMM may not be able to answer all your
questions, so identifying the limitations of your planned
MMM solution will help you manage expectations and find
complementary solutions.

2
 Is your MMM designed to answer the following questions?

Mark your answers with a

Basic:
Yes No Unsure

What percentage of total sales is driven by each marketing channel?

How does the sales contribution from each channel compare to its share of investment?
How do non-media factors contribute to sales?
What percentage of sales would be lost if all media investments were cancelled?
Does digital media drive offline sales?
What is the historic ROI for each media channel?
What is the marginal ROI for each media channel?
What is the recommended budget allocation for each channel?

Advanced:
Yes No Unsure

Which tactics (creative, targeting, ad formats) contribute most to my media profitability?

What are the direct and indirect effects of media channels?
How does my media drive both short and long-term sales?
How do media channels compare in their ability to drive sales and brand outcomes?
How do I balance my media mix to drive both short and long-term sales?
What are the synergies between media channels?

Thought starter: Do you plan to use an MMM in order to answer a

question that is not listed above? If so, which one?
3
 Measure the impact on the right KPIs

Accurate marketing mix models recognise that

different media campaigns drive different outcomes.
While some campaigns drive brand perceptions,
others generate leads or drive sales online or offline.
Today, it is not uncommon to see MMM built around
several KPIs via a model design called structural
equation modelling, also known as nested models.

Nested models measure the impact on more than

one KPI. For example, if brand-building campaigns
are designed to drive brand preference, a brand
preference index from a brand tracker would be the
ideal outcome to measure in the model alongside
sales. Sometimes continuous data for brand index is
not available. In this case, proxies can be used such
as Share of Search.

Which of the following KPIs are (planned to be) included in your MMM?

Mark your answers with a

Yes No Unsure Yes No Unsure

Online sales Visit to site

Offline sales Share of Search
Total sales Google query volume
Online action (leads, subscriptions, Purchase intent
contracts, orders, etc.) Brand consideration
Offline actions Brand preference/liking
Total actions Brand awareness

Thought starter: Does your model capture any other KPIs/outcomes?

If so, which ones?
4
 Aim to capture everything

A distinctive strength of MMM over other media effectiveness measurement solutions is the flexibility of
model design and its ability to include outcomes and drivers that are relevant to the business while
measuring all these relationships in the same analysis. If there is uncertainty about whether sales are
impacted by an event or specific factor, MMM can confirm the impact by testing the relationship
between the outcomes and potential drivers and, when a relationship is confirmed, estimate the strength
of this relationship. With that in mind, it is important to be diligent in designing a comprehensive model by
including all possible outcome drivers.

Traditionally, an MMM aimed to capture the impact of marketing on bottomline outcomes only: sales,
transactions, or subscriptions. As more brands embraced full-funnel marketing, it became important to
measure campaigns based on the job they were created for and to recognise that not all campaigns are
designed to drive sales. Some marketing activity is reserved for building brand strength.

Luckily, brand measurement has existed for some time, with an ecosystem of brand tracking providers
offering customer surveys that measure the share of the audience that is aware of the brand (brand
awareness), has a positive mindset around it (preference) and intends to buy it (consideration or
purchase intent). Modern MMM increasingly focus on the multi-functionality of media, measure impact
on outcomes across the customer journey, and can even link media, brand strength and sales.

Which factors do you (plan to) include in your MMM?

Mark your answers with a

Yes No Unsure Yes No Unsure

Media support by channel Competitive media

Price changes Competitive price changes

Distribution, if applicable (e.g. Product launches

number of stores) Other intermediate outcomes

External events (e.g. COVID-19) (Google query volume, share of

Economic factors, if applicable (gas Search, site traffic)

prices, confidence index, other) Brand outcomes (brand

Weather awareness, consideration,

Seasonality preference)

Thought starter: Which additional drivers of sales does your MMM

include or which factors do you plan to test?
5
 The insight is in the details

You’ve probably heard the expression “garbage in,

garbage out” — this is very true in MMM. While data has
generally become more accurate and detailed thanks to
new data collection and storing and management tools, it
should not be taken for granted that data accuracy and
granularity are the foundation
of a robust MMM.

Receiving media data from trusted sources, such as the

Google MMM data request tool, can help ensure data
accuracy and completeness. It is a best practice to
validate data by comparing a high-level summary (e.g.
impressions or investment) with a similar summary from
other sources, regardless of the method used to extract
the data.

One of the most common criticisms of MMM is that the

results it produces are not detailed enough. For example,
channel-level ROIs can help make the right decision to
balance budgets, but without campaign or tactic-level
ROIs, marketers do not know what drives channel
performance and how to improve the overall channel
ROI. In contrast, channel performance by target, creative,
or even optimisation method can help identify which
tactics work better than others. The savviest marketers
differentiate campaigns by their marketing objectives
and measure their effectiveness on the relevant KPIs
within an MMM.

While designing models for granularity, it is important to be aware of the risk of “over-fitting the
model”, which occurs when too many variables are included and the reliability of the model is
sacrificed. One way to overcome this challenge is to rise the upper limit of the number of input
variables by increasing the number of data points in the modeled KPI. he most common way of doing
this is by segmenting all data by, for example, geographical areas or product segments, and stacking
those segments in the modelled dataset.

Another obstacle to granularity can be the nature of media campaigns: multiple channels can be used
simultaneously, leading to correlation of inputs. This compromises the accuracy of attribution
of impacts on sales between correlated channels. Sometimes the only reliable way to measure such
channels is to combine them in one input.

6
 How comprehensive and accurate is your data?

Mark your answers with a

Yes No Unsure

My media data...

is extracted from trusted sources.

is validated by comparing totals or trends with a different source.
splits tactics by channel.
splits tactics by creative.
splits tactics by targeting.
splits tactics by marketing objectives.

“The savviest marketers differentiate

campaigns by their marketing objectives
and measure their effectiveness on the
relevant KPIs within an MMM.”

Thought starter: Which additional dimensions exist in your MMM

media data?
7
 Use best practices for capturing media in MMM

MMM have been around for over five decades and this existence has resulted in a body of best
practices that lean on proven relationships. It is most important that you know what you are
modelling and understand how media works. Digital channels offer a greater variety of formats,
targeting, and optimisation options than traditional channels.

“'Carryover' is a type of transformation

from the original media input in order to
better represent the staying power of ads.”

Understanding the terminology

Seeing an ad today may result in a purchase decision a week from now. Carryover is a type of
transformation from the original media input (e.g. impressions) in order to better represent the
staying power of ads.

Since most media campaigns run over time, ranging from several days to weeks, the carry-over effect
from each day of the campaign must be added to the immediate effects in the subsequent days.
Media inputs need to be transformed to account for this effect. This is called the adstock effect.

As advertisers continue to increase investments in a media channel, the channel nears the point where
it has influenced all the people it could. The closer the investment levels to this point, the lower the
return is on investment. This is called diminishing returns. Media inputs need to be transformed to
account for this effect.

The level of granularity has a big impact on the calculated ROIs from MMM. Generally speaking, more
media inputs (e.g. by campaign, creative, bidding strategy, or target audience) produce more
accurate results by providing a differentiated view of the performance for each of these tactics.

8
“The nested model approach can calculate
the sum of the short and long-term sales
ROI, where the impact via improving brand
KPIs is measured.”
One of the key considerations for measuring the ROI of upper-funnel or brand building channels is
capturing total ROI, which is the sum of the short and long-term sales ROI. The latter can be calculated
using a nested model approach, where the direct impact on sales and the indirect impact via
improving brand KPIs are measured.

The types of inputs used to represent a channel also make a significant difference. For example, some
studies show that branded search modelled with clicks instead of impressions show a significantly
different ROI for branded search, while the difference is less noticeable for generic ads and shopping
ads.

It is up to the modeller's discretion to use clicks or impressions as inputs in the model, but it is
recommended to avoid using costs as input variables, as this can lead to misleading results. Costs can
vary based on targeting, advertiser's, and competitors’ bidding strategies, and can cause
misrepresentation of ad effectiveness.

Changes in media strategy are common and frequent in digital media. Moreover, our account teams
work relentlessly to help advertisers optimise their media. Therefore, it is important to recognise that
media ROI is expected to change over time. As a result, it is recommended to use campaign-level
inputs wherever possible or to distinguish between time periods when a channel-level variable must be
used to detect the changes in media ROI over time.

Finally, no model is perfect. Each model will have its own set of limitations in capturing digital channels,
such as:

• Collinearity: e.g. YouTube is flighted at exactly the same time as TV ads

• Low levels of spend may cause lack of a detectable ROI
• Lack of variation (common in always-on campaigns) may cause a lack
of detectable ROI

9
It is important to take note of the model’s limitations and keep them in mind when turning MMM
results into insights and budget recommendations. Sometimes model limitations lead to
supplementary analyses for a more conclusive result, or at least caution when acting on the
model’s insights.

Do you apply these best practices to digital channels in your MMM?

Mark your answers with a

Yes No Unsure

Each media channel has its own differentiated carryover effect.

Each channel input has been transformed to account for adstock.
Each media channel has been transformed to account for diminishing returns.
We and our modelling partners understand digital media channels and how to identify
which part of the customer journey they address based on creative, formats, targeting,
and bidding or buying strategy.
Our models capture digital media channels’ effectiveness at sub-channel level (by
creative, targeting, or campaign).
Our digital media channels are captured with impressions, clicks, or views instead of cost.
Our models are designed to capture direct and indirect effects of media.
Media ROI is differentiated by time periods via cutting variables in intervals in the model or
grouping campaigns by time period in ROI calculations.
Our modelling partners have explored and shared with us the model’s limitations which are
considered in forming recommendations.
Our model is based on additive multivariate regression.
Our model is based on multiplicative multivariate regression.
Our model is based on Bayesian approach.

Thought starter: Which additional best practices does your

MMM include?
10
 What to ask for from an MMM partner

Selecting the right MMM partner and building a high-impact collaboration requires the partner's
insight into your company's unique needs. Partners differentiate by methodology, technology, and
services. However, there are several aspects of MMM that are consistently appreciated by most
advertisers.

Refer to the checklist below to compare deliverables and the service level you receive from your
partner, and decide which of them are most important to your organisation. Keep in mind that with
MMM, like any other sector, a greater service implies higher costs.

Do you receive the below outputs or services from your MMM partner?

Mark your answers with a

Yes No Unsure

Historic ROIs (past return on investment).

Marginal ROIs (return on the next $1 invested).
Response curves (also known as saturation curves).
Hypotheses as to which factors influenced media ROIs.
Budget optimisation scenarios.
Recommended actions outside of the media budgets.
ROIs by channel.
ROIs by ad format.
ROIs by creative.
ROIs by targeting.
ROI by bidding strategy or buy model, e.g. CPM, CPV, CPA (if applicable).
Clear and informative charts and data visualisations.
Explanation of the model results and consultation.

Thought starter: Which additional outputs or services does your

MMM partner provide?
11
 Taking MMM to the next level

While marketing mix modelling has a high degree of

versatility in what KPIs it can analyse and which
factors can be included, it also has an important
Experiments
limitation — it cannot handle a high number of input
variables which precludes deep granularity of results.

To meet this challenge, a blended methodology Unified

approach is leveraged by some advertisers, merging Methods
attribution and MMM to augment results with more
MTA
granular media insights at a tactical level. MMM Multi-touch
attribution

One common point of criticism of MMM is the fact

that it is not causal in nature, but based on correlation.
There are ways to test for causality within MMM but it
can also be identified by combining MMM with other tactics, such as geo experiment or
incrementality testing. Such approaches can also help calibrate the incremental volume
from MMM.

Yet another way to take an MMM to the next level is to enrich the design or the insights with
campaign-level measurements such as brand lift studies that measure changes in a brand KPI
due to exposure to media on YouTube. This approach helps link improvements in brand KPIs
to sales.

Which analyses do you currently supplement your MMM with?

Mark your answers with a

Yes No Unsure Yes No Unsure

Attribution Test-and-learn programs

Experiments Other

Brand lift studies

Thought starter: Which additional analyses are used to complement

your MMM?
12