Evidence Based Marketing | The Scientific Method
How a systematic approach to marketing activities enhances success rates
Every revolutionary marketing idea starts off with a bit of healthy speculation, but if that’s where the hard work stops, there could be trouble ahead.
A stunning idea isn’t a marketing solution until you’ve done your homework.
Effective, efficient marketing needs a more systematic approach – we need ‘Evidence Based Marketing’. We call it EBM for short…
A Sanity Check
The challenge arises every day:
At Mutiny, video content is a core part of our business, and as broadband access sweeps the world, it has become a critical part of our marketing activity. Requests for video budget are not unusual.
A 5-figure sum is also fairly manageable, but a recent demand was specifically for the editing suite, where one of my colleagues wanted to take a hatchet to existing tutorials.
The premise was that ‘Millennials like shorter videos’, and that was where the justification both began and ended.
At heart, the ambition is a great one, we need to know the impact of length on customer engagement and conversion.
The problem was that the premise of the expenditure boiled down to ‘let’s do this and see what happens’, and we were never going to sign off that kind of budget on a wing and prayer.
We needed to get some sanity into our marketing investments.
Create a Framework
The pace of change in industry and technology is so great it means that there’s no such thing as ‘business as usual’ in marketing.
There’s no guarantee that what worked last week will work next week, and that means that every piece of tactical activity we engage in is an experiment.
It also introduces a new phrase to our conversations: test and iterate.
While we accept that marketing is an art, for us it is also an investment, and we need to be able to create conditions for those tests that enhance their chances of success.
We need to find out what works and what doesn’t. We need to differentiate between our activities based on two principal criteria:
This means making a commitment to treat marketing as a science, and we found plenty of guidance from that discipline.
Science, naturally, has the scientific method:
The question was what that means for us, and it became the inspiration for a similar process covering marketing activities, creating a short cut to the results we needed.
An EBM Methodology
Translating the scientific method to a marketing environment means accommodating more of the activities that stimulate ideas, facilitate and justify marketing success:
We go into these in more detail in later tutorials, but a quick overview below illustrates the main principles of Evidence Based Marketing.
There’s a mythology around digital marketing that insists that the answers to all marketing challenges lie in the data.
In practice the foundation for great ideas come from lateral thinking and drawing connections between apparently unrelated situations.
Intuition is a fantastic source of inspiration!
For example, there’s an apocryphal tale in marketing that relates how someone discovered diaper sales and 6 packs of beer both surged early on Friday evenings as new dads picked up some ‘Dutch Courage’ to face a long weekend at home with the baby.
Shrewd retailers saw their chance and put them side by side on the shelf. Like magic, sales surged.
Whilst the data certainly would have suggested the correlation, it takes a human being to connect the two and see the opportunity.
As Steve Jobs once said…
Creativity is just connecting things. When you ask creative people how they did something, they feel a little guilty because they didn't really do it, they just saw something. It seemed obvious to them after a while. That's because they were able to connect experiences they've had and synthesize new things.
The trick when it comes to marketing ideation lies beyond the idea itself, it’s in the simplicity of the proposition.
We use a short mnemonic:
- Get… [someone]
- to… [do something]
- by… [performing this activity]
It’s called a ‘one-line strategy’.
You can imagine how it played out for our beer company:
“We can get anxious young fathers to invest in Friday night beers by selling them side by side with diapers”
Whatever your idea (your one-line strategy) is, make sure of three things:
- It delivers a business outcome
- It is memorable
- It is intriguing
Data plays a critical role early in EBM by validating the idea.
Whether it is retail data or customer research, data not only quantifies the opportunity, but it defines the terms for success.
The structure of a one-line strategy means that we need at minimum of three points of validation:
- The audience
- The act
- The execution
For our beer company the strategist needed three data points:
- The size of the customer base represented by young fathers
- The frequency of diaper purchase
- The correlation between diapers and the beer purchase
With those three points intact, the stage is set for the hypothesis.
A hypothesis differs from a one-line strategy because it has a focus on specific, measurable, testable outcomes.
A hypothesis is not a fact, it’s a proposition that can be proven or disproven, and if it’s going to be useful, it can be used as a foundation for further action.
For marketing purposes, we try to break this down into a single sentence that covers 4 key issues:
For example, our beer retailer hypothesis could propose:
“Young fathers making short-notice commuter visits to purchase diapers can be influenced by proximity marketing to increase beer sales”
Marketing departments are inundated with measurement opportunities, so the skill in EBM is to minimise the data to cover just those metrics which meet the terms of the hypothesis.
For primary metrics, the fewer the better:
- What quantity of the audience was reached
- What proportion of the audience encountered the promotion
- How did the purchase habits of the audience change
- What was the outcome in terms of sales
For secondary metrics it may be worth exploring other factors that might have affected the test, especially those know as ‘externalities’
Externalities are those events which cannot be anticipated or controlled during the experiment.
For example, for our beer retailer there may have been social or sporting events that could also influence purchases over the weekend, so it would be important to know how beer sales had changed in other locations.
It’s in the predictive phase of tactical planning that EBM transforms marketing efficiency.
It is the predictions that you make that dictate your budget:
A sale that you make is a customer that you have engaged, and every customer has a value.
We calculate that using ‘Customer Lifetime Value’, and you can read about it in other tutorials by clicking on CLV here. We won’t go through the details here.
The key point is that if you know these three figures…
- Marketing expenditure
- Customers acquired
- Customer lifetime value
… then you can forecast the return on investment (ROI) from your activity.
If the ROI is negative, or doesn’t compare well with other activities, then you either need to rework your test to deliver better outcomes or dump it altogether.
We mentioned earlier that every piece of tactical activity we engage in is an experiment.
We can expand on that: every experiment is an A/B test.
An A/B test is a situation where all the environmental factors for customers are duplicated in a second exercise, excluding the marketing activity itself:
- The ‘test’ environment (A) includes the marketing activity
- The ‘control’ environment (B) is identical except for the marketing activity
We do this to ensure that as many factors as possible can be excluded when judging the outcome of the test.
The actual success of your test is not defined by the outcomes in the test environment, but by the difference in outcomes between A and B.
For a robust test, you need to deliver it with enough scale for the outcome to rule out random variation – but that’s a separate tutorial.
Every analysis starts with the outcome.
It’s helpful to state this in a one sentence summary:
“Our investment of AA dollars delivered an additional XX customers compared with the control activity, and at an average CLV of BB dollars, our campaign delivered an ROI of YY%.”
This is the overriding message we want colleagues to understand.
The prompt for further analysis is that it’s unlikely your outcome matched your prediction. Whether it’s a better or worse outcome than you anticipated, we need to know two things:
- What went well, that we can do more of
- What didn’t go so well, that we need to modify
If a marketing experiment hasn’t got an action point, it was a waste of time.
Demonstrate a clear set of learnings that will influence your next activity.
And then, of course, return to the first stage… Ideation.
Squaring the Circle
Returning momentarily to the original challenge, what happens if you think Millennials prefer shorter videos?
Well, if you couch it in the terms our editor did, nothing at all.
She didn’t get the budget.
But when she came back a month later with a project that relied on video soundbites and enrolment incentives for a social media campaign empowered by shareability, she got it.
(She also smashed the forecast).
As usual with marketing competencies, there’s no justification in delaying EBM.
For your next project, summarise it according to the 8-step process:
There’s nothing to hold you back!