Evidence Based Marketing | The Scientific Method

Evidence Based Marketing | The Scientific Method

How a systematic approach to marketing activities enhances success rates

Nick Fawbert

Every revolutionary marketing idea starts off with a bit of healthy speculation, but if that’s where the hard work stops, there could be trouble ahead.

A stunning idea isn’t a marketing solution until you’ve done your homework.

Effective, efficient marketing needs a more systematic approach – we need ‘Evidence Based Marketing’. We call it EBM for short…

A Sanity Check

The challenge arises every day:

At Mutiny, video content is a core part of our business, and as broadband access sweeps the world, it has become a critical part of our marketing activity. Requests for video budget are not unusual.

A 5-figure sum is also fairly manageable, but a recent demand was specifically for the editing suite, where one of my colleagues wanted to take a hatchet to existing tutorials.

The premise was that ‘Millennials like shorter videos’, and that was where the justification both began and ended.

At heart, the ambition is a great one, we need to know the impact of length on customer engagement and conversion.

The problem was that the premise of the expenditure boiled down to ‘let’s do this and see what happens’, and we were never going to sign off that kind of budget on a wing and prayer.

We needed to get some sanity into our marketing investments.

This tutorial is part of our Asia Pacific workshop series covering how effective marketing is empowered by data driven decision making.

You can find out more about our Data Driven Decision Making Workshops here.

Create a Framework

The pace of change in industry and technology is so great it means that there’s no such thing as ‘business as usual’ in marketing.

There’s no guarantee that what worked last week will work next week, and that means that every piece of tactical activity we engage in is an experiment.

It also introduces a new phrase to our conversations: test and iterate.

While we accept that marketing is an art, for us it is also an investment, and we need to be able to create conditions for those tests that enhance their chances of success.

We need to find out what works and what doesn’t. We need to differentiate between our activities based on two principal criteria:

  • Effectiveness
  • Efficiency

This means making a commitment to treat marketing as a science, and we found plenty of guidance from that discipline.

Science, naturally, has the scientific method:

  • Question
  • Hypothesis
  • Prediction
  • Test
  • Analysis

The question was what that means for us, and it became the inspiration for a similar process covering marketing activities, creating a short cut to the results we needed.

An EBM Methodology

Translating the scientific method to a marketing environment means accommodating more of the activities that stimulate ideas, facilitate and justify marketing success:

  • Ideation
  • Validation
  • Hypothesis
  • Measurement
  • Prediction
  • Test
  • Analysis
  • Iteration

We go into these in more detail in later tutorials, but a quick overview below illustrates the main principles of Evidence Based Marketing.


There’s a mythology around digital marketing that insists that the answers to all marketing challenges lie in the data.

In practice the foundation for great ideas come from lateral thinking and drawing connections between apparently unrelated situations.

Intuition is a fantastic source of inspiration!

For example, there’s an apocryphal tale in marketing that relates how someone discovered diaper sales and 6 packs of beer both surged early on Friday evenings as new dads picked up some ‘Dutch Courage’ to face a long weekend at home with the baby.

Shrewd retailers saw their chance and put them side by side on the shelf. Like magic, sales surged.

Whilst the data certainly would have suggested the correlation, it takes a human being to connect the two and see the opportunity.

As Steve Jobs once said…

Creativity is just connecting things. When you ask creative people how they did something, they feel a little guilty because they didn't really do it, they just saw something. It seemed obvious to them after a while. That's because they were able to connect experiences they've had and synthesize new things.

The trick when it comes to marketing ideation lies beyond the idea itself, it’s in the simplicity of the proposition.

We use a short mnemonic:

  • Get… [someone]
  • to… [do something]
  • by… [performing this activity]

It’s called a ‘one-line strategy’.

You can imagine how it played out for our beer company:

“We can get anxious young fathers to invest in Friday night beers by selling them side by side with diapers”

Whatever your idea (your one-line strategy) is, make sure of three things:

  • It delivers a business outcome
  • It is memorable
  • It is intriguing


Data plays a critical role early in EBM by validating the idea.

Whether it is retail data or customer research, data not only quantifies the opportunity, but it defines the terms for success.

The structure of a one-line strategy means that we need at minimum of three points of validation:

  • The audience
  • The act
  • The execution

For our beer company the strategist needed three data points:

  • The size of the customer base represented by young fathers
  • The frequency of diaper purchase
  • The correlation between diapers and the beer purchase

With those three points intact, the stage is set for the hypothesis.


A hypothesis differs from a one-line strategy because it has a focus on specific, measurable, testable outcomes.

A hypothesis is not a fact, it’s a proposition that can be proven or disproven, and if it’s going to be useful, it can be used as a foundation for further action.

For marketing purposes, we try to break this down into a single sentence that covers 4 key issues:

  • Audience
  • Context
  • Mechanism
  • Outcome

For example, our beer retailer hypothesis could propose:

“Young fathers making short-notice commuter visits to purchase diapers can be influenced by proximity marketing to increase beer sales”


Marketing departments are inundated with measurement opportunities, so the skill in EBM is to minimise the data to cover just those metrics which meet the terms of the hypothesis.

For primary metrics, the fewer the better:

  • What quantity of the audience was reached
  • What proportion of the audience encountered the promotion
  • How did the purchase habits of the audience change
  • What was the outcome in terms of sales

For secondary metrics it may be worth exploring other factors that might have affected the test, especially those know as ‘externalities’

Externalities are those events which cannot be anticipated or controlled during the experiment.

For example, for our beer retailer there may have been social or sporting events that could also influence purchases over the weekend, so it would be important to know how beer sales had changed in other locations.


It’s in the predictive phase of tactical planning that EBM transforms marketing efficiency.

It is the predictions that you make that dictate your budget:

A sale that you make is a customer that you have engaged, and every customer has a value.

We calculate that using ‘Customer Lifetime Value’, and you can read about it in other tutorials by clicking on CLV here. We won’t go through the details here.

The key point is that if you know these three figures…

  • Marketing expenditure
  • Customers acquired
  • Customer lifetime value

… then you can forecast the return on investment (ROI) from your activity.

If the ROI is negative, or doesn’t compare well with other activities, then you either need to rework your test to deliver better outcomes or dump it altogether.


We mentioned earlier that every piece of tactical activity we engage in is an experiment.

We can expand on that: every experiment is an A/B test.

An A/B test is a situation where all the environmental factors for customers are duplicated in a second exercise, excluding the marketing activity itself:

  • The ‘test’ environment (A) includes the marketing activity
  • The ‘control’ environment (B) is identical except for the marketing activity

We do this to ensure that as many factors as possible can be excluded when judging the outcome of the test.

The actual success of your test is not defined by the outcomes in the test environment, but by the difference in outcomes between A and B.

For a robust test, you need to deliver it with enough scale for the outcome to rule out random variation – but that’s a separate tutorial.


Every analysis starts with the outcome.

It’s helpful to state this in a one sentence summary:

“Our investment of AA dollars delivered an additional XX customers compared with the control activity, and at an average CLV of BB dollars, our campaign delivered an ROI of YY%.”

This is the overriding message we want colleagues to understand.

The prompt for further analysis is that it’s unlikely your outcome matched your prediction. Whether it’s a better or worse outcome than you anticipated, we need to know two things:

  • What went well, that we can do more of
  • What didn’t go so well, that we need to modify


If a marketing experiment hasn’t got an action point, it was a waste of time.

Demonstrate a clear set of learnings that will influence your next activity.

And then, of course, return to the first stage… Ideation.

Squaring the Circle

Returning momentarily to the original challenge, what happens if you think Millennials prefer shorter videos?

Well, if you couch it in the terms our editor did, nothing at all.

She didn’t get the budget.

But when she came back a month later with a project that relied on video soundbites and enrolment incentives for a social media campaign empowered by shareability, she got it.

(She also smashed the forecast).

Get Started

As usual with marketing competencies, there’s no justification in delaying EBM.

For your next project, summarise it according to the 8-step process:

  • Ideation
  • Validation
  • Hypothesis
  • Measurement
  • Prediction
  • Test
  • Analysis
  • Iteration

There’s nothing to hold you back!

1631 Rate this article:
Nick Fawbert

Nick FawbertNick Fawbert

With 25 years in the media and marketing industry, and almost 20 of those in digital marketing, Nick is one of the most experienced practitioners in the Asia marketing industry. He is Founder and CEO of Mutiny Consulting.

Other posts by Nick Fawbert
Contact author

Please login or register to post comments.

Get Started with Digital Strategy

Latest Digital Insights
Get In Touch About Workshops

Let us know how we can help...

Latest Market Maker Interviews
Browse our Workshop Series