fabernovel loader

Oct 18, 2018 | 4 min read


Measuring & Experimenting: a job just for media agencies?  

Shifting towards a cross-expertise approach to measuring and experimenting

Adrien Delepelaire

Lead Marketing Strategist

Measuring results is good. But measuring, experimenting, learning and adapting is even better. As with growth hacking, the idea here is to adopt a state of mind and apply these approaches to all areas of expertise and levers across the board. Let's look at how we can make this cross-expertise approach to measuring and experimenting work for the media/creative duo.

Testing: should it always be left to media agencies?

Testing is generally left to media agencies, with the latter attempting to optimize a campaign across its scope of action. Audience testing springs to mind, in which the goal is to identify the best-performing targets (e.g. lookalike, affinity audience targeting, remarketing, etc.), as does lever testing (e.g. Paid Search vs. Affiliation), or media buying testing to identify which buying method is most effective and relevant (e.g. CPM, CPC, CPA, device).

Yet when we know that media is only responsible for 30% of a campaign’s performance, compared to 70% for creative agencies (source: Google Media Lab Research), it would seem smarter to take a more cross-expertise-focused approach to reap the rewards of testing that covers both areas, media and creative. Here are four areas to consider:

Horizontally: Does the test deliver results during or after the campaign?
Vertically: Is the test straightforward or complex to set up?

A/B creative target testing

A/B testing advertising lets you easily identify high-performing target markets at minimal expense. All you need is to have multiple creatives working on a same target, with specific set KPIs. This is where perfect integration between media and creative agencies is key, because planning and forecasting ahead of A/B creative testing will allow you to roll out several variations before you even launch the campaign, to test out specific hypotheses. This approach allowed us to identify engagement rates that were sometimes up to 10 times higher for certain types of creative material, enabling us to subsequently optimize the campaign using the highest performing assets.

Ultimately, the goal here is to define creative variations ahead of campaign roll-out, based on strategic control of the media plan to ensure we are able to efficiently optimize performances.

Identifying new personas for an existing product

A creative brief is often based on a persona or character the advertiser has in mind as the target for their campaign. In this type of scenario, the creative/media strategy will involve imagining and bringing to life a creative concept that speaks to the target audience. But media teams can (and should!) seek inspiration in the thoughts and mind-sets of the advertiser’s real-life targets. Thanks to the data available on advertising spaces’ platforms (e.g. Facebook), it’s possible to test (and then confirm or disprove) these new targets’ appetite for the campaign, and map out the creative variations to be used for the new target. Clan Campbell is a good example. Following their testing phase, they decided to integrate a new type of persona, gamers, and adapted their campaign materials to the gaming community.


Video to be inserted into the online article with the subtitle: Video adapted to the gamer target community

Identifying the target audience for a new product on the market

Media and creative integration make it faster to introduce a new product to your target market. In this case, the target and the approach to be taken with the target haven’t yet been clearly defined. The goal is to strip away the mystery surrounding certain unknown factors thanks to media and creative teams. How? Firstly, by having media and creative teams work together to develop a matrix of tests that encompass two aspects:

  • Audiences — the service’s potential target audiences (media strategy)
  • Messages — the value proposition to be tested (creative strategy)

This targeting matrix makes it possible to identify the top-performing audience/message combination. By gaining insight into your potential market, you can adjust your product’s positioning to optimize how you reach your core target market, to adapt your mission statement or to add new targets to your original list.

Matrix used by Youse (formerly Caploc) to find its product/market fit

Multivariate testing to identify the best user/creative combo

Media/creative testing can be taken much further to refine results and performance to the max. As an example, Dynamic Creative Optimization (DCO) lets you maximize a display banner’s performance based on multiple testing variables (e.g. product packshot, promotion, call-to-action). But an even better example is Netflix. The platform’s strategy is a masterclass in how to use data and testing to optimize performance. Reed Hastings, the CEO of Netflix, said it himself: testing — which he calls consumer science — is the secret to his company’s success.

“Leaders like Steve Jobs have a sense of style and what customers seek, but I don’t. We need consumer science to get there.”

Reed Hastings, CEO Netflix

Because they realized that picture quality varies from country to country, Netflix prioritizes developing and systematically testing different visuals and teasers for a single series to maximize user interest, wherever those users are.

Here’s an example using the series Sense 8 (source: The power of a picture)

But Netflix doesn’t stop there. The company hones its targeting vs. message strategy by drawing on its users’ interests and preferences, meaning that depending on whether you love thrillers, comedies or documentaries, you’ll see different visuals.

Netflix is a trailblazer in the field of media x creative testing and makes a fascinating and informative case study. If you’re interested in reading more on the subject and specifically on how technology is used by creative and marketing teams at Netflix, here are a few ideas to get you started.

More than just a fad, taking a cross-expertise approach to creative/media measuring and testing ensures you get the results you need and helps you avoid making risky decisions. The four applications we’ve covered are simply examples of what’s possible, and other applications may easily emerge depending on the constraints, challenges and goals of any given brand. But for now, let’s turn to the final pillar in the three-way relationship between creative agencies, media agencies and advertisers: technology.

logo business unit


is a multi-specialist performance agency committed to transform the digital complexity into pure performance. Expert in traffic acquisition, transformation optimization and data treatment and analysis, FABERNOVEL DATA & MEDIA identifies and activates the most suited and innovative solutions for its clients.

next read