UA Creative Lab #2: Create better (and more) with A/B testing 

Mais produire plus ne suffit pas : encore faut-il savoir quoi produire, pourquoi, et comment itérer efficacement en s’appuyant sur des insights concrets. C’est là que l’A/B testing entre en jeu. Bien mené, il permet de structurer une démarche de création efficace, d’optimiser chaque élément, et de capitaliser sur ce qui fonctionne vraiment.

But producing more isn’t enough. You need to know what to produce, why, and how to iterate effectively based on actionable insights. That’s where A/B testing comes in. When done right, it helps structure a clear creative approach, optimize each element, and build on what truly works. 

Why run A/B tests in user acquisition? 

A/B testing isn’t just a one-off validation tool, it’s a strategic lever to shape and guide your creative direction over time. 

It helps to: 

• Identify which creative mechanics actually drive conversions, based on real data. 
• Adapt assets to each platform by testing what works on Meta, TikTok, Google, or Snapchat depending on placements and audiences. 
• Refresh campaigns intelligently by building on proven concepts, instead of starting from scratch. 
• Send strong signals to algorithms to improve delivery and lower costs. 

More importantly, in a context where brands produce dozens of visuals or videos each week, testing becomes the key to turning creative volume into real performance. 

A/B testing in the age of large-scale creative production 

Not all brands produce at scale. But as soon as you start adapting creatives for multiple markets, platforms, or even testing different visuals or messages, large-scale production quickly becomes essential. That said, creating without a clear strategy often means wasting budget. This is where prioritization becomes critical, and where testing proves its value. 

In this context, producing in volume without a testing strategy is like flying blind. A/B testing helps to: 

• Structure creative production: testing upfront makes it easier to define the most effective format/message/angle combinations to scale. 
• Prioritize ideas: not all creative concepts are worth developing. Testing reveals which ones truly perform. 
• Avoid creative sprawl: without testing, brands risk producing a large number of disconnected assets. Testing provides a clear framework to produce less, but better. 
• Iterate faster: the more frequently you test, the more learnings you generate, fueling a continuous optimization loop. 

In short, A/B testing is no longer a side process. It’s a foundational element of any large-scale creative strategy. 

For example, with one of our gaming clients, over 4,000 creatives were produced in just 6 months to support campaigns across multiple countries and platforms. Low-budget tests were used to identify the highest-performing combinations before rolling them out more broadly. 

Results:

• ROAS multiplied by 3 
• IPM up by 70% 
• Lower testing budget needed to validate winning concepts 

Best practices for structuring your A/B tests 

Testing is essential, but only if it’s done right. To ensure each test delivers actionable learnings, especially when it guides dozens of creative variations, here are the principles to follow: 

• One variable at a time

This is the golden rule. If you change the message, offer, format, and audience all at once, you won’t know what actually impacted performance. A solid test is based on a single clear hypothesis; for example: does an emotional hook perform better than a functional message?

• A clearly defined KPI

Every test should be tied to a specific objective. You won’t look at the same metrics when testing a top-of-funnel video (CTR, VTR…) versus a performance-oriented asset (CPI, ROAS, install rate). Defining your KPI in advance helps avoid misinterpretation. 

• Sufficient data volume

A test needs to run long enough to generate reliable insights. That means allocating the right budget, choosing the right exposure period, and ensuring balanced delivery between variants.

• A stable environment

To isolate the impact of the tested variable, everything else must remain constant: audience, country, bidding strategy, budget, time of day. Even small, unintended changes can skew results.

• An iterative logic embedded in production

A test is never the final step. It should feed into future variations: a winning hook can be adapted across formats, or a strong combination tested in new markets. This direct connection between testing and creative production is what makes scale efficient. 

• Consistent learnings and documentation

Each test should leave a trail: what worked, what didn’t, and what deserves further testing. This growing knowledge base helps sharpen production over time and avoid reinventing the wheel.

What not to do (especially when producing at scale) 

Running creative production without a testing strategy can quickly lead to inefficiency. Here are the most common pitfalls to avoid: 

• Testing without a clear hypothesis

A test should answer a specific question. If you don’t know what you’re trying to prove or improve, you risk wasting time, budget, and drawing the wrong conclusions.

• Drawing conclusions without the right testing conditions 

A test that runs for less than 24 hours or lacks sufficient budget won’t deliver reliable results. Without enough volume and stability, data is inconsistent, unrepresentative, and insights often misleading.

• Overlooking context

A hook that works well in France on Meta might not perform the same on TikTok in Brazil. Every insight must be recontextualized based on platform, geography, and audience, and ideally retested in each scenario.

• Ignoring seasonality and market dynamics

Running a test during a major sale, a slow period, or a time of high competition can skew results. Factoring in timing and market conditions is essential for meaningful analysis.

• Failing to act on insights

Testing only has value if results are applied. Too often, learnings remain theoretical and aren’t translated into concrete next steps. A test that isn’t followed by action is a missed opportunity.

Key takeaways 

• A/B testing is a strategic lever to structure and refine creative production, especially at scale. 
• It helps turn a volume-driven approach into a quality-driven strategy by quickly identifying what actually performs. 
• Each well-executed test informs future production: variations, adaptations, alternative scenarios… 
• Testing also helps feed algorithms with stronger signals, driving more effective and sustainable campaign performance. 
• Without testing, large-scale creative production loses impact, consistency, and profitability.