A campaign translated for the US receives fewer clicks than a more direct local rewrite. The team calls the rewrite a winner and prepares to use it across every channel. But did the message change, the audience change, or the offer change? And did more people understand what they were buying?
These questions matter because localization testing can easily become a test of several things at once. New photography, a discount, a shipping promise and a different audience may all arrive with the new words. The result may be commercially interesting, but it cannot isolate the wording.
Start with one decision: whether a particular explanation helps a defined group complete a buying task. Keep broader questions about product-market fit, price and service separate until the evidence supports combining them.
Build a test that answers the actual question
Illustrative scenario: Carrywell is a fictional Singapore-based travel-accessory brand evaluating a US campaign. CW-2 is a kit of two empty travel cases, measuring 20 × 12 × 6 cm and 15 × 10 × 4 cm. The example assumes the US offer, service arrangements, claim review and rights are already ready for a bounded test. No actual campaign, platform account or result is represented.
Both candidates show the same product image,US$40 price, contents, call to action and destination page with the same delivery and return terms. The intended US audience, eligibility, device treatment and scheduled observation window stay fixed. Only the headline changes:
A — close translation of the existing campaign: “Bring order to every journey.”
B — task-oriented adaptation: “Two cases for cables and small essentials.”
B explains a specific use. A expresses a broader brand idea. Neither is allowed to claim that the kit includes cables, guarantees organization or meets every traveller's needs. The actual image and destination must make the empty-case contents clear.
The test hypothesis is that B makes the task easier to recognize. It is not that US customers as a whole prefer direct copy, or that one phrase reveals a national culture.
Keep the difference small enough to interpret
| Condition | Keep the same for A and B | Why it matters |
|---|---|---|
| Offer | Product, price, contents and service terms | Otherwise a message result can be a different-deal result |
| Audience and eligibility | The same defined eligible population and assignment method | Prevent a better audience being credited to better words |
| Exposure period | A shared scheduled window | Avoid comparing different seasons, promotions or weekdays without explanation |
| Creative and destination | The same image, action and page | Keep the intended headline change identifiable |
Use a suitable assignment design for the question. Google Ads describes experiments that share an original campaign's traffic and budget, with configuration choices that depend on campaign type. That is a platform mechanism to investigate, not a guarantee that an arbitrary comparison is randomized or valid.
Check the actual assignment unit and repeat exposure. If the same person sees both versions or if delivery changes who is exposed, interpretation needs care. A sequential week-one/week-two comparison is not equivalent to a simultaneous experiment simply because both versions have reports.
Also verify the candidate before exposure. Language adaptation cannot introduce unsupported product claims, unauthorized imagery or a service promise the business cannot fulfil. Those are review failures to fix, not risky variants to run until the numbers settle.
A worked result that resists the obvious conclusion
Suppose the fictional test produces the following observations within the same fixed reporting window. Each arm has US$400 media spend and 10,000 rendered impressions. Clicks are events, not an asserted count of unique people. “Useful action” here means opening the size-and-contents module on the common destination page; it does not mean understanding or a purchase.
| Observation | A | B | What it shows |
|---|---|---|---|
| Click events | 200 | 300 | More recorded clicks for B |
| Click-through rate | 2% | 3% | Clicks divided by impressions |
| Size-and-contents opens | 60 | 60 | Equal recorded module-open events |
| Opens per click | 30% | 20% | Different composition after the click |
| Media cost per open | US$6.67 | US$6.67 | US$400 divided by 60, rounded |
B has 50% more clicks and a one-percentage-point higher click-through rate. It does not have more module opens. Opens per impression are 0.6% for both. If the intended outcome is use of product-choice information, the apparent attention winner has not delivered more of the chosen action in these observations.
Even that conclusion stays narrow. An open can be accidental or repeated; tracking may fail; a reader may understand from information already visible. If configuration, eligibility or repeat-event handling differs, reconcile those before using the table. The numbers do not establish statistical significance or causal business improvement, and the example supplies no purchase or retained-margin data.
Google's experiment-monitoring guidance provides reporting and interpretation tools. Any platform confidence or winner label must still be read against the metric and design. A result for clicks does not answer a different question about customer understanding or profit.
Add a task check, not a cultural verdict
The team also conducts a separate fictional, small comprehension exercise: six relevant participants see A and six see B with the same page. Five in A and four in B correctly identify that the kit contains two empty cases and no cables. These people are not pooled into the advertising event table, and the small groups are not a representative survey.
This is not evidence that A is statistically better. It is a reason to investigate the misunderstanding before scaling B. What made some readers expect cables? Was it the phrase, the context photograph or the way contents were presented? Ask them to explain what they saw rather than teaching the right answer and recording agreement.
A next candidate could say “Two empty cases for your cables and small essentials,” subject to checking the whole impression and preserving the same product facts. It needs a fresh review and a new bounded comparison. Do not silently change B midway and combine its earlier and later results.
The task check gives localization a practical purpose. It helps explain a mechanism that a click report cannot. It does not allow the team to label one market “literal” or another “emotional.”
Choose the next action at the right level
For Carrywell, the completed decision is to defer broad scaling of B as a business winner. Keep the useful observation that a task-oriented headline attracted more click events, investigate the contents misunderstanding, and prepare a clearer candidate. Review whether module opens are an adequate proxy for the actual customer task.
If the business objective is profitable retained orders, extend measurement to the relevant mature order cohort, costs and customer outcomes with a design suited to the causal question. If the goal is initial comprehension, spend the next learning effort there rather than pretending that a click test answered it.
Do not keep changing metrics until one makes the preferred version win. State the main decision and guardrails before exposure. Preserve the result, including when it supports no clear action. If a safety, claim, consent or service problem appears, act on that problem rather than waiting for the planned window to end.
When should the whole market story change?
A message test is too small to resolve every objection. If customers understand both versions but see no useful reason to buy the product at the stated price, test the proposition or offer through an appropriate separate design. If service is unsuitable, repair or narrow the route. If a genuinely different audience has a different task, commission a new brief and do not call the comparison a wording-only test.
The transferable habit is to write the exact changed message, list what remains fixed, define what the measured action means and record what it cannot show. A campaign becomes more useful when the team can explain why the next decision follows. A winner badge is not that explanation.