Nobody asks this first. Everybody is thinking about it. And the honest answer starts with a distinction most agencies skip, because it decides almost everything about how much risk you are actually taking.
Page Contents
The question is whether the product is proven
What “proven” means here is narrower than it sounds. A proven product is one that has already run paid user acquisition and made it work at some scale, even a small one. It has bought users at a cost, and those users behaved well enough to justify buying more. Paid acquisition has been shown to function for this product at all.
An unproven product is everything else, including products with strong organic growth. Organics tell you people like the product once they find it. They do not tell you whether a paid ad can profitably send a stranger to it, because organic users and paid users arrive with different intent and behave differently. Until paid has been made to work, that is an open question regardless of how healthy the organic numbers look.
For a proven product, the goal is not to find out whether paid works. It already does. The goal is to scale it profitably: to find the creative that holds its economics as budget grows, rather than the creative that looks cheapest at a small budget. With enough concepts, enough budget per test and a disciplined read, that search terminates.
For an unproven product, testing is establishing whether paid acquisition can work at all. That is genuinely uncertain, and creative volume does not change it. If paid cannot be made to work for this product, better ads find that out faster.
Most disappointment comes from running the second situation while expecting the first.
What “doesn’t work” actually means in each case
For an unproven product, it usually means the honest answer arrived: the market did not respond, and you now know that for the price of a test programme rather than a year of building.
For a proven product, failure looks different and more specific. It rarely means “no creative worked.” It usually means you found creative that works and could not scale it profitably. Something performs at $20k a month and falls apart at $60k. That is a real failure mode and a much more useful one, because it points somewhere specific: the audience is thinner than hoped, the economics are tighter than modelled, or the concept was narrower than it looked.
Agreeing which of these you are in, before you start, is the single most useful thing you can do.
You should never be surprised at the end
Whichever case you are in, the process should be legible week by week.
Weeks one and two: cut decisions, not results. Concepts get tested, most get killed. If everything is still running, nothing is being learned.
Weeks three and four: a direction. Not necessarily a winner, but a narrowing. Which angles are alive, which are dead, what the next batch is built on.
Week five onward: scale or a documented pivot. A winner getting budget, or an explicit change of approach with the reasoning written down.
Reaching halfway with no cut decisions, no narrowing and no pivot is the signal to intervene. Not the final number.
What creative cannot fix
Worth saying plainly, because these get misdiagnosed as creative problems constantly:
- Retention. If users churn in week one, better ads buy more churn faster.
- Pricing and paywall. Creative moves people to the paywall. It does not make the paywall convert.
- Product-market fit. Volume will not locate an audience that is not there.
When performance disappoints, the honest question is which of three things is wrong: the creative, the measurement, or the product. Those are three different diagnoses with three different fixes, and only one of them is creative.
This post is part of our guide to how to choose a mobile UA agency.
Frequently asked questions
Why isn’t my ad creative working? It depends whether the product has proven paid acquisition. For proven products, failure usually means you found working creative but could not scale it profitably, which points at audience size or unit economics. For unproven products, it may mean the market did not respond, which is a real answer rather than a failed process.
How long before I know if creative testing is working? You should see cut decisions within two weeks, a narrowing direction by week four, and either scaling or a documented pivot by week five. The final number should never be the first signal.
Can creative fix bad retention or a bad paywall? No. Creative brings people to the product. If they churn or do not convert once there, more creative buys more of the same outcome faster.
How do I know whether the agency is actually learning? Ask what was killed and why. A pipeline of tests with no kill decisions is not a test programme.
