Most agency selection goes wrong in the first ten minutes, when the buyer asks for case studies and the agency shows a logo wall. Both sides are being reasonable. Both are also skipping the questions that actually predict whether the engagement will work.
A UA creative agency has one job: produce ad creative that makes paid acquisition profitable, and keep producing it as each winner wears out. Everything below is a way of testing whether a given agency can do that job for your product.
Page Contents
First, be clear what kind of shop you are talking to
The phrase “creative agency” covers three quite different businesses, and the confusion costs buyers real money.
- A production shop makes assets to your brief. UGC videos, statics, edits. You decide what to make and whether it worked.
- A media buying agency manages campaigns and reports on them. Creative comes from you, or from a production shop they subcontract.
- A UA creative agency decides what to make, makes it, runs it or hands it to whoever runs it, reads the result, and feeds the read back into what gets made next. The loop is the product.
If you need assets, a production shop is cheaper. If you need the loop, a production shop will leave you doing the hardest part yourself. We drew the line in more detail in creative agency vs media buying agency.
The one question to ask: what gets tested, and how is it split?
Not “how many creatives do you make”. What matters is what goes into the test slots each week, and how those slots are divided between four kinds of work:
- New ideas. A different reason to install: a new problem, a new person, a new promise. This is where the next winner comes from, and it fails most often.
- Building on past winners. A proven idea taken somewhere new: a different opening, a different format, a different audience for the same claim. This is where most of the reliable gains come from.
- Variations. The same ad with one thing changed: the hook, the first three seconds, the end card, the caption. Cheap, fast, and how a winner is kept alive.
- Indirect variations. A winner’s mechanism applied to a different subject: the structure that worked for one feature, rebuilt for another.
None of these is less important than the others. An agency that only ships new ideas is throwing away what it has learned. An agency that only ships variations will run out of road when the current winner fatigues, which it will. Anything at one extreme is a red flag; the work is finding the balance, and the balance moves as the account matures.
So ask: of what you tested last week on your last two accounts, how many were new ideas, how many built on winners, how many were variations, and how many were indirect? An agency that runs a real programme can answer from its test log. The arithmetic behind how many slots an account can support is in how many ad creatives you should test per week.
Seven questions that predict the outcome
1. How do you decide what to make? The answer should involve research: your competitors’ ad libraries, your app store reviews, your existing winners, the objections in your support inbox. An agency that starts from a “creative brainstorm” is guessing. An agency that starts from what is already working in your category, and why, is not. Ask to see how they deconstruct a competitor’s ad account.
2. How do you read results? This is where the most confident agencies fall down. If the answer is click-through rate, thumb-stop rate or a platform dashboard, keep interviewing. Platform metrics describe attention, not value, and on iOS the platform’s own conversion numbers are estimates. A serious agency will talk about modelled cost per retained or paying user, about reading results at the concept level rather than the ad level, and about what they do when the platform’s numbers disagree with your MMP’s. We set out what that looks like in how to analyze ad creative performance.
3. How do you structure a test? Look for a method: how many concepts run at once, how much budget each needs before it can be read, when a losing concept is cut, when a winner is promoted. Look for an opinion on whether a new concept gets its own ad set. If the agency cannot describe its testing structure in two minutes, it does not have one. See how to structure creative tests for Meta ads.
4. Where does AI sit in the process? Almost every ad made today involves AI somewhere, so “do you use AI” tells you nothing. The useful question is where. Using it to research the category, generate hundreds of hooks and scripts and select the ones worth producing is what makes concept volume possible, and it is why we built BruteForce AI rather than relying on a general-purpose chatbot. Generating a finished ad from a single prompt is cheap and usually looks it. Related: does AI reduce creative team size.
5. What is in scope, and what is not? Creative strategy, concept development, production, testing, iteration and reporting are the core loop, and an agency should be able to price them as a package. App store optimisation, paywall work and web-to-app funnels vary too much by product to sit in a standard package, and a good agency will say so rather than fold them in vaguely. Ask whether source files are yours. Ask who owns the ad account. Ask what happens to the creative if you leave.
6. How soon is the first creative live, and how long is the commitment? Reasonable answers: first creative inside a week, and a two-month minimum. Shorter than two months does not give a creative programme a full cycle to test, read, iterate and scale, so an agency offering a one-month trial is either padding the pipeline or not running a real programme. Longer lock-ins should be justified by something specific.
7. How do you price, and what does the fee buy? The industry has two models: a percentage of ad spend, or a fixed retainer for a defined scope. Either can be fair. What to watch is whether the fee tracks the work. A percentage that does not fall as spend rises is charging for a workload that is not there. We published our own numbers rather than make you ask: how much a user acquisition agency costs, what percentage agencies charge and the minimum ad spend that makes a creative programme work.
When not to hire one
If you are below roughly $50,000 a month in ad spend, a full creative programme cannot run: there is not enough testing budget to read results, whoever you hire. If your product has not yet proven it can retain users, creative will not fix that, and you will spend real money proving it. If your in-house team is shipping enough concepts and winning, buy them tools, not an agency.
This post is part of our guide to how to choose a mobile UA agency.
Frequently asked questions
What does a UA creative agency do? Decides what ads to make based on research, produces them, tests them in structured rounds, reads the results on modelled value rather than platform metrics, and feeds the read back into the next round. The loop is the service; assets are a by-product.
How should an agency split its creative tests? Across four kinds of work: new ideas, building on past winners, variations, and indirect variations. None is less important than the others, and a split sitting at either extreme, all new ideas or all variations, is a red flag. Ask for last week’s split on their last two accounts.
How should an agency be using AI? For research, hook and script generation at volume, and selection: the parts that make concept volume possible. A finished ad from a single prompt usually is not the answer. Ask where AI sits in their process rather than whether they use it.
How much does a UA creative agency cost? Retainers for a full creative and growth programme start in the mid four figures a month for serious agencies; ours starts at $7,500. Percentage models should be under 5% at scale and fall as spend rises. Below about $50,000 a month in ad spend, a full programme cannot run properly.
How long before the first creative is live? About a week from kickoff is a reasonable expectation. Measurement setup runs alongside it, which is part of why two months is the shortest engagement that tells you anything.
