How to choose a mobile user acquisition agency: track record vs system

Short answer: judge the system, not the case studies. Ask how an agency decides what to test, how it reads a result, and what it does when a creative fails. Case studies are cherry-picked and cannot separate luck from skill; a repeatable process can be interrogated. The eight questions below are the ones that actually predict performance, and the fee and minimum-spend context sits in how much a user acquisition agency costs and what minimum ad spend you need.

You can’t pick a good mobile UA agency from their case studies. They’re cherry-picked. You can’t tell luck from skill by looking at them. And the best agencies often don’t even publish their strongest work — clients won’t let them.

What actually predicts performance: weekly creative testing cadence, who owns the test plan, what reporting looks like before it’s prettied up, and whether the founder is still hands-on. The agency whose process you can describe in detail after one call is the one that will deliver.

Most agency vetting starts with case studies. They are cherry-picked, and you cannot tell luck from skill by looking at them. The agencies worth hiring have a track record AND a system. Most have one without the other, and the questions you ask reveal which.

I have been on both sides of this equation. At Bash Gaming, PuzzleSocial, and FreshPlanet — three companies I led that exited for over $200M combined — I hired agencies for specific countries, specific languages, specific channels.

Some were great. Some were terrible. The great ones had a tight lane and ran their corner of the work better than I could in-house. The bad ones tried to do everything and pattern-matched against case studies.

Now I run a mobile UA agency at RocketShip HQ. We have managed over $100M in client spend across mobile gaming and subscription apps.

I have watched founders run this evaluation well, and I have watched them run it badly. The difference is rarely about who they picked. It is about what they evaluated for.

This guide is what I would tell a founder calling me tomorrow asking how to vet the agencies they are interviewing — including how to vet us.

Why does standard agency vetting fail?

The fundamental problem with conventional agency vetting: you are trying to predict future performance from past artifacts that are partially or fully unreliable.

Industry rankings like the AppsFlyer Performance Index tell you about top-performing ad networks. They tell you nothing about whether a specific agency will perform on your specific account — which is the prediction you actually need to make.

Five things most founders look at, and why each one is weaker signal than it appears:

What we see: the artifacts agencies use to sell are not the artifacts that predict whether they will deliver for you specifically.

The shift you need to make: stop evaluating what they have done. Start evaluating how they think and how they work. The first set of signals is curated by them. The second set is harder to fake.

First, get clear on what YOU are hiring for

Before you can evaluate any agency, you have to know what you actually want them to do. This is the step most founders skip, and it makes every later evaluation noisier.

The agencies I hired at Bash Gaming and PuzzleSocial were not generalists. Each had a tight lane. Each ran their lane to complement what we did in-house — not to replace our internal work. Specialization was the entire reason we hired them.

We had our hands full in-house. That is why we brought in agencies — to support us in the lanes we could not staff ourselves, not to do the entirety of our job.

Three honest framings of what you are hiring for, before you start evaluating agencies:

What we see: the framing you pick determines which questions matter. Vetting a creative-production specialist on the same dimensions as a full-stack agency makes neither evaluation sharp.

The full-stack option is realistic — but only if YOU have enough understanding of the discipline to evaluate them mid-engagement. If you are hiring full-stack because you do not know UA, you cannot tell good work from bad work.

Specific-lane hires are the easiest to evaluate. Each agency owns one bounded slice. Channel-specific specialists in particular tend to outperform generalists on their channel because the rep counts compound.

Get this framing right and the questions in the next section narrow naturally. Get it wrong and you will be evaluating every agency against a fuzzy bar that is impossible to fail or pass.

Who should you consider? Build the shortlist from the lane, not from a list

“We’re looking for a mobile UA agency, who should we consider” is the question I get most, and a list of names is the wrong answer to it. The agency that was right for a gaming studio at $800K a month is wrong for a subscription app at $60K, and every list online was written by someone selling placement on it.

Start from the framing above. There are four kinds of shop, and the one you need depends on what is broken and what you already have in-house:

• Creative-led. Your media buying and measurement are fine; creative is the bottleneck. They own concepts, production and the test plan. The question that separates them is in how to choose a UA creative agency: ask for last week’s test split.
• Media-led. Creative exists; buying, account structure and measurement are the gap. They own campaigns, budgets and attribution. Ask to see their measurement audit before they touch spend, and how they read iOS numbers.
• Full-service. You have no growth team yet. They own everything end to end, and the thing to ask for is the one business metric they will answer to, not a CPI promise. The split between the first two kinds is in creative agency vs media buying agency.
• In-house plus specialists. Over about $1.5M a month sustained, or a product unusual enough that an agency would need months to learn it. You own strategy and measurement; one specialist per lane sits on top.

Then build a shortlist of two or three, not ten:

1. Find them through the work, not the ranking. Published systems are the tell: a weekly test cadence described in detail, an audit checklist, a channel comparison with real numbers. An agency that publishes its process can be held to it.
2. Ask one question only an operator can answer. “How many distinct concepts did you ship last week for an account at our scale, and how many were new ideas versus variations?” A sales team cannot answer it. A strategist answers it in one breath.
3. Check who you would actually work with. The founder on the call is not the person in your account. Ask who runs the weekly review and whether the founder still reads it.
4. Get a reference at your spend tier. A reference from a client five times your size describes a different service.
5. Run a paid pilot on one metric. Two months, one number agreed up front, full account access on both sides. A pilot answers in eight weeks what a pitch cannot answer at all.

And if you are under roughly $50,000 a month in spend, do not shortlist agencies at all yet. The minimum spend is arithmetic, not snobbery; a freelancer or an internal generalist is the better answer until the testing volume is there.

What questions should you ask a mobile UA agency?

Each question + why it works + what good and bad answers sound like.

1. Is the founder still hands-on, or are they only the CEO?

Good answer: “I review every account weekly, sit in test plan reviews, write strategy briefs for new accounts. And I’m hands-on with our AI tooling — the workflows, the prompts, the integrations.”

“The agencies that scale in 2026 are the ones whose founders are building the AI layer themselves, not delegating it to a tools team.” Or some specific version of doing the work AND owning the AI infrastructure.

Bad answer: “I am the CEO. I run the business. The team handles execution.” In the age of AI, this is a problem at any agency size — including 200-person shops.

A founder who is not doing some hands-on work is too distant from the front lines. It is unclear whether they will be able and willing to adapt their agency’s workflow to AI tooling, which is now the differentiating capability.

Founder-led without hands-on craft is a sales-led organization with the founder as the rainmaker. The work is delivered by whoever happens to staff your account — and the people building the AI tooling are not the people deciding strategy.

2. What AI tools have you BUILT — not just what AI tools do you use?

This question separates 2026 agencies from agencies that bolted ChatGPT onto a 2022 workflow.

Good answer: “We have built X for competitive ad teardowns, Y for creative variant generation, Z for weekly account reads. Here is how the tooling integrates with our test cadence.” Concrete, specific, defended by the workflow it sits inside.

Proprietary AI tooling is becoming the moat in agency economics. Agencies without their own infrastructure are competing on labor cost — which is a losing race in 2026.

Industry analysis at Mobile Dev Memo has tracked the post-ATT era’s structural shifts in mobile UA economics. The agencies adapting are the ones building tools, not just buying licenses to other people’s tools.

Bad answer: “We use ChatGPT for ideation” or “We use Cursor for the creative team.” Using off-the-shelf AI is table stakes.

Building AI infrastructure that compounds is the actual differentiator. If they cannot describe what they have built and why, they are running last decade’s workflow with a chatbot bolted on.

3. What is your specific lane?

Tied to your “what am I hiring for” framing in the section above.

Good answer: A clear, narrow specialization that they can defend with depth. “We are best at subscription apps spending $200K-$2M/month on Meta and TikTok in Tier 1 English markets.” Specific. Bounded.

Bad answer: “We do everything for everyone at every scale.” This is rarely true. When it is true, you are paying for capacity, not expertise.

4. Walk me through your weekly creative testing cadence.

This is the question that exposes whether they have a system or wing it.

Good answer: Specific. “Every Monday we review last week’s results against the test plan, kill or scale based on these criteria, and brief next week’s tests by Wednesday. Production happens Thursday-Friday. New tests live Sunday.” With named meeting structures, named outputs, named owners.

Bad answer: “We test creatives continuously.” Or worse: “It depends on the client.” Both mean there is no system. The work happens when someone remembers to do it.

For context on what “good” testing cadence looks like at scale, the A/B testing framework we use describes the structural pieces: hypothesis design, kill criteria, learning capture, and weekly cadence rhythm.

5. How many distinct concepts per week are you producing for an account at our scale?

Volume of creative is rarely the constraint. Concept diversity is. Fifty ads from five concepts will saturate the same five audiences.

Good answer: A specific number anchored in the account’s spend level (e.g., “8-12 distinct concepts per week for accounts at $200K+/month, with 2-3 variants per concept”). They distinguish concept count from variant count.

Bad answer: A big-sounding asset count without distinguishing concepts from variants. “We produce 50 creatives a week” tells you nothing about diversity. Volume is cheap. Concept diversity is what compounds — and most agencies blur this distinction on purpose by reporting variant counts as if they were concept counts.

6. When a campaign underperforms, what are your first three actions?

Tests their failure-mode response. Underperformance is when work-quality shows up most clearly.

Good answer: A diagnostic ladder. “First, check whether the underperformance is structural (audience saturation, attribution drop) or creative (the concept itself fatiguing). Second, run the next 3 candidates from the testing queue. Third, redesign the concept brief if the issue is concept-level.”

Bad answer: “We optimize the campaign.” Vague. Or: “We make new creatives.” Reactive, not diagnostic.

7. Show me what your weekly client report looks like — a real one, names redacted.

Tests transparency. The agency that hides behind summary metrics has something to hide.

Good answer: They send the actual artifact. Real numbers, full-funnel cuts, last week’s tests with results, this week’s plan, open questions. You can read it without clarification.

Bad answer: A pretty PDF deck with hand-curated highlights. Or refusal to share until you sign. The reporting they show in a sales process is the BEST version they have. If that version is theater, the working version is theater.

8. What would you NOT take on, and why?

Tests strategic honesty. An agency with a real strategy turns work down.

Good answer: Specific exclusions with reasons. “We don’t take on accounts under $10K/month that also have unclear goals and targets — without baseline clarity, neither side can tell if we’re succeeding.”

“We don’t run web-only attribution clients — we don’t have the depth there.”

Bad answer: “We work with anyone serious.” Or evasive non-answers.

An agency that takes any check that clears does not have a strategy. They have an inbox. Strategic discipline shows up most clearly in what gets refused.

Red flags during pitches and discovery calls

Pattern-match these against the agencies you are evaluating.

What we see: the slickness signal is the most reliable single tell. A pitch that feels too polished usually is.

The best agencies I have worked with — both as a buyer and as a peer — are visibly imperfect in their pitches. They show their work instead of selling around it. They will admit the things they don’t know.

When should you hire in-house instead of an agency?

The honest version: agency is not always the right answer. Three thresholds where in-house wins.

1. Spend level above $1.5M/month sustained. At this scale, the math on agency fees vs. building a 3-5 person internal team usually flips toward in-house. The exception: a specialty agency in a specific lane (creative production, a specific geo) on top of internal account ops.
2. Vertical complexity that requires deep product knowledge. If your monetization, retention loops, or core mechanics are unusual enough that a UA team needs months to understand them, in-house is faster. Agencies amortize learning across clients; you don’t get that benefit if your account is the only one of its kind.
3. You have or can hire a senior UA leader. The single biggest predictor of in-house success is whether you have a senior person who has run this work before. Without that person, in-house is a more expensive way to make the same mistakes an agency would have caught. Even with a senior leader, an agency can still complement and support — a specialty agency for creative production, channel ops, or specific geos works alongside a strong internal lead, not as a replacement for one.

The hybrid model — internal lead + agency for creative production or specific channels — is often the right answer between $300K/month and $1.5M/month spend.

Internal owns strategy and measurement. Agency owns specific execution lanes. Building in-house creative capabilities alongside an agency is increasingly common at scaled subscription apps. RocketShip’s consulting and advisory services are designed for this hybrid context — internal teams that need senior advisory without full agency lift.

What we see: the right answer depends on your spend level and your team’s UA fluency, not on a generic “use an agency” or “go in-house” prescription.

Frequently asked questions

What should a UA agency cost in 2026?

Percentage pricing has traditionally run 10 to 20% of ad spend and is under real pressure now that targeting and analysis are automated; at scale it should be well under 5% and stepping down (why the 15% fee is under pressure). Retainers for a full creative and growth programme start around $5,000 a month from serious agencies, and at $7,500 with us (what a user acquisition agency costs). Creative-only specialists quote per concept or per playable.

Avoid agencies that won’t unbundle pricing — you can’t evaluate value without seeing what you’re paying for. See our mobile app UA cost breakdown for full benchmarks.

How long should a UA agency contract be?

Three to six months is standard. Beware of any agency that wants 12+ months before any work has happened. The work should speak for itself.

Build a clear exit clause into whatever contract you sign — usually 30 days notice — so the relationship is mutual choice every month, not a hostage situation.

Should I hire one agency or split between specialists?

Depends on your spend level and internal capability. Below $300K/month, one full-service agency is usually right — coordinating multiple specialists is its own overhead cost.

Above $500K/month with a senior internal lead, splitting is often better: one creative-production specialist + one paid-channels specialist + your internal lead orchestrating.

How do I know if my current agency is performing?

Three signals. First: are CPIs and CACs trending in the direction you agreed at engagement start? Second: is creative diversity expanding (concept count, not variant count) week over week?

Third: are weekly reports diagnostic (test plan, results, next plan) or descriptive (here’s what happened)? Diagnostic reports compound. Descriptive reports do not.

Your attribution data — whether you are on AppsFlyer, Adjust, or another MMP — should be checked monthly against the agency’s reported numbers. If the agency dashboard and the MMP diverge, ask why.

Can I trust an agency with my Meta account access?

Yes, if it’s structured correctly. You own the Meta Business Manager, you own the ad account, you grant the agency admin access.

Never let an agency create the account for you. Never let them be the only admin. The account is your asset; their access is conditional.

What’s the difference between a UA agency and a creative agency?

UA agencies own paid acquisition end-to-end: campaigns, creative, measurement, reporting. Creative agencies produce ad assets but typically don’t run the account.

Many of the best modern setups split: a UA agency or in-house team running paid + a specialist creative agency producing high volume of concepts. The integration matters — both sides need shared test plans and shared learning loops.

Should I work with an agency that handles a competitor?

Yes. The whole point of hiring an agency is access to their cross-client learnings — including learnings from accounts in your category.

An agency that has never run a competitor doesn’t have the pattern recognition you’re paying for. If you’re nervous about competitive data crossing over, the right tool is an NDA — not a blanket no-compete clause.

A well-scoped NDA protects the sensitive specifics — your exact numbers, your account structure, your roadmap — while still letting the agency share the high-level learnings and pattern recognition you hired them for. If you cannot tolerate any overlap at all, hire in-house instead.

How long until I see results from a new UA agency?

The honest answer: 4-8 weeks for first signal, 12+ weeks for trust. The first 4 weeks are setup, learning, and baseline tests.

Weeks 4-8 are when their actual creative testing should start producing results vs. baseline. Anyone promising results in 2 weeks either is buying installs cheap from a low-quality channel or is being dishonest about timeline.

Related reading

• Why your top-performing ad creative should worry you — concentration risk and parallel scaling
• How to build in-house creative capabilities for mobile UA — when in-house wins
• What does mobile app user acquisition cost in 2026? — pricing and benchmarks
• How to scale mobile UA from $1K to $100K per day — scaling structure
• Dynamic creative optimization for mobile apps — when DCO helps and when it hurts
• How to choose a UA creative agency — the one question to ask: what gets tested, and how is it split
• Creative agency vs media buying agency — which lane you are actually hiring
• How much does a user acquisition agency cost — retainers, percentages and what each buys
• What percentage do UA agencies charge — why the 15% fee is under pressure
• Minimum ad spend for a performance creative agency — the arithmetic behind the $50K floor
• What a mobile measurement audit looks like — the five checks before anyone scales iOS spend
• Google Ads vs Apple Search Ads vs Meta for app installs — how to sequence channels