Originally published September 2019. Some details have changed since: Play Console’s menus have been reorganised and “global experiments” are now called default graphics experiments.
Short answer: once you know your Google Play conversion rate has room to improve, you run A/B tests on your store listing, one asset at a time, and let Play Console tell you whether a variant beat the current listing.
This post accompanies a six-minute video by Shamanth Rao, How to improve your Google Play conversion rates through AB testing. Its description says it covers the key elements of an A/B test and how to run one in the Google Play Developer Console. The video is the source for the walkthrough; this page does not reproduce it step by step.
Page Contents
How do you run an A/B test on Google Play?
You use store listing experiments in Play Console. According to Google’s Play Console Help, “Run A/B tests on your store listing”:
- You can run one default graphics experiment, or up to five localised experiments at the same time.
- Default graphics experiments test the icon, feature graphic and screenshots in your default listing language. Localised experiments also test descriptions, in up to five chosen languages.
- You set up an experiment under Grow users, Store presence, Store listings, then choose a target metric (unique user install clicks, open clicks or pre-registration clicks).
- Advanced settings include the number of variants (up to two), audience percentage, minimum detectable effect and confidence level.
- Google recommends testing one asset at a time, and experiments stop automatically after six months.
How do you read the result?
Google’s result uses both the confidence interval and minimum detectable effect (MDE). Its guide treats differences below the MDE as a draw. If the result says “More data needed”, check back later rather than calling it early.
What to do after the test
Apply a supported winning variant; keep the current listing if it performs best. A draw requires judgment, and an inconclusive result does not establish a winner. Test store listing changes alongside your ad creative, since the page people land on is part of the same funnel; see how to structure creative tests for Meta ads and, for judging your starting point, how to compare your Google Play conversion rate with your competitors.
Frequently asked questions
Can you A/B test your Google Play listing? Yes. Play Console has store listing experiments for the icon, feature graphic, screenshots and, in localised experiments, descriptions.
How many experiments can you run at once? One default graphics experiment, or up to five localised experiments, according to Google’s help page.
How long does an experiment run? Play Console estimates the time and data needed. Experiments stop automatically after six months, even if the evidence remains inconclusive; no variant is automatically applied at that cutoff.
