Skip links
ASO Case Study

the advantages of A/B testing ASO screenshots

COUP, a major urban mobility player, is an electric scooter provider available in 3 major European cities: Berlin, Madrid and Paris. Everyday, they provide hundreds of rides and is considered as the biggest scooter sharing app in Europe. Our relationship started in April 2019, when they required support regarding their ASO projects. Our goals were to improve their app ranking as well as providing the set of screenshots every month to find the perfect designs that would fit the best its users.

Before:

This is the ‘white’ version set of ASO screenshots that COUP approached us with. This set of screens are modern, clean and clear, however, we felt they were too minimal and lacked the depth and excitment that would capture the attention of a user and persuade them to download the app. Our mission was to therefore diversify the type and range of screenshot sets that could be tested. Our design goal was to introduce more dynamic imagery and colour.

Why is A/B testing crucial for ASO?

ASO (app store optimization) is a service that Customlytics provides under our Marketing Service. With over 2 million apps available today, it is becoming increasinly difficult to hit a high number of views and downloads. ASO is a process that we use to help apps improve the ranking of their mobile applications (apps) directly within the app stores – Apple App Store and the Google Play Store.


As part of the ASO process it is possible to A/B test the icon, app name, subtitle, screenshots, and the short and long descriptions. Here we will be specifically discussing the process of A/B testing the screenshots. This involves comparing two sets of screenshots against each other to determine which performs better. It is essentially an experiment to compare different elements to help identify the preferences of users. For example, you can test and compare elements such as colour, the size and type of typography, the imagery and the story told through the selection of screens from the app.

For Coup, our goal for A/B testing was to develop two opposing sets of screenshots that were both on point to COUP’s brand. In order for us to be able to clearly identify which specific design elements would lead to increased conversion rates, it was important to set clear parameters.

We asked our design team to focus on three main elements – colour, graphic layout and imagery. In each set, these elements would have to oppose each other. For example, one would use COUP’s Mint Green and the other Coup’s Blue; one would have a solid background whilst the other would use illustration and maps.

By setting these parameters we were able to ensure the return of constructive and effectual results that would tell us exactly what COUP’s users liked. This information is crucial for both us and our client, COUP, in order to understand future ASO requirments.

General design considerations:

Imagine you only have 8 seconds to pitch your app to someone to download it… How do you do it?

A picture is worth a thousand words. And that’s where app screenshots come in.

Humans are visual creatures; design, layout, hierarchy, typography and color all influence the way in which we perceive and react to an app’s listing. Screenshots paint a clear picture of what the app actually does. It is the role of our design team to design a visual story that highlights what the app offers the user and how the user moves through it.

Our design checklist:

  • Design the screenshots in accordance to the brand and style requirements provided by the client (logo, colour, typography, imagery)
  • Follow the app store screenshot guidelines – both Apple and Google have detailed instructions
  • The design should have a sense of hierarchy, displaying the app’s core features and unique selling points first
  • The design of the first 3 screenshots is crucial to spark the viewers attention and interest – you have 3 to 5 seconds to captivate the user
  • Use engaging captions – text should always be legible, short and to the point
  • Localize the screenshot designs – adapt the language and imagery to match the country/city of the user.
  • Highlight important UI elements to show the app’s functionality
  • Use the latest device models: iPhone 8, iPhone XS Plus, Android Pixel 2XL

After:

Test 1

The focus of the test 1 test design was getting from A to B. A map and line showing a route (cohesive with the app interface) were used in the background to give the information context. Some UI elements were highlighted to create a 3D look for visual appeal. The element of the pin (also consistent with the app UI) is repeated over the screens.

All screens were given a clear message and purpose – either to show a real-life scenario of riding a COUP scooter or showcasing the in-app experience. Our goal of incorporating the situational screens was to provide potential users with an emotional response – to see themselves riding a COUP. The other more information-focused screens were used to educate the users about the app’s look and functions and the cities where Coup is available.

Test 2

The goal for test 2 was to transfer the visual language of the COUP homepage (www.joincoup.com) to the app store platform. By adopting the already well-known design style, we wanted to minimize irritation so that the users could focus on new information.

We designed the structure of the screens in three clear parts to give the user a complete overview of the product. Screens 1 to 4 for a visual emotional response, screens 5 to 7 for a visual informative response (front facing device for content focus) and screen 8 as a reminder of the product advantages and a final call to action.

Test 3

We approached the design of test 3 to focus on the rider’s feelings and experience when using COUP, rather than the app experience as we had done so in previous versions. The most challenging aspect of designing this test was the brief provided by COUP – to think “outside of the box” and be “radical”, whilst still staying true and close to the COUP brand and style. 

So, rather than a complete design overhaul, we had to instead pair back and conceptualize a slight shift or twist and create something unexpected. We did this by selecting imagery that was quirky and unexpected, and that represented speed and mobility to visually represent how a Coup rider can fly independently with power.

The results:

Out of all the screenshots tested, our expectations of which would be the most successful before launching them in the app store was test 2. However, at the end of the test phase the most successful was test 1. This clearly proves the importance of A/B testing in order to gain a better insight and understanding into the preferences of users. The results showed us a clear differentiation with the blue version bringing a significant uplift in conversion to app installs.

In our next test phase, we will be testing test 3 against the winning test 1. Stay tuned for the results!..