Create a A/B test for click rate on Firefox for iOS with and without an Apple logo.



3 years ago
2 years ago


(Reporter: edwong, Assigned: edwong)


Firefox Tracking Flags

(Not tracked)


User Story

Services team would like to A/B test Apple logo click rate to support bug 1199354


(1 attachment)

Comment hidden (empty)


3 years ago
User Story: (updated)

Comment 1

3 years ago
:rfeeley, please provide copy and mocks for this the A/B snippets test
Flags: needinfo?(rfeeley)

Comment 2

3 years ago
:edwong to talk to cmore about snippet vs funnel cake test.
Flags: needinfo?(edwong)
Created attachment 8693597 [details]

Text only: 
Download __Firefox for Android__ or __Firefox for iOS__ to sync with your mobile device.

With icons: 
Download Firefox for __[icon] Android__ or __[icon] iOS__ to sync with your mobile device.
Flags: needinfo?(rfeeley)

Comment 4

3 years ago
Hi Jean, do you know how we can schedule a snippet to test download rate with and without icons?
Flags: needinfo?(edwong) → needinfo?(jcollings)
Hi Ed,

Unfortunately, for snippets we always have an icon so we wouldn't be able to test just text only. However, I know that Alex Davis is running some tests on the iOS and Android product pages that a) have the logos or b) do not have logos. I believe his tests may help your case. 

I'll add Alex here, and if his tests aren't what you are looking for - we can possibly look at using snippets on the beta release that could work like this:
1) General mobile icon with "Download Firefox for Android to sync with your mobile device."
2) Android icon with "Download Firefox for Android to sync with your mobile device."
3) General mobile icon with "Download Firefox for iOS to sync with your mobile device."
4) iOS icon with "Download Firefox for Android to sync with your mobile device."
Flags: needinfo?(jcollings) → needinfo?(adavis)
Sorry for the delay, back from my honeymoon and Mozlando.

We ran a test on the Android product page about logo vs no logo:

During this test, we did user testing too where we consistently heard from users that the Google Play store logo provided confidence and trust. When it was not there, users were a bit more hesitant. (which is why we are preparing a second test)

It has led us to prepare the following test where we make sure that the logos are visible even if we use a send to device form:

In another test I did with the heartbeat team, we created a landing page promoting our mobile apps. We observed that when the Google Play and App Store buttons were on a page, it triggered a majority of users to just pull their phones out of their pockets and to download the app directly from there (rather than click on the buttons and visit the stores from desktop). There was no control for this one but I don't think users would intuitively reach for their phones in such high volumes without the logos or buttons on the page.

I hope this helps.
Flags: needinfo?(adavis)

Comment 7

3 years ago
From Alex's comments, it's clear using a logo increase clicks. This supports our use in bug 1199354
Assignee: jcollings → edwong
Last Resolved: 3 years ago
Resolution: --- → FIXED

Comment 8

3 years ago
NI: nchapman - can you approve this from product perspective and add this to 'organizational memory' per karlof's recommendation.
Flags: needinfo?(nchapman)
Yes approved (somewhat retroactively now). Will do.
Flags: needinfo?(nchapman)
You need to log in before you can comment on or make changes to this bug.