1241114 - (fennec-nightly-aurora-on-gp) [meta] Publish Nightly/Aurora to Google Play

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, defect)

Description

[Sebastian Kaspari (:sebastian; :pocmo)]

Updating Nightly (or Aurora) manually is somewhat annoying. But automatic updates are only available through the Play store.

Google Play has a Publishing API:
https://developers.google.com/android-publisher/#publishing

Google Play has a (public) beta/alpha program:
https://support.google.com/googleplay/android-developer/answer/3131213?hl=en

With the Publishing API we could automatically upload builds to Google play. With the beta/alpha program we can prevent that the app is publicly visible for everyone. Instead a user would have to enroll on a website. After that the app is visible and installable through Google play for this user.

Comment 1

[:Margaret Leibovic]

I think this is a good idea, since this is a big complaint we get from Nightly users, and also leads to lots of people being on old versions of the browser.

One concern is that we'll lose testing for our own updater, so if we ever want to use that for something else, there will be more risk. But we could still keep nightly.mozilla.org going for people who want non-Play options.

Barbara, what do you think? Can we go ahead and do this? We could talk to the releng team about helping us out with this.

Comment 2

[Barbara Bermes [:barbara] - NI please!]

Have we never used the Publishing API from Google because it didn't exist back then or was there a specific reason why we shipped it via our updater?

I can't think of any horrible disadvantage except for giving Google a bit more info about ourselves. However, we might also be able to get a bit more Goolge play stats from it? I just read we still would need to create a feedback channel as this will not be handled by the public Google store review feature. So no advantage there of having a tool to gather that.

We probably would have to communicate this somehow to new and existing Nightly users can use the new process. Have we already though about that migration process? I assume Google will provide the enrollment website Sebastian was talking about?

What other complaints did we get besides the auto update feature currently missing?

Comment 3

[Kevin Brosnan [Ex-Mozilla]]

Yes the Publishing API did not exist when we created Firefox for Android. There was concern about other app's trademarks, confusion and brand dilution if Aurora (DevEdition) and Nightly were searchable from the play store. When the Publishing API allowed for beta apps off the release app I brought up publishing Aurora to Firefox Beta's beta channel. When I talked to the release team about it was considered too much work. (this has changed with the scriptable interface for Google play and our increased userbase). 

Nightly will be tricky to upload as someone stole the app id org.mozilla.fennec see bug 809700 for details.

I don't see why if we have a Google Play option that we need to remove the manual updater code right away. Though I suspect that we will need a way to know if Play is controlling updates so we don't download updates outside of Play. There are a set of users that see Mozilla as the not Google option and being able to keep users up to date who don't have/use the Play store would be nice.

Comment 4

[Richard Newman [:rnewman]]

(In reply to Kevin Brosnan [:kbrosnan] from comment #3)

> I don't see why if we have a Google Play option that we need to remove the
> manual updater code right away. Though I suspect that we will need a way to
> know if Play is controlling updates so we don't download updates outside of
> Play.

And we can do so, apparently reliably: Bug 1220773.

Comment 5

[Sebastian Kaspari (:sebastian; :pocmo)]

I agree, this should not kill our updater. In fact we have been thinking about enabling the updater for beta/release users without play store: bug 1192279. Both things seem to require that we take care of our updater and google play not fighting each other (bug 1220773 and follow-ups).

Comment 6

[:Margaret Leibovic]

I agree we should not kill our updater. As a first step, I think we should keep our Nightly/Aurora downloads/updates the same, and try creating alpha/beta channels on our Firefox Beta listing.

Sylvestre, what do you think? Have you investigated this before? I think this could help us get more test coverage on Nightly/Aurora, which would help with our stability testing issues.

Comment 7

[Sylvestre Ledru [:Sylvestre]]

This would be trivial from the automation perspective. The hard part is getting people involved but not a big deal I think.

The only issue that I see would be the naming of all this:
"Aurora could be the beta program of Firefox beta" (?!) :p

Another way could be to rename Firefox beta to Firefox experimental or something and use the production channel for beta, beta for aurora and alpha for nightly (ok, it is still confusing).

Anyway, I am in! I would love to have that.

Comment 8

[:Margaret Leibovic]

Are you the right person to own this? Who should we talk to from release engineering to make this happen?

We should also make a post to release-drivers and mobile-firefox-dev to gather feedback about the right approach here.

Comment 9

[Barbara Bermes [:barbara] - NI please!]

I'd also like marketing involved in this, especially around naming, as they have their eyes on the Google Play Store as well.

NI Mike Han and Alex Davis.

Just to confirm, by keeping the updater and offering another channel for Nightly through the Google Play Store, will we still be able to keep clean track of stats around downloads, updates etc.?

Comment 10

[Sylvestre Ledru [:Sylvestre]]

Yes, that would be me. I don't think we need releng for now.
We (release management) have the automation in place to manage that and transfer it to releng when it is time.

Here it is:
https://hg.mozilla.org/mozilla-central/file/tip/testing/mozharness/scripts/get_apk.py
https://hg.mozilla.org/mozilla-central/file/tip/testing/mozharness/scripts/push_apk.py

FYI, we are updating the beta & release description daily with this command:
python update_apk_description.py --service-account foo --channel beta --store google --credentials googleplay.p12 --update-apk-description
(same location as the two scripts mentioned earlier)

Credentials can be generated by any GP admin (all release management & Lawrence).

Comment 11

[[e-mail]]

Sounds like this will not be publically available on the Google Play store?  If we are targeting the same people who are using the Android Nightly builds then I recommend we keep consistency and use the same name (Firefox for Android (Nightly))?

Comment 12

[Alex Davis [:adavis]]

(In reply to Barbara Bermes [:barbara] from comment #9)
> I'd also like marketing involved in this, especially around naming, as they
> have their eyes on the Google Play Store as well.
> 
> NI Mike Han and Alex Davis.
> 
> Just to confirm, by keeping the updater and offering another channel for
> Nightly through the Google Play Store, will we still be able to keep clean
> track of stats around downloads, updates etc.?

Let me start by saying I love the idea of making Nightly available on the Google Play Store. From an app store optimization perspective there are a lot of benefits and from an in-product A/B test perspective, there are many more benefits.

As per the product name, if I can jump in here. I would follow the naming convention of our Beta app since I recently optimized it a bit and increased our app ranking for our top keywords. I would call it: "Firefox Nightly — Web Browser". Having the words "web browser" is pretty important.

Mike, since I don't want to contradict you, please confirm that you're ok with this proposed name.

Comment 13

[:Margaret Leibovic]

To clarify, I don't think this app would have a separate product name or listing... this would just be the opt-in "beta" and "alpha" channels of Firefox Beta.

I'm not familiar with how the Play Store handles these pre-release channels, but I don't think we need to worry about discoverability of the app itself.

However, I would be interested to know if the app needs the same app id, in which case we may need to build these Aurora and Nightly versions with the beta branding. We should do some testing/investigation.

Comment 14

[Alex Davis [:adavis]]

(In reply to :Margaret Leibovic from comment #13)
> To clarify, I don't think this app would have a separate product name or
> listing... this would just be the opt-in "beta" and "alpha" channels of
> Firefox Beta.
> 
> I'm not familiar with how the Play Store handles these pre-release channels,
> but I don't think we need to worry about discoverability of the app itself.
> 
> However, I would be interested to know if the app needs the same app id, in
> which case we may need to build these Aurora and Nightly versions with the
> beta branding. We should do some testing/investigation.

I clearly misunderstood. I thought you guys were going to distribute nightly separately in the Play Store, similarly to how our release and Beta are currently their own apps with their own names and descriptions. I understand the difference now.

I imagine our nightly will be similar to Facebook's beta (opt-in): https://play.google.com/apps/testing/com.facebook.katana/
Here are the steps and explanations they provide their users: https://www.facebook.com/notes/facebook-engineering/introducing-the-facebook-for-android-beta-testing-program/10151529228878920/

Comment 15

[Mark Finkle (:mfinkle) (use needinfo?)]

The opt-in Alpha/Beta programs seem like more work and overhead for the user. Just look at the Facebook instructions.

I know we initially were looking at "Chrome Dev" which is a separate app.

Comment 16

[[e-mail]]

ahh..I misunderstood too.  If we need another name, we can go with Alex's recommendation Firefox Nightly — Web Browser

Comment 17

[Sebastian Kaspari (:sebastian; :pocmo)]

(In reply to Mark Finkle (:mfinkle) from comment #15)
> The opt-in Alpha/Beta programs seem like more work and overhead for the
> user. Just look at the Facebook instructions.

This has become much easier now for public alpha/beta programs: Just opt-in to testing and download the app from Google play. Currently the overhead/annoyance for the user is updating the app.

(In reply to :Margaret Leibovic from comment #13)
> To clarify, I don't think this app would have a separate product name or
> listing... this would just be the opt-in "beta" and "alpha" channels of
> Firefox Beta.

You can also use alpha/beta programs for unpublished apps. I used this in the past and with that the app is only visible to testers. I like the ability to have all "trains" on a device. If we use Beta then you could only have one (Beta/Aurora/Nightly) through Google Play.

(In reply to :Margaret Leibovic from comment #13)
> However, I would be interested to know if the app needs the same app id, in
> which case we may need to build these Aurora and Nightly versions with the
> beta branding. We should do some testing/investigation.

Yeah, you have to use the same app id. In fact Google play would allow you to promote an app build from alpha -> beta -> release if you want to. Thinking of that: If we actually are going to use the beta channel for this then we need to take care of creating versionCode values that do not conflict and are sequentially.

(In reply to Mark Finkle (:mfinkle) from comment #15)
> I know we initially were looking at "Chrome Dev" which is a separate app.

I proposed the alpha/beta program because I thought we might not want to have Aurora publicly on the play store for everyone. If we are okay with that then yeah, let's publish a Firefox Dev or something like that. However I'd still like to get automatic updates for Nightly too. :)

Comment 18

[Sylvestre Ledru [:Sylvestre]]

Coming back on this topic, if product thinks it is worth it, I would be happy to take the technical lead here.
I can manage the creation of the new app (Firefox Dev ?) and manage the daily uploads. We have already the scripts to do it.

Comment 19

[:Margaret Leibovic]

(In reply to Sylvestre Ledru [:sylvestre] from comment #18)
> Coming back on this topic, if product thinks it is worth it, I would be
> happy to take the technical lead here.
> I can manage the creation of the new app (Firefox Dev ?) and manage the
> daily uploads. We have already the scripts to do it.

Thank you for poking this again! I've been meaning to get back to pushing for this.

Thinking about this more, I feel like it would be simplest to just put Nightly and Aurora directly in the play store. One challenge, however, is that someone once uploaded a org.mozilla.fennec build to the Play store, and we're not sure who, so we don't have control over that package name. We may need to do some build tweaks to rename the package for upload to the Play store.

Barbara is PTO this week, but we can check in with her next week. Or if we want to move more quickly we could ask Karen this week.

Comment 20

[Sylvestre Ledru [:Sylvestre]]

> Thinking about this more, I feel like it would be simplest to just put
> Nightly and Aurora directly in the play store.
Not sure we want nightly but this is not my call :)

>  One challenge, however, is
> that someone once uploaded a org.mozilla.fennec build to the Play store, and
> we're not sure who, so we don't have control over that package name.
This would be me, sorry. I used this for various tests (automatic update of the description, split of the apk, etc).
However, we can probably delete / create it.

Comment 21

[:Margaret Leibovic]

(In reply to Sylvestre Ledru [:sylvestre] from comment #20)
> > Thinking about this more, I feel like it would be simplest to just put
> > Nightly and Aurora directly in the play store.
> Not sure we want nightly but this is not my call :)

I think it would be really nice for our Nightly testers to not need to manually update every day. This is the biggest complaint we get from them, and also we want to do everything we can do make sure people are testing the latest builds.

We should give marketing a heads up about making new Play store listings, but we should make it very clear that these are early development builds for testing, not for general purpose use. I'd be interested to see if we can somehow downplay these in the store if they're coming from the official Mozilla account. Also, these build will get ratings, which is a little weird. Will bad ratings for these apps have an impact on the overall impression of Mozilla's account? Maybe we should explore the idea of an alternate account to publish these developer builds, like "Mozilla Developer"?

Comment 22

[[e-mail]]

thanks for the heads up here.  Though we cant dictate customer star ratings, we can be overly communicative that these are potentially unstable, pre-beta builds in our messaging touchpoints to set expectations.  How many users do we expect to be using this build?

Comment 23

[:Margaret Leibovic]

(In reply to [e-mail] from comment #22)
> thanks for the heads up here.  Though we cant dictate customer star ratings,
> we can be overly communicative that these are potentially unstable, pre-beta
> builds in our messaging touchpoints to set expectations.  How many users do
> we expect to be using this build?

I'm having a tough time finding consistent Nightly/Aurora ADI/DAU numbers, but it should be on the order of thousands.

Comment 24

[Dirkjan Ochtman (:djc)]

FWIW, I'm one anecdotal Nightly-on-desktop user who runs Beta on Android just because that's on the Play Store. I would definitely run Nightly on my phone if it was more easily available there (i.e. via the Play Store).

Comment 25

[Nicholas Nethercote [inactive]]

> I'm having a tough time finding consistent Nightly/Aurora ADI/DAU numbers,
> but it should be on the order of thousands.

Today's ADI numbers for Fennec from arewestableyet.com:
- Nightly 1.1k
- Aurora  1.6k
- Beta    136k
- Release 6.1M

In my experience the ADI numbers on arewestableyet.com bounce around quite a bit, but this should give you a rough idea.

Comment 26

[[e-mail]]

Hi,

Wanted to get feedback on the following:

1.  Adding a banner to the screenshots (only 2-3% of people ever see this):
"This is a developmental version of Firefox.  
Your experience may be unstable."

2.  Long description (taken from cnet download page) and other links are from our firefox beta.. please confirm these are the right links we want to use:

Firefox Aurora is a developmental channel for new Mozilla Firefox releases. 

Download the GA version of Firefox here: <link>

Firefox Aurora is designed to showcase the more experimental builds of Firefox. The Aurora channel allows users to experience the newest Firefox innovations in an unstable environment and provide feedback on features and performance to help determine what makes the final release.

Find a bug? Report it at http://mzl.la/android_bugs.

Want to know more about the permissions Firefox requests? http://mzl.la/Permissions

See our list of supported devices and latest minimum system requirements at https://www.mozilla.org/en-US/firefox/mobile/platforms/.

Mozilla marketing: In order to understand the performance of certain Mozilla marketing campaigns, Firefox sends data, including a Google advertising ID, IP address, timestamp, country, language/locale, operating system, app version, to our third party vendor. Learn more by reading our Privacy Notice here: https://www.mozilla.org/en-US/privacy/firefox/

3.  Short description:
For Developers - Be the first to test future releases of Firefox 

4.  Title:
Firefox Aurora- for Developers

Comment 27

[Sylvestre Ledru [:Sylvestre]]

catlee, Sebastian, I tried to push the latest 50.0a2 to Google play and I am getting "Version 2015398425 is not served to any device configuration: all devices that might receive version 2015398425 would receive one of versions 2015442756, 2015442753."
https://ftp.mozilla.org/pub/mobile/nightly/latest-mozilla-aurora-android-x86/fennec-50.0a2.multi.android-i386.apk
 	fennec-50.0a2.multi.android-i386.apk 	51M 	19-Sep-2016 09:26
https://ftp.mozilla.org/pub/mobile/nightly/latest-mozilla-aurora-android-api-15/fennec-50.0a2.multi.android-arm.apk
 	fennec-50.0a2.multi.android-arm.apk 	42M 	19-Sep-2016 09:14

I guess they need to have the same version, am I correct?

Comment 28

[Sylvestre Ledru [:Sylvestre]]

I reported this bug https://github.com/mozilla-l10n/stores_l10n/issues/71 to get the various translations

Comment 29

[Chris AtLee [:catlee]]

I don't think they need to have the same version, but the version always needs to go up.

What is the app name for fennec on aurora? Is it perhaps conflicting with some other product?

Comment 30

[Sylvestre Ledru [:Sylvestre]]

The name of the app is "org.mozilla.fennec_aurora". I would get a different error message if it was about that.

OK, it is because we uploaded a different version in the alpha channel, this conflicted with the work I was doing. Sorry about the noise.

Comment 31

[Sylvestre Ledru [:Sylvestre]]

I just published it.

Comment 32

[Sylvestre Ledru [:Sylvestre]]

here it is:
https://play.google.com/store/apps/details?id=org.mozilla.fennec_aurora

Comment 34

[Johan Lorenzo [:jlorenzo]]

Fennec Aurora 53.0a2 was the first to be published to Google Play, entirely within Taskcluster. Yesterday was also the day where Fennec Nightly is publish in lieu of Aurora (bug 1357808), the latter being discontinued (Dawn project)

Thanks to everybody who participated!