1396092 - No Nightly updates since 2017-08-29 for android-api-16 (was 15)

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Jan Henning [:JanH]]

At a guess this is because of the renaming from API-15 to API-16 in bug 1384482.

Not sure whether both Nightly versions (new/old package ID) are affected, I've tested with the old one (org.mozilla.fennec).

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Could you clarify if you mean the multi-locale build via Google Play, or a single-locale builds via Mozilla's update server ? Some steps to reproduce would be handy.

Comment 2

[Jan Henning [:JanH]]

Sorry for being unclear about this.

I'm referring to Nightly builds installed from http://ftp.mozilla.org/pub/mobile/nightly/, no matter which combination of multi/en-US and "old-id" (org.mozilla.fennec)/not "old-id" (org.mozilla.fennec_aurora).
When installing an older version, the built-in updater installs the 2017-08-29 build. Afterwards, I'm no longer getting any update notifications and manually checking on about:firefox only yields "No updates available".

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Johan, looks like there is a android-api-16 platform in Fennec-mozilla-central-nightly-latest with the latest data, rather than putting it in Android_arm-eabi-gcc3. There must be some deploy step for https://hg.mozilla.org/build/tools/rev/a22f7926f4d85c5e00b2ee9a112e20af8aea1b35 into the funsize workers that we're missing.

Comment 4

[Johan Lorenzo [:jlorenzo]]

It seems scriptworkers aren't update with the latest version of the repo. We only clone it[1]. :sfraser confirmed it by looking at the logs. So the hotfix is to pull the latest changes on every balrog_scriptworker. I'm not sure what's the best to keep a repo updated with Puppet. Keeping the NI to look into it more deeply.


[1] https://dxr.mozilla.org/build-central/rev/fb373bfb7a7d2b81af727cb5bfda5ff9aaf1b2cb/puppet/modules/balrog_scriptworker/manifests/init.pp#112

Comment 5

[Johan Lorenzo [:jlorenzo]]

I connected to balrogworker-{1,2} to manually update build/tools. The repos both went from https://hg.mozilla.org/build/tools/rev/d1572c366de1 to https://hg.mozilla.org/build/tools/rev/574d2cbc9956.

I'll have a look at next nightly (which starts in 3 minutes) to see if that was the right fix.

Comment 6

[Johan Lorenzo [:jlorenzo]]

Comment 5 was enough to upload the right blob[1]. The latest run[2] uploaded blobs to Android_arm-eabi-gcc3 instead of android-api-16, like yesterday[3].

I see a couple of ways to prevent this from happening again:
* Maybe we should make balrogscript fail if we don't detect a known blob.
* Or like said in comment 4, let puppet update the repo. 

Anyway, let's implement this in a follow up.

[1] https://aus4-admin.mozilla.org/releases#Fennec-mozilla-central-nightly-latest
[2] https://public-artifacts.taskcluster.net/Lwd3mAvlQwulMUc0sUywag/0/public/logs/live_backing.log
[3] https://public-artifacts.taskcluster.net/UX23dH4sTLCPHuvy_YpxUg/0/public/logs/live_backing.log

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Splendid! I cleaned android-api-16 and android-aarch64 out of Fennec-mozilla-central-nightly-latest, and android-api-16-old-id out of Fennec-mozilla-central-old-id-nightly-latest.