1135781 - generate builds per checkin on beta/release/esr that allow installation of unsigned addons

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Chris AtLee [:catlee]]

Addon developers will need a version of beta/release Firefox that does allow installation of unsigned addons.

We spoke with mconnor a few weeks ago and agreed that having en-US builds produced per push on beta/release with the appropriate build configs set would be sufficient for now.

Comment 1

[Chris AtLee [:catlee]]

In buildbot land, this is a parallel set of desktop platforms, with different mozconfigs. Perhaps they should be a different product as well.

Comment 2

[Chris AtLee [:catlee]]

These builds will not have updates enabled.

Comment 3

[Mike Connor [:mconnor]]

So... there's definitely a strong ask to have updates.  It's okay if that takes a little longer, but we'll almost certainly need to figure out the update story in finite time.  Given that Fx41 will be the first release where there's no pref, that's when we'll need/want updates enabled.

Comment 4

[Kim Moir [:kmoir] ET]

:mossop What is the compile time pref that enables/disables the ability to run unsigned add ons?

Comment 5

[Dave Townsend [:mossop]]

(In reply to Kim Moir [:kmoir] from comment #4)
> :mossop What is the compile time pref that enables/disables the ability to
> run unsigned add ons?

The preprocessor is looking for MOZ_REQUIRE_SIGNING though I just realised I never included adding that into moz.build so that will have to be done:

http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/extensions/internal/XPIProvider.jsm#7850

Comment 6

[Kim Moir [:kmoir] ET]

Could we limit the number of platforms we build this on to a smaller subset that usual. ie. Windows 8, OSX 10.10, Linux 64?

Comment 7

[Kim Moir [:kmoir] ET]

:mossop what moz.build file should this be added into to enable this?

Comment 8

[Dave Townsend [:mossop]]

I just put up a patch for bug 1163046 which will make signing mandatory in branded beta and release builds. I understand that the plan for these developer builds is for them to be unbranded so they will allow users to turn off signing via the runtime pref with no extra work.

Comment 9

[Chris Cooper [:coop] (he/him)]

 (In reply to Kim Moir [:kmoir] from comment #6)
> Could we limit the number of platforms we build this on to a smaller subset
> that usual. ie. Windows 8, OSX 10.10, Linux 64?

I don't think we need to. Check-in volume on beta/release/esr is negligible compared to other branches. Also, we'll need to disable any addon-mitigated testing on the signed addon builds, so we're not doubling the testing requirements here.

Talking with catlee on IRC, it would be fine to get a single platform (say linux64) setup first as a proof-of-concept.

Comment 10

[Jordan Lund (:jlund)]

Attachment: 150712_1135781_no_add_on_signing_builds-linux64_mozconfig-gecko-tree.patch

gecko tree mozconfig for linux64 no add-on signing required.

my naive assumption that this will override https://dxr.mozilla.org/mozilla-central/source/browser/confvars.sh#76 even if used on mozilla-beta

once tested and reviewed, this will land on m-c and get uplifted to m-b

Comment 11

[Jordan Lund (:jlund)]

Attachment: 150712_1135781_no_add_on_signing_builds-linux64_m-b-buildbot-config.patch

this adds a linux64 no addon signing required build that runs along side its linux64 opt counterpart.

it will ride the trains and eventually stop at esr38.

untested for now. uses new mozconfig from https://bugzilla.mozilla.org/show_bug.cgi?id=1135781#c10

Comment 12

[Jordan Lund (:jlund)]

Hey Kim, I played around with this today.

staging master: http://dev-master2.bb.releng.use1.mozilla.com:8037/builders

user-mozilla-beta: http://hg.mozilla.org/users/jlund_mozilla.com/mozilla-beta

linux64 slave: dev-linux64-ec2-012 https://bugzil.la/1182930

buildbot-configs used in staging:
   - repo: https://github.com/lundjordan/build-buildbot-configs/tree/no-addon-sign-dev
   - based off everything in my no-addon-sign branch + configured for staging (my loaned slave and branches m-b, m-r, and esr38)

the above master is currently running a linux64-no-add-on-sign build for mozilla-beta based on mozconfig and buildbot-config patch in comment's 10 and 11.

I also started an etherpad to track notes and deployment strategy: https://etherpad.mozilla.org/no-add-on-signed-builds

I'll check back on the builds tomorrow.

Comment 13

[Jordan Lund (:jlund)]

Attachment: 150714_1135781_no_add_on_signing_builds-linux64_m-b_ride_trains-buildbot-config.patch

version 2. this one allows us to ride the trains once 41 hits beta and will stop when 41 hits esr38, automagically


next steps:

- switch tests from existing builds to use these builds
- add other platforms (win32, win64, linux32, osx64, + debug equivalents)

Comment 14

[Jordan Lund (:jlund)]

I'm having trouble trying to flip on MOZ_REQUIRE_SIGNING. I found: https://bugzilla.mozilla.org/show_bug.cgi?id=1164168#c19 which landed on beta and includes all patches for MOZ_REQUIRE_SIGNING except it doesn't include the bits that actually force it for beta/release/esr.

I found those bits that force it in the original patch (that landed on central) https://hg.mozilla.org/mozilla-central/rev/22761f1474f0 and cherry-picked it to my user mozilla-beta repo: http://hg.mozilla.org/users/jlund_mozilla.com/mozilla-beta/rev/001ead1cfc2e

then did a staging build with that http://dev-master2.bb.releng.use1.mozilla.com:8037/builders/Linux%20x86-64%20mozilla-beta%20build/builds/377

finally, I poked the objdir but it seems that MOZ_REQUIRE_SIGNING was never set:

[[e-mail] m-beta-l64-0000000000000000000]# grep MOZ_REQUIRE_SIGNING build/obj-firefox/config.status
    (''' MOZ_REQUIRE_SIGNING ''', r'''  '''),


not sure what I am doing wrong at this point

Comment 15

[Jordan Lund (:jlund)]

ah, polling MOZ_UPDATE_CHANNEL (what we condition off of: https://hg.mozilla.org/mozilla-central/rev/22761f1474f0#l1.15) shows us that our checkin builds are not set everytime:

[[e-mail] m-beta-l64-0000000000000000000]# grep MOZ_UPDATE_CHANNEL build/obj-firefox/config.status     (''' MOZ_UPDATE_CHANNEL ''', r''' default '''),

where default comes from http://mxr.mozilla.org/mozilla-central/source/configure.in#4004

iiuc - we only set that env var for 'nightly' builds (m-c,m-a only): http://mxr.mozilla.org/build/source/buildbotcustom/misc.py#1858

I think we will need to condition off a different var maybe something like:
    (''' MAR_CHANNEL_ID ''', r''' firefox-mozilla-beta '''),

or maybe this is something that can be pref'd on or off in mozconfigs...

point is, I don't think https://hg.mozilla.org/mozilla-central/rev/22761f1474f0#l1.13 will work. it will still allow unsigned add-ons to be installed on our beta builds iiuc. Apologies if I am being to verbose. Just want to sanity check as I come familiar with all of this.

Comment 16

[Jordan Lund (:jlund)]

> iiuc - we only set that env var for 'nightly' builds (m-c,m-a only):
> http://mxr.mozilla.org/build/source/buildbotcustom/misc.py#1858

correction, and actual releases too: http://mxr.mozilla.org/build/source/buildbotcustom/process/release.py#705

so we just don't set that for our per-checkin continuous integration builds.

Comment 17

[Jordan Lund (:jlund)]

mossop: see comment 14-16

I believe we need to find an alternative to how we have implemented MOZ_REQUIRE_SIGNING, at least the confvars.sh bits that check branch with MOZ_UPDATE_CHANNEL. bhearsum has informed me that even that won't be reliable for releases soon.

Maybe instead this should be a top mozconfig item and we turn it on or off at for each platform/branch mozconfig. Though I'm not sure the impact that would have on your current implementation.

gps, do you have any thoughts here as the reviewer from https://bugzil.la/1164168

Comment 18

[Kim Moir [:kmoir] ET]

Attachment: WIP bug1135781.patch for additional platforms

Comment 19

[Dave Townsend [:mossop]]

(In reply to Jordan Lund (:jlund) from comment #17)
> mossop: see comment 14-16
> 
> I believe we need to find an alternative to how we have implemented
> MOZ_REQUIRE_SIGNING, at least the confvars.sh bits that check branch with
> MOZ_UPDATE_CHANNEL. bhearsum has informed me that even that won't be
> reliable for releases soon.

I don't care how we do it, up to build config peers and releng really.

Comment 20

[Gregory Szorc [:gps]]

Generally speaking, making things conditional on MOZ_UPDATE_CHANNEL, MOZ_OFFICIAL_BRANDING, MOZ_BRANDING_DIRECTORY, or any other of the settings we turn on for release builds is a bit fragile. I think it's OK to turn on things if these variables are present. But hiding settings behind them can lead to pain like is felt in this bug.

I think adding a real flag to configure and having the mozharness config or in-tree mozconfigs pass this flag for certain build configurations is the right way forward. i.e. remove the references to MOZ_OFFICIAL_BRANDING and MOZ_UPDATE_CHANNEL from browser/confvars.sh.

Comment 21

[Jordan Lund (:jlund)]

okay thanks.

So iiuc, we have a prerequisites before even tackling this bug:

1) in addition to releases, we need to get beta 41 per-checkin jobs to force 'require add-on signing'. Because, as it stands, https://bugzil.la/1164168 will only force 'require add-on signing' for release jobs (not per-checkin).
    - platforms that should set this:
      - desktop Firefox, and Android Fennec (I'm assuming this includes all variants like asan, debug, etc)?
    - platforms that should ignore/not set this:
      - Thunderbird, Seamonkey, b2g desktop builds?

2) confirmed by gps: rather than using MOZ_OFFICIAL_BRANDING and MOZ_UPDATE_CHANNEL, add configure flags for MOZ_ADDON_SIGNING and MOZ_REQUIRE_SIGNING and use those for the platforms mentioned above in upcoming beta 41

3) rename MOZ_REQUIRE_SIGNING to MOZ_REQUIRE_ADDON_SIGNING to distinguish between requiring Firefox to be signed vs the addons themselves


mossop: can you sanity check part 1 above or point me to someone that would know? Also, what does MOZ_ADDON_SIGNING do? Does it just check that add-ons are signed while MOZ_REQUIRE_SIGNING actually enforces that they are signed?

Comment 22

[Dave Townsend [:mossop]]

(In reply to Jordan Lund (:jlund) from comment #21)
> okay thanks.
> 
> So iiuc, we have a prerequisites before even tackling this bug:
> 
> 1) in addition to releases, we need to get beta 41 per-checkin jobs to force
> 'require add-on signing'. Because, as it stands, https://bugzil.la/1164168
> will only force 'require add-on signing' for release jobs (not per-checkin).
>     - platforms that should set this:
>       - desktop Firefox, and Android Fennec (I'm assuming this includes all
> variants like asan, debug, etc)?
>     - platforms that should ignore/not set this:
>       - Thunderbird, Seamonkey, b2g desktop builds?

This looks correct with one modification. We've decided to slip requiring signing by one release so we should only 'require add-on signing' in 42. I was going to file a bug to make that change to the current config but if you're changing all that anyway I guess you might as well roll that in.

Comment 23

[Kim Moir [:kmoir] ET]

Attachment: bug1135781mozconfigs.patch

mozconfigs for desktop and android builds now that changes in bug 1186522  have landedc

Comment 24

[Kim Moir [:kmoir] ET]

Attachment: bug1135781-2.patch

Comment 25

[Jordan Lund (:jlund)]

Attachment: 150829_add-on-signing_enable_linux-add-on-devel_builders_to_beta-bbot-cfgs.patch

kim, this combines our previous couple bbot-cfg patches and:

* enables only linux* add-on-devel builders to beta for proof of concept
* locks these builders to beta for now till we are ready to ride trains
* disables sendchanges completely so we don't run uneeeded dupe tests
* fixes indent/whitespace


interdiff since your patch: http://people.mozilla.org/~jlund/add-on-devel-bbot-cfg-interdiff.patch

buid_master diff from clean checkout: http://people.mozilla.org/~jlund/enable_linux-add-on-devel_builders_to_beta

this patch requires before landing:
* the mozconfig patch to land and be uplifted to beta
* ensure our infra (e.g. cloud-tools, th) knows about these new builders

Comment 26

[Kim Moir [:kmoir] ET]

Comment on attachment 8654540 [details] [diff] [review]
150829_add-on-signing_enable_linux-add-on-devel_builders_to_beta-bbot-cfgs.patch

looks good, I'll attach a patch in bug 1199745 (treeherder) and (bug 1200346) watch_pending.cfg

Comment 27

[Randell Jesup [:jesup] (needinfo me)]

Any word on when this will be resolved and available to developers?  We state in the wiki it "will" be available; not having this I'm sure causes huge pain for said developers, and blocks (for example) external people using extensions in test harnesses (as we do as well) from running beta or release tests.

Comment 28

[Kim Moir [:kmoir] ET]

I'm at a work week focusing on release promotion right now.  I ran a try run for this but it failed. I'll look at this again today once I land some more release promotion patches.

Comment 29

[Kim Moir [:kmoir] ET]

Most of the the comments regarding testing the builds to install unsigned addons are in bug 1186522.

Comment 30

[Hector Zhao [:hectorz]]

(In reply to Kim Moir [:kmoir] from bug 1186522 comment #132)
> 
> there is still the work to generate unbranded builds with the pref turned
> off in bug 1135781, I'm having problems with the branding on that, still
> working on it.

Hello, when can I expect these builds to be available? Thanks.

Comment 31

[Kim Moir [:kmoir] ET]

making some progress on this, iterating through failures on try runs

Comment 32

[Kim Moir [:kmoir] ET]

Last night I ran a try run for an unbranded build with the signing preferences disabled.  The signing preferences work, the unbranded build did not.  It is still branded as nightly. I'm running the try run against the mozilla-beta repo.

Here is my try run from last night

https://treeherder.mozilla.org/#/jobs?repo=try&revision=95e9c0bd6e46c3c6779589ce2faedcf5816a6b22

Here are the changes
https://hg.mozilla.org/try/rev/95e9c0bd6e46c3c6779589ce2faedcf5816a6b22

Do I need to adjust the whitelist so that the builds use browser/branding/unofficial branding? I'm stuck on how to enable an unbranded build.  Obviously when these unbranded builds are enabled in automation they will have separate mozconfigs etc.

Comment 33

[Michael Shal [:mshal]]

Not sure if you saw my ping in IRC - I believe the branding in this build is in fact using the unofficial branding. Looking at firefox-branding.js in browser/omni.ja, you can see this:

// Give the user x seconds to react before showing the big UI. default=24 hours
pref("app.update.promptWaitTime", 86400);

Which matches the lines in the unofficial branding:

https://dxr.mozilla.org/mozilla-beta/rev/609000bcc11211d7c27ceea36fa2d2262fa0523f/browser/branding/unofficial/pref/firefox-branding.js#8

The nightly branding has a wait time of 2 hours:

https://dxr.mozilla.org/mozilla-beta/rev/609000bcc11211d7c27ceea36fa2d2262fa0523f/browser/branding/nightly/pref/firefox-branding.js#8

Let me know if I'm misunderstanding the issue!

Comment 34

[Kim Moir [:kmoir] ET]

Okay, thanks for the clarification. That is very confusing.  I assumed that build would be lacking all firefox images as an unbranded build. So every time my try run build had nightly images, I assumed it was wrong and the configs had to be updated.

I was releasing b5 on Friday morning and then rushed off to catch my plane home, that's how I missed your ping.

10:26 AM <mshal> kmoir: I think the build in 1135781 might be unbranded... it looks like the "unbranded" content still calls itself Nightly in the graphics and such
M <mshal> err, "unofficial", not "unbranded"
10:27 AM <mshal> the firefox-branding.js file in your build matches the one in the unofficial directory I believe

I'll make run another try run with patches for the other platforms to make sure that the binaries reflect the state that we are looking for and then add them as a periodic build to automation.

Comment 35

[Kim Moir [:kmoir] ET]

I've verified that the mac build works from the latest try run.  Here are the the builds from other platforms if others are interested in verifying them

https://treeherder.mozilla.org/#/jobs?repo=try&author=[e-mail]&selectedJob=20890116

Linux64 http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-7233a93f13b9aeaa358866dd1be88f8504c5ecda/try-linux64/firefox-47.0.en-US.linux-x86_64.tar.bz2

Win32 http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-7233a93f13b9aeaa358866dd1be88f8504c5ecda/try-win32/firefox-47.0.en-US.win32.zip

win64 http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-7233a93f13b9aeaa358866dd1be88f8504c5ecda/try-win64-debug/irefox-47.0.en-US.win64.zip

macosx http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-7233a93f13b9aeaa358866dd1be88f8504c5ecda/try-macosx64/firefox-47.0.en-US.mac.dmg


I'll attach patches to build these in automation.  Probably just opt win64, win32, and macosx, not all the variants of these builds.

Comment 36

[Kim Moir [:kmoir] ET]

Attachment: bug1135781bbconfigs.patch

wip patches

Comment 37

[Kim Moir [:kmoir] ET]

Attachment: bug1135781bbconfigs.patch wip

Comment 38

[Kim Moir [:kmoir] ET]

Attachment: bug1135781intree.patch wip (m-b)

Comment 39

[Kim Moir [:kmoir] ET]

running tests in staging now to generate the builds

Comment 40

[Mindaugas <LA-MJ>]

(In reply to Kim Moir [:kmoir] from comment #35)
> I've verified that the mac build works from the latest try run.  Here are
> the the builds from other platforms if others are interested in verifying
> them
> 
> https://treeherder.mozilla.org/#/jobs?repo=try&author=kmoir@mozilla.
> com&selectedJob=20890116
> 
> Linux64
> http://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-
> 7233a93f13b9aeaa358866dd1be88f8504c5ecda/try-linux64/firefox-47.0.en-US.
> linux-x86_64.tar.bz2
> 

Here's linux64:
http://storage8.static.itmages.com/i/16/0520/h_1463749650_1551499_ad7ee926ec.png

I assume nightly=unbranded?

> I'll attach patches to build these in automation.  Probably just opt win64,
> win32, and macosx, not all the variants of these builds.

linux64 would be welcome as well

Comment 41

[Kim Moir [:kmoir] ET]

Attachment: bug1135781intree-2.patch wip (m-b)

Comment 42

[Kim Moir [:kmoir] ET]

Attachment: bug1135781bbconfigs-2.patch wip

Comment 43

[Kim Moir [:kmoir] ET]

Attachment: bug1135781bbconfigs-3.patch

Comment 44

[Kim Moir [:kmoir] ET]

Comment on attachment 8760931 [details] [diff] [review]
bug1135781bbconfigs-3.patch

diff old new
9a10
> Linux x86-64 add-on-devel mozilla-beta build SigningScriptFactory
13a15
> OS X 10.7  add-on-devel mozilla-beta build SigningScriptFactory
15a18
> WINNT 5.2 add-on-devel mozilla-beta build SigningScriptFactory
17a21
> WINNT 6.1 x86-64 add-on-devel mozilla-beta build SigningScriptFactory

Comment 45

[Kim Moir [:kmoir] ET]

Comment on attachment 8760931 [details] [diff] [review]
bug1135781bbconfigs-3.patch

as a note, there are patches in bug 1186522 to re-enable the signing pref on beta

Comment 46

[Jordan Lund (:jlund)]

Comment on attachment 8760931 [details] [diff] [review]
bug1135781bbconfigs-3.patch

Review of attachment 8760931 [details] [diff] [review]:
-----------------------------------------------------------------

looks good. we'll need a custom mozharness config for each platform now though because of release promotion!

::: mozilla/config.py
@@ +498,5 @@
> +            'unittest_platform': 'linux64-opt',
> +            'app_name': 'browser',
> +            'brand_name': 'Minefield',
> +            'base_name': 'Linux x86-64 add-on-devel %(branch)s',
> +            'mozconfig': 'linux64/%(branch)s/add-on-devel',

I don't think this will be used by mozharness. when I wrote https://bug1135781.bmoattachments.org/attachment.cgi?id=8654540 we were still doing CI through buildbot on beta.  but now that we are completely ported to mozharness on every branch, we will want to have a custom mozharness config that defines things like platform name, mozconfig, etc.

almost all of these platform items are for factory based builds aside from things like mozharness_python and mozharness_desktop_build.

@@ +3254,5 @@
>          if platform not in ['linux64-debug']:
>              continue
>          del branch['platforms'][platform]
>  
> +# mozilla-central's gecko version

s/mozilla-central/mozilla-beta/

@@ +3262,5 @@
> +#      1) rm all no-add-on-sign builds from any branch less than m-b's gecko_version
> +#      2) rm all no-add-on-sign builds from any branch at least m-b's gecko_version
> +# Note: for now, we will lock this to m-b but eventually we will want to replace mb_gecko_version
> +# with a static version so that this can ride the trains to esr
> +for name, branch in items_before(BRANCHES, 'gecko_version', mb_gecko_version):

are we still wanting add-on-devel builds to ride the trains with addon signing enforcement? if so and we are ready to start enforcing next cycle, we might as well s/mb_gecko_version/48/ so things happen automagically for us.

@@ +3267,5 @@
> +    for platform in ['linux64-add-on-devel', 'macosx64-add-on-devel', 'win32-add-on-devel', 'win64-add-on-devel']:
> +        if platform in branch['platforms']:
> +            del branch['platforms'][platform]
> +for name, branch in items_at_least(BRANCHES, 'gecko_version', mb_gecko_version):
> +    if name in ['mozilla-beta']:

I think we want this to be: `if name in ['mozilla-beta', 'mozilla-release', 'mozilla-esr45']:`

that way, once we are ready to ride the trains we can simply replace mb_gecko_version with say 48.

if we are not riding trains, we can simplify and combine the items_before and items_at_least loops and do something like:

```
for name, branch in items_at_least(BRANCHES, 'gecko_version', 45):
	
    if name in ['mozilla-beta']:
         continue
    for platform in ['linux64-add-on-devel', 'macosx64-add-on-devel', 'win32-add-on-devel', 'win64-add-on-devel']:
        if platform in branch['platforms']:
             del branch['platforms'][platform]
```

Comment 47

[Jordan Lund (:jlund)]

Comment on attachment 8755007 [details] [diff] [review]
bug1135781intree-2.patch wip (m-b)

Review of attachment 8755007 [details] [diff] [review]:
-----------------------------------------------------------------

I'm out of touch with the latest flag names but this seems correct to me

Comment 48

[Kim Moir [:kmoir] ET]

Attachment: bug1135781bb-configs4.patch

Comment 49

[Kim Moir [:kmoir] ET]

Attachment: bug1135781intree-3.patch

Comment 50

[Kim Moir [:kmoir] ET]

Comment on attachment 8761367 [details] [diff] [review]
bug1135781bb-configs4.patch

addon developer builds will ride trains

Comment 51

[Jordan Lund (:jlund)]

Comment on attachment 8761367 [details] [diff] [review]
bug1135781bb-configs4.patch

Review of attachment 8761367 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla/config.py
@@ +3241,5 @@
>      for platform in branch['platforms'].keys():
>          if platform not in ['linux64-debug']:
>              continue
>          del branch['platforms'][platform]
> +# mozilla-central's gecko version

s/mozilla-central/mozilla-beta/

Comment 52

[Jordan Lund (:jlund)]

Comment on attachment 8761375 [details] [diff] [review]
bug1135781intree-3.patch

Review of attachment 8761375 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/mozharness/configs/builds/releng_sub_linux_configs/64_add-on-devel.py
@@ +17,5 @@
> +    'publish_nightly_en_US_routes': False,
> +    'build_type': 'add-on-devel',
> +    'platform_supports_post_upload_to_latest': False,
> +    'enable_signing': False,
> +    'enable_talos_sendchange': False,

iiuc - we want to do unittests but not talos on linux only? I notice on the other platforms you have commented out `sendchange` action altogether.

::: testing/mozharness/mozharness/mozilla/building/buildbase.py
@@ +340,5 @@
>      # against the current platform and bits.
>      # *It will warn and fail if there is not a config for the current
>      # platform/bits
>      build_variants = {
> +        'add-on-devel': 'builds/releng_sub_%s_configs/%s_add-on-devel.py',

++

Comment 53

[Kim Moir [:kmoir] ET]

Attachment: bug1135781intree-4.patch

Comment 54

[Kim Moir [:kmoir] ET]

Attachment: bug1135781intree-5.patch

Comment 55

[Kim Moir [:kmoir] ET]

Attachment: bug1135781intree-6.patch

Comment 56

[Kim Moir [:kmoir] ET]

Current status: staging builds are failing due to bug 1244446, so I uplifted the patches in that bug to my  user repo for beta and am running additional builds.

Comment 57

[Kim Moir [:kmoir] ET]

Still same issue, will talk to someone in London re why this is happening

18:42:21     INFO -  78a6445bc1d44c20f44c073eecd1cdd4506417b5
18:42:21     INFO - old package hash: 78a6445bc1d44c20f44c073eecd1cdd4506417b5
18:42:21     INFO - Our list of packages hasn't changed; skipping re-initialization
18:42:21     INFO - Running command: ['mock_mozilla', '-r', 'mozilla-centos6-x86_64', '--install', 'autoconf213', 'python', 'mozilla-python27', 'zip', 'mozilla-python27-mercurial', 'git', 'ccache', 'perl-Test-Simple', 'perl-Config-General', 'yasm', 'wget', 'mpfr', 'xorg-x11-font*', 'imake', 'gcc45_0moz3', 'gcc454_0moz1', 'gcc472_0moz1', 'gcc473_0moz1', 'yasm', 'ccache', 'valgrind', 'dbus-x11', 'glibc-static', 'libstdc++-static', 'gtk2-devel', 'libnotify-devel', 'alsa-lib-devel', 'libcurl-devel', 'wireless-tools-devel', 'libX11-devel', 'libXt-devel', 'mesa-libGL-devel', 'gnome-vfs2-devel', 'GConf2-devel', 'gcc45_0moz3', 'gcc454_0moz1', 'gcc472_0moz1', 'gcc473_0moz1', 'yasm', 'ccache', 'pulseaudio-libs-devel', 'gstreamer-devel', 'gstreamer-plugins-base-devel', 'freetype-2.3.11-6.el6_1.8.x86_64', 'freetype-devel-2.3.11-6.el6_1.8.x86_64']
18:42:21     INFO - Copy/paste: mock_mozilla -r mozilla-centos6-x86_64 --install autoconf213 python mozilla-python27 zip mozilla-python27-mercurial git ccache perl-Test-Simple perl-Config-General yasm wget mpfr xorg-x11-font* imake gcc45_0moz3 gcc454_0moz1 gcc472_0moz1 gcc473_0moz1 yasm ccache valgrind dbus-x11 glibc-static libstdc++-static gtk2-devel libnotify-devel alsa-lib-devel libcurl-devel wireless-tools-devel libX11-devel libXt-devel mesa-libGL-devel gnome-vfs2-devel GConf2-devel gcc45_0moz3 gcc454_0moz1 gcc472_0moz1 gcc473_0moz1 yasm ccache pulseaudio-libs-devel gstreamer-devel gstreamer-plugins-base-devel freetype-2.3.11-6.el6_1.8.x86_64 freetype-devel-2.3.11-6.el6_1.8.x86_64
18:42:21     INFO -  INFO: mock_mozilla.py version 1.0.3 starting...
18:42:21     INFO -  State Changed: init plugins
18:42:21     INFO -  INFO: selinux disabled
18:42:21     INFO -  State Changed: start
18:42:21     INFO -  Mock Version: 1.0.3
18:42:21     INFO -  INFO: Mock Version: 1.0.3
18:42:21     INFO -  State Changed: lock buildroot
18:42:21     INFO -  INFO: installing package(s): autoconf213 python mozilla-python27 zip mozilla-python27-mercurial git ccache perl-Test-Simple perl-Config-General yasm wget mpfr xorg-x11-font* imake gcc45_0moz3 gcc454_0moz1 gcc472_0moz1 gcc473_0moz1 yasm ccache valgrind dbus-x11 glibc-static libstdc++-static gtk2-devel libnotify-devel alsa-lib-devel libcurl-devel wireless-tools-devel libX11-devel libXt-devel mesa-libGL-devel gnome-vfs2-devel GConf2-devel gcc45_0moz3 gcc454_0moz1 gcc472_0moz1 gcc473_0moz1 yasm ccache pulseaudio-libs-devel gstreamer-devel gstreamer-plugins-base-devel freetype-2.3.11-6.el6_1.8.x86_64 freetype-devel-2.3.11-6.el6_1.8.x86_64
18:42:21     INFO -  WARNING: Command failed. See logs for output.
18:42:21     INFO -   # umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/tmp/ccache
18:42:21     INFO -  WARNING: Command failed. See logs for output.
18:42:21     INFO -   # umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/targetdata/
18:42:21     INFO -  ERROR: Command failed. See logs for output.
18:42:21     INFO -   # mount -n --bind /builds/mock_mozilla/mozilla-centos6-x86_64/targetdata/  /builds/mock_mozilla/mozilla-centos6-x86_64/root/targetdata/
18:42:21    ERROR - Return code: 32
18:42:21    ERROR - 32 not in success codes: [0]
18:42:21  WARNING - setting return code to 3

http://dev-master2.bb.releng.use1.mozilla.com:8039/builders/Linux%20x86-64%20add-on-devel%20mozilla-beta%20build/builds/24/steps/run_script/logs/stdio

Comment 58

[Masatoshi Kimura [:emk]]

I thought the addon signing requiremnt is not enforced on ESR builds[1]. Is the plan changed?

[1] https://wiki.mozilla.org/Add-ons/Extension_Signing#Timeline "The first ESR version to include signing support will be the Firefox ESR 45 release. Signing enforcement will be on by default, and can be disabled using the xpinstall.signatures.required preference. "

Comment 59

[Kim Moir [:kmoir] ET]

So I deleted the cache here /builds/mock_mozilla/mozilla-centos6-x86_64/ on the build machine it has gotten further in that it is actually compiling the source.

Comment 60

[Kim Moir [:kmoir] ET]

Attachment: bug1135781intree-7.patch

Comment 61

[Kim Moir [:kmoir] ET]

Comment on attachment 8761367 [details] [diff] [review]
bug1135781bb-configs4.patch

I ran test builds on Linux64 and macosx64 and they seem fine.

Comment 62

[Kim Moir [:kmoir] ET]

Comment on attachment 8763046 [details] [diff] [review]
bug1135781intree-7.patch

realized I need another patch to have the correct packageFilename

Comment 63

[Jordan Lund (:jlund)]

Comment on attachment 8763046 [details] [diff] [review]
bug1135781intree-7.patch

Review of attachment 8763046 [details] [diff] [review]:
-----------------------------------------------------------------

this is the same as comment 60 yeah? just making sure as comment 62 suggests you wanted to add something new

Comment 64

[Kim Moir [:kmoir] ET]

jlund: so the problem is that the binaries are created with the same filename as the regular ff build. I want to have add-on-devel appended to the end of the filename before the suffix. I investigated this and determined that I need to change PKG_BASENAME but I can't seem to find where this needs to be changed - do you have any suggestions?  https://bugzilla.mozilla.org/show_bug.cgi?id=1135781

Comment 65

[Jordan Lund (:jlund)]

(In reply to Kim Moir [:kmoir] from comment #64)
> jlund: so the problem is that the binaries are created with the same
> filename as the regular ff build. I want to have add-on-devel appended to
> the end of the filename before the suffix. I investigated this and
> determined that I need to change PKG_BASENAME but I can't seem to find where
> this needs to be changed - do you have any suggestions? 
> https://bugzilla.mozilla.org/show_bug.cgi?id=1135781

hm, just guessing but poking some other variants that are largely based off opt builds with minor tweaks, this option seems suspiciously like something we want:

https://dxr.mozilla.org/mozilla-central/rev/51377a64158941f89ed73f388ae437cfa494c030/browser/config/mozconfigs/linux64/opt-tsan#9
https://dxr.mozilla.org/mozilla-central/rev/51377a64158941f89ed73f388ae437cfa494c030/browser/config/mozconfigs/linux64/nightly-asan#19

see here for usage: https://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/installer/package-name.mk#50

Comment 66

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2d985ed9d8dc
generate builds per checkin on beta/release/esr that allow installation of unsigned addons r=jlund

Comment 67

[Kim Moir [:kmoir] ET]

Attachment: bug1135781intree-8.patch

unbitrotten patch 

landed on inbound will ask for approval to land on beta

Comment 68

[Kim Moir [:kmoir] ET]

Attachment: bug1135781intree-package-name.patch

thanks Jordan.  I'll try that. I looked at the preference earlier but saw that it only seemed to to apply to asan and tsan and there are other artifacts with names that are non-standard yet don't specify that option.

Comment 69

[Kim Moir [:kmoir] ET]

My workaround bug 1281024 (user repo issue) was to land my reviewed patches on m-i and setup my staging env so it thinks m-i is beta in order to run tests which I'm doing now.

Comment 70

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/2d985ed9d8dc

Comment 71

[Kim Moir [:kmoir] ET]

not fixed, still more patches to land

Comment 72

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8eb61c59452b
generate builds per checkin on beta/release/esr that allow installation of unsigned addons r=me workaround for bug 1281024 DONTBUILD

Comment 73

[Kim Moir [:kmoir] ET]

Comment on attachment 8764349 [details] [diff] [review]
bug1135781intree-package-name.patch

worked in staging

Comment 74

[Kim Moir [:kmoir] ET]

Attachment: bug1135781-m-b.patch

last two patches consolidated for m-b

Comment 75

[Kim Moir [:kmoir] ET]

Comment on attachment 8764676 [details] [diff] [review]
bug1135781-m-b.patch

Approval Request Comment
[Feature/regressing bug #]: 1135781
[User impact if declined]: addon developers will not be able to test unsigned addons before release
[Describe test coverage new/current, TreeHerder]: N/A, this is already covered by existing tests
[Risks and why]: enabling addon signing by default in all automated testing is desired on beta
[String/UUID change made/needed]: none

Comment 76

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/8eb61c59452b

Comment 77

[Kim Moir [:kmoir] ET]

patches still need to land on beta

Comment 78

[Sylvestre Ledru [:Sylvestre]]

We use the status flags for the other branches. We can close it

Comment 79

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8764676 [details] [diff] [review]
bug1135781-m-b.patch

[Triage Comment]
As we landed this in nightly and you requested an uplift to beta, I guess we need that in aurora too.

Taking it 
Should be 48 beta 4

Comment 80

[Carsten Book [:Tomcat]]

(In reply to Sylvestre Ledru [:sylvestre] from comment #79)
> Comment on attachment 8764676 [details] [diff] [review]
> bug1135781-m-b.patch
> 
> [Triage Comment]
> As we landed this in nightly and you requested an uplift to beta, I guess we
> need that in aurora too.
> 
> Taking it 
> Should be 48 beta 4

landed on beta https://hg.mozilla.org/releases/mozilla-beta/rev/dfcd06e29e75

but for aurora this has conflicts (if we want it there): 
renamed 1135781 -> bug1135781-m-b.patch
(working directory not at a head)
applying bug1135781-m-b.patch
patching file testing/mozharness/mozharness/mozilla/building/buildbase.py
Hunk #1 FAILED at 335
1 out of 1 hunks FAILED -- saving rejects to file testing/mozharness/mozharness/mozilla/building/buildbase.py.rej

Comment 81

[Kim Moir [:kmoir] ET]

I'll attach an updated patch for aurora.  The patch in bug 1186522 is really needed for the add devel builds to work in beta. It doesn't matter for now since I didn't land the patch to enable the builders.

Comment 82

[Kim Moir [:kmoir] ET]

Attachment: bug1135781-m-a.patch patch for m-a

Comment 83

[Carsten Book [:Tomcat]]

thanks kim!

https://hg.mozilla.org/releases/mozilla-aurora/rev/034fb275b742

Comment 84

[Sylvestre Ledru [:Sylvestre]]

Comment on attachment 8765442 [details] [diff] [review]
bug1135781-m-a.patch patch for m-a

[Triage Comment]

We need that, taking it.

Comment 85

[Kim Moir [:kmoir] ET]

Attachment: bug1135781bb-configs5.patch unbitrotten

Comment 86

[Kim Moir [:kmoir] ET]

Attachment: bug1135781beet.patch

patch to beetmove the addondevel builds once so we can move them in release promotion

Comment 87

[Jordan Lund (:jlund)]

Comment on attachment 8765971 [details] [diff] [review]
bug1135781beet.patch

Review of attachment 8765971 [details] [diff] [review]:
-----------------------------------------------------------------

must be time to go home but I'm confused. I thought these addondevel build variants are their own job that creates their own build artifacts? IIUC, this patch suggests the addondevel binary is an artifact created from the official CI build that we promote?

clearing review for now until confusion is cleared up.

Comment 88

[Kim Moir [:kmoir] ET]

Yes, they are their own build artifact that we create during CI builds.  They are not transformed during release promotion. I thought we had to move all artifacts using beetmover (since some CI artifacts are not published to a beta).  Perhaps my assumption is incorrect. I just want these artifacts to be publicly available.

Comment 89

[Kim Moir [:kmoir] ET]

Comment on attachment 8765971 [details] [diff] [review]
bug1135781beet.patch

Talked to Jordan and it's not appropriate to use beetmover

8:17 PM <jlund> kmoir-afk: beetmover will take artifacts generated from release_platform based CI builds: linux32, linux64, win32, win64, and osx. it doesn't know about debug, asan, addondevel based builds that are outside of release_platforms
8:18 PM <kmoir-afk> jlund: so it's not the right way to get the addon-devel builds in the external s3 bucket after a beta is released?
8:19 PM <jlund> correct, it's not.
8:19 PM <kmoir-afk> okay
8:19 PM <jlund> do you want to release these? e.g. provide balrog updates? sign them? (firefox that is, not the addons themselves)?
8:20 PM <kmoir-afk> no, they are not signed, they are unbranded builds
8:20 PM <kmoir-afk> no updates
8:20 PM <kmoir-afk> I just want to have them externally available for addon developers to use for testing
8:21 PM <jlund> okay. so they don't sound like releases. is it good enough to just show developers how to find the artifacts and install them locally as they create them in automation?
8:21 PM <kmoir-afk> I will ask on the bug

Comment 90

[Kim Moir [:kmoir] ET]

Is it sufficient for addon developers to download the addon-devel builds in automation (i.e. from treeherder)?

Comment 91

[Andy McKay]

(In reply to Kim Moir [:kmoir] from comment #90)
> Is it sufficient for addon developers to download the addon-devel builds in
> automation (i.e. from treeherder)?

As long as there's a stable link we can put in the docs to point people to a folder that has them, that works for me. It doesn't have to be anything more fancy than that.

Comment 92

[Jordan Lund (:jlund)]

(In reply to Andy McKay [:andym] from comment #91)
> (In reply to Kim Moir [:kmoir] from comment #90)
> > Is it sufficient for addon developers to download the addon-devel builds in
> > automation (i.e. from treeherder)?
> 
> As long as there's a stable link we can put in the docs to point people to a
> folder that has them, that works for me. It doesn't have to be anything more
> fancy than that.

this should be easy with taskcluster index. we could point to a static -latest/ dir that always points to tip.

Comment 93

[Andy McKay]

(In reply to Jordan Lund (:jlund) from comment #92)
> this should be easy with taskcluster index. we could point to a static
> -latest/ dir that always points to tip.

Awesome, throw me a link I'll stick it in the docs.

Comment 94

[Kim Moir [:kmoir] ET]

Comment on attachment 8765607 [details] [diff] [review]
bug1135781bb-configs5.patch unbitrotten

will merge to production a bit later when I'm sure my last commit to beta is okay

Comment 95

[Kim Moir [:kmoir] ET]

Attachment: Screen Shot 2016-06-30 at 10.47.57 AM.png

merged to production this morning and triggered builds on the latest beta revision for add-on-devel builds which are running here

will verify once they are finished

Comment 96

[Kim Moir [:kmoir] ET]

So I've verified the build for macosx64.  The windows and linux builds failed because I needed some additional properties when I triggered them manually.  They are rerunning now.  As for the links to the latest artifacts that jlund mentioned above.

https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-beta.latest.firefox/gecko.v2.mozilla-beta.latest.firefox.macosx64-add-on-devel

which is available.  The links to the other builds when ready will look like this
https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-beta.latest.firefox/gecko.v2.mozilla-beta.latest.firefox.win32-add-on-devel
https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-beta.latest.firefox/gecko.v2.mozilla-beta.latest.firefox.win64-add-on-devel
https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-beta.latest.firefox/gecko.v2.mozilla-beta.latest.firefox.linux64-add-on-devel

These are the latest links to the builds that run.  If you want links that point to the build revisions that we the beta or release is associated with, it will be a different link.

Comment 97

[Kim Moir [:kmoir] ET]

The linux* builds generated from commit on m-b are failing with 

12:52:54     INFO -  python /tools/checkouts/build-tools/release/signing/signtool.py --cachedir /builds/slave/m-beta-l64-add-on-devel-000000/signing_cache -t /builds/slave/m-beta-l64-add-on-devel-000000/token -n /builds/slave/m-beta-l64-add-on-devel-000000/nonce -c /tools/checkouts/build-tools/release/signing/host.cert -H gpg:sha2signcode:osslsigncode:signcode:mar:jar:emevoucher:signing4.srv.releng.scl3.mozilla.com:9110 -H gpg:sha2signcode:osslsigncode:signcode:mar:jar:emevoucher:signing5.srv.releng.scl3.mozilla.com:9110 -H gpg:sha2signcode:osslsigncode:signcode:mar:jar:emevoucher:signing6.srv.releng.scl3.mozilla.com:9110 -H dmgv2:mac-v2-signing1.srv.releng.scl3.mozilla.com:9110 -H dmgv2:mac-v2-signing2.srv.releng.scl3.mozilla.com:9110 -H dmgv2:mac-v2-signing3.srv.releng.scl3.mozilla.com:9110 -H dmgv2:mac-v2-signing4.srv.releng.scl3.mozilla.com:9110 -H dmgv2:mac-v2-signing6.srv.releng.scl3.mozilla.com:9110 -H dmgv2:mac-v2-signing7.srv.releng.scl3.mozilla.com:9110 -f gpg '../../dist//firefox-48.0.en-US.linux-x86_64-add-on-devel.checksums'
12:52:54     INFO -  python: can't open file '/tools/checkouts/build-tools/release/signing/signtool.py': [Errno 2] No such file or directory

http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-linux64-add-on-devel/1467313167/mozilla-beta-linux64-add-on-devel-bm72-build1-build0.txt.gz

but if I look at the instance where it ran, the file is there.  Not sure why this is failing. Or why signing is enabled in the first place.  talking to jlund in irc re this.  

The windows builds are failing but actually producing artifacts
http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-win32-add-on-devel/1467313167/
sh

and 
http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-win64-add-on-devel/1467313167/

the error message here is 
can't set the following properties: buildid, sourcestamp, appVersion, and appName. Required paths missing. Verify both c:\builds\moz2_slave\m-beta-w64-add-on-devel-000000\build\src\config\printconfigsetting.py and c:\builds\moz2_slave\m-beta-w64-add-on-devel-000000\build\src\obj-firefox/dist/bin/application.ini exist. These paths require the 'build' action to be run prior to this
Running post_fatal callback...

so the artifacts aren't uploaded to taskcluster.  Investigating this as well.

Comment 98

[Kim Moir [:kmoir] ET]

Attachment: bug1135781signing.patch

jordan suggested the issues wrt to the linux64 builds might be related to signing related variables in the configs so he suggested removing them. I deployed the linux64 change and will rerun a job to ensure it works.

Comment 99

[Kim Moir [:kmoir] ET]

Attachment: bug1135781maconly.patch

patch to disable linux and windows builds temporarily if the previous patch doesn't resolve the issue.

Comment 100

[Kim Moir [:kmoir] ET]

Comment on attachment 8766943 [details] [diff] [review]
bug1135781signing.patch

The resolved the issue with  python: can't open file '/tools/checkouts/build-tools

now we are onto another issue

I already landed the linux64 bit to test with the latest ci build

Comment 101

[Kim Moir [:kmoir] ET]

The linux and windows builds are failing with this issue

    INFO - Return code: 0
16:01:39     INFO - setting properties set by mach build. Looking in path: /builds/slave/m-beta-l64-add-on-devel-000000/build/src/obj-firefox/dist/mach_build_properties.json
16:01:39     INFO - No mach_build_properties.json found - not importing properties.
16:01:39    FATAL - Can't set the following properties: buildid, sourcestamp, appVersion, and appName. Required paths missing. Verify both /builds/slave/m-beta-l64-add-on-devel-000000/build/src/config/printconfigsetting.py and /builds/slave/m-beta-l64-add-on-devel-000000/build/src/obj-firefox/dist/bin/application.ini exist. These paths require the 'build' action to be run prior to this
16:01:39    FATAL - Running post_fatal callback...

http://buildbot-master94.bb.releng.use1.mozilla.com:8001/builders/Linux%20x86-64%20add-on-devel%20mozilla-beta%20build/builds/0/steps/run_script/logs/stdio

the actual artifacts are being built

Comment 102

[Jordan Lund (:jlund)]

(In reply to Kim Moir [:kmoir] away until July 11 from comment #101)
> The linux and windows builds are failing with this issue
> 
>     INFO - Return code: 0
> 16:01:39     INFO - setting properties set by mach build. Looking in path:
> /builds/slave/m-beta-l64-add-on-devel-000000/build/src/obj-firefox/dist/
> mach_build_properties.json
> 16:01:39     INFO - No mach_build_properties.json found - not importing
> properties.
> 16:01:39    FATAL - Can't set the following properties: buildid,
> sourcestamp, appVersion, and appName. Required paths missing. Verify both
> /builds/slave/m-beta-l64-add-on-devel-000000/build/src/config/
> printconfigsetting.py and
> /builds/slave/m-beta-l64-add-on-devel-000000/build/src/obj-firefox/dist/bin/
> application.ini exist. These paths require the 'build' action to be run
> prior to this
> 16:01:39    FATAL - Running post_fatal callback...
> 
> http://buildbot-master94.bb.releng.use1.mozilla.com:8001/builders/
> Linux%20x86-64%20add-on-devel%20mozilla-beta%20build/builds/0/steps/
> run_script/logs/stdio
> 
> the actual artifacts are being built

I think that's because mozharness is looking in the wrong path for the objdir:

it is trying:
/builds/slave/m-beta-l64-add-on-devel-000000/build/src/obj-firefox/

but your objdir is:
/builds/slave/m-beta-l64-add-on-devel-000000/build/src/obj-add-on-devel/

looking into this, it seems like the confusion lies in the build sys reading env['MOZ_OBJDIR'] but mozharness trusts self.config['objdir'] [1]

I think the best fix would be to refactor query_objdir in buildbase.py to read env['MOZ_OBJDIR'] and fallback on self.config['objdir']. But for your scope specifically, you will either want to change env['MOZ_OBJDIR'] to the default: 'obj-firefox' or, if you want a custom named objdir, do what horizon/graphene do by defining self.config['objdir'] in your sub config file[2]

[1] https://dxr.mozilla.org/mozilla-central/rev/b69a5bbb5e40bd426e35222baa600b481e50d265/testing/mozharness/mozharness/mozilla/building/buildbase.py#810

[2] https://dxr.mozilla.org/mozilla-central/search?q=objdir+path%3Amozharness+path%3Agraphene&redirect=true

Comment 103

[Kim Moir [:kmoir] ET]

Attachment: bug1135781objdir.patch

resetting objdir to default as Jordan suggested

Comment 104

[Kim Moir [:kmoir] ET]

Attachment: bug1135781objdir-2.patch

Comment 105

[Kim Moir [:kmoir] ET]

Comment on attachment 8767537 [details] [diff] [review]
bug1135781objdir-2.patch

I noticed with this patch that the taskcluster index lists add-on-devel builds under 

https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-beta.latest.firefox/gecko.v2.mozilla-beta.latest.firefox.win64-opt

for win32 too

which is strange

I would have thought it would create a new category like it did for macosx and linux64 addon-devel builds.  Investigating.

Comment 106

[Kim Moir [:kmoir] ET]

Addon devel builds (unbranded and with signed addons preference not enfored) for Linux64, win32, win64 and macosx are green now on beta

https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta

I will enable them on release/esr on the next uplift as appropriate

Comment 107

[Hector Zhao [:hectorz]]

(In reply to Kim Moir [:kmoir] away until July 11 from comment #96)
> ...  As for the links to the latest artifacts
> that jlund mentioned above.
> 
> https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-beta.latest.
> firefox/gecko.v2.mozilla-beta.latest.firefox.macosx64-add-on-devel
> 
> which is available.  The links to the other builds when ready will look like
> this
> https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-beta.latest.
> firefox/gecko.v2.mozilla-beta.latest.firefox.win32-add-on-devel
> https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-beta.latest.
> firefox/gecko.v2.mozilla-beta.latest.firefox.win64-add-on-devel
> https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-beta.latest.
> firefox/gecko.v2.mozilla-beta.latest.firefox.linux64-add-on-devel
> 
> These are the latest links to the builds that run.

Although I can find the win32/win64 builds at https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&filter-platform=addon

The above mentioned taskcluster artifacts viewer urls for win32/win64 will show this error message:
"Task not found!  No task is indexed under gecko.v2.mozilla-beta.latest.firefox.win{32,64}-add-on-devel"

> If you want links that
> point to the build revisions that we the beta or release is associated with,
> it will be a different link.

Will the builds corresponding to release/beta builds be available under some other "latest" route, or we'll have to figure out e.g. 48.0b6 build 1 is built from d142c49033c015f67272562b37dbe2912cfc7f14 and visit https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-beta.revision.d142c49033c015f67272562b37dbe2912cfc7f14.firefox/gecko.v2.mozilla-beta.revision.d142c49033c015f67272562b37dbe2912cfc7f14.firefox.*-add-on-devel directly?

Thanks.

Comment 108

[Kim Moir [:kmoir] ET]

hectorz:

For the specific builds associated with a beta revision, there currently isn't a way to list all the artifacts without knowing the revision.  Regarding the missing windows builds, in the "latest" links, I've opened bug 1286021 with the tc team.  In the interim, the artifacts that were created with 48.0b6 are available here 

https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&filter-platform=addon&revision=d142c49033c015f67272562b37dbe2912cfc7f14

Comment 109

[Hector Zhao [:hectorz]]

(In reply to Kim Moir [:kmoir] from comment #108)

Thanks for the clarification

Comment 110

[Kim Moir [:kmoir] ET]

Going to close this bug, talked to Kev and Shell in a meeting today and they are happy with the state of builds that are being created on commit.  I'll follow up with the issues re taskcluster index but the builds are available via treeherder.

There should be jobs created on release with the next uplift, I'll verify this.

Comment 111

[Kim Moir [:kmoir] ET]

Verified that addon devel builds are running on m-r on commit after m-b -> to m-r uplift

https://tools.taskcluster.net/index/artifacts/#gecko.v2.mozilla-release.latest.firefox/