If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Build Thunderbird in parallel on new systems

RESOLVED FIXED

Status

Release Engineering
General Automation
P1
normal
RESOLVED FIXED
6 years ago
4 years ago

People

(Reporter: jhopkins, Assigned: jhopkins)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(3 attachments, 2 obsolete attachments)

(Assignee)

Description

6 years ago
We want Thunderbird builds building in parallel on Firefox infrastructure.

Builds will go to the http://stage.mozilla.org/pub/mozilla.org/thunderbird-test/ area while legacy infrastructure will continue to deliver builds to http://stage.mozilla.org/pub/mozilla.org/thunderbird/
(Assignee)

Comment 1

6 years ago
Created attachment 616678 [details] [diff] [review]
patch to append build and symbol upload locations and channel names with "-test"

Patch is based on this repo: https://hg.mozilla.org/users/john.hopkins_mozillamessaging.com/buildbot-configs-stage-approved/
Attachment #616678 - Flags: review?(mbanner)
Attachment #616678 - Flags: review?(bhearsum)
Comment on attachment 616678 [details] [diff] [review]
patch to append build and symbol upload locations and channel names with "-test"

Looks reasonable to me, though I've only taken a quick glance.

Something to note is that we also need to update the mozconfigs for nightly so that --enable-update-channel is $(MOZ_UPDATE_CHANNEL} rather than "nightly". If you want to do that the same as the other files rs=me for comm-central (please land with a DONTBUILD).
Attachment #616678 - Flags: review?(mbanner) → review+
(Assignee)

Comment 3

6 years ago
FYI, I'm working through a problem with missing comm-* builders on staging, so I will need to do a follow-up patch to address that.

standard8: I'll try and get a patch created today with MOZ_UPDATE_CHANNEL.  Aiming to get this deployed tomorrow morning Eastern time.
Comment on attachment 616678 [details] [diff] [review]
patch to append build and symbol upload locations and channel names with "-test"

Review of attachment 616678 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/thunderbird_production_config.py
@@ +27,5 @@
>      'comm-aurora': {
>          'tinderbox_tree': 'Thunderbird-Aurora',
>      },
>      'try-comm-central': {
>          'tinderbox_tree': 'ThunderbirdTry',

Do you really want to send results to the same trees as the current production machines? That could be very confusing for developers.

::: mozilla/release-thunderbird-comm-beta.py
@@ +100,5 @@
>      'macosx64': 'mail/config/mozconfigs/macosx-universal/release',
>      'win32': 'mail/config/mozconfigs/win32/release',
>  }
>  releaseConfig['xulrunner_mozconfigs']          = {}
> +releaseConfig['releaseChannel']      = 'beta-test'

This is confusing, as it conflicts with "betatest"...but I don't think we'll be running releases in parallel, so let's just not change the release configs?

::: mozilla/thunderbird_config.py
@@ -33,1 @@
>      # It's a little unfortunate to have both of these but some things (HgPoller)

You'll need to change product_name, too, because we use it for some URL building in factory.py, and also as an argument to post_upload.py for repacks. I did a cursory glance and don't _think_ it will break anything.

@@ +802,5 @@
>  # If True, a complete update snippet for this branch will be generated and
>  # uploaded to. Any platforms with 'debug' in them will not have snippets
>  # generated.
>  BRANCHES['comm-central']['create_snippet'] = True
> +BRANCHES['comm-central']['update_channel'] = 'nightly-test'

I don't think we should adjust the channels, it'll make it much harder to compare test vs. real. Instead, you can adjust aus2_base_upload_dir and the l10n version to use Thunderbird-test. We can create those directories on AUS if we need to.

@@ +808,5 @@
>  BRANCHES['comm-central']['create_partial_l10n'] = True
>  BRANCHES['comm-central']['aus2_user'] = 'tbirdbld'
>  BRANCHES['comm-central']['aus2_ssh_key'] = 'tbirdbld_dsa'
>  BRANCHES['comm-central']['aus2_base_upload_dir'] = '/opt/aus2/build/0/Thunderbird/comm-central'
>  BRANCHES['comm-central']['aus2_base_upload_dir_l10n'] = '/opt/aus2/build/0/Thunderbird/comm-central'

While not directly related to this patch, I don't think these are the right upload paths for snippets on aus3-staging. We upload directly to /opt/aus2/incoming/2 for Firefox, for example.
Attachment #616678 - Flags: review?(bhearsum) → review-
(In reply to John Hopkins (:jhopkins) from comment #3)
> standard8: I'll try and get a patch created today with MOZ_UPDATE_CHANNEL. 
> Aiming to get this deployed tomorrow morning Eastern time.

I decided over on bug 745536 that this wasn't necessary. Testers can just set app.update.url.override to hit aus3.m.o and use nightly test.

(In reply to Ben Hearsum [:bhearsum] from comment #4)
I agree with a lot of what Ben has said here, with a few tweaks. How about I upload the aus changes I already wrote, and you do the upload one separately ?
Created attachment 616778 [details] [diff] [review]
Update changes

This sets the keys up how I'd like to have them (not reusing tbirdbld_dsa, I've revoked access to aus3-staging for that), and is similar to what I did in bug 741648 when we moved FF to aus3-staging. We'll upload the test snippets to a comm-central-test dir, which matches the aus config I wrote on bug 745536.

It looks like thunderbird_production_config.py handles GLOBVAL_VARS differently to the matching staging config, so my changes are really just reminders for when the staging style gets merged over to production.
Attachment #616778 - Flags: review?(bhearsum)

Updated

6 years ago
Attachment #616778 - Flags: review?(jhopkins)
(Assignee)

Updated

6 years ago
Attachment #616778 - Flags: review?(jhopkins) → review+
(Assignee)

Comment 7

6 years ago
Created attachment 616855 [details] [diff] [review]
update logic to accomodate product 'thunderbird-test'
Attachment #616855 - Flags: review?(nrthomas)

Updated

6 years ago
Attachment #616855 - Flags: review?(nrthomas) → review+
(Assignee)

Comment 8

6 years ago
Created attachment 616857 [details] [diff] [review]
buildbot-configs patch

Updated patch which includes nthomas' patch and incorporates bhearsum's feedback on my last patch.
Attachment #616678 - Attachment is obsolete: true
Attachment #616778 - Attachment is obsolete: true
Attachment #616778 - Flags: review?(bhearsum)
Attachment #616857 - Flags: review?(nrthomas)
Attachment #616857 - Flags: review?(bhearsum)
(Assignee)

Comment 9

6 years ago
Comment on attachment 616855 [details] [diff] [review]
update logic to accomodate product 'thunderbird-test'

Review of attachment 616855 [details] [diff] [review]:
-----------------------------------------------------------------

Landed in http://hg.mozilla.org/build/buildbotcustom/rev/9f0168ec2c1f
Attachment #616855 - Flags: checked-in+
Comment on attachment 616857 [details] [diff] [review]
buildbot-configs patch

Review of attachment 616857 [details] [diff] [review]:
-----------------------------------------------------------------

r+ on the AUS parts of this.
Attachment #616857 - Flags: review?(nrthomas) → review+
(In reply to Ben Hearsum [:bhearsum] from comment #4)
> Comment on attachment 616678 [details] [diff] [review]
> patch to append build and symbol upload locations and channel names with
> "-test"
> 
> Review of attachment 616678 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: mozilla-tests/thunderbird_production_config.py
> @@ +27,5 @@
> >      'comm-aurora': {
> >          'tinderbox_tree': 'Thunderbird-Aurora',
> >      },
> >      'try-comm-central': {
> >          'tinderbox_tree': 'ThunderbirdTry',
> 
> Do you really want to send results to the same trees as the current
> production machines? That could be very confusing for developers.

I was assuming these results would be going to tbpl.mozilla.org not tinderbox.mozilla.org. Assuming that is the case, then the results are going to a different location, so it doesn't matter if the tree names are the same.

Btw, please make the try tree Thunderbird-Try (include the dash) as that's what I've just landed to tbpl.mozilla.org (not published yet).
Depends on: 747328
(In reply to Mark Banner (:standard8) from comment #11)
> (In reply to Ben Hearsum [:bhearsum] from comment #4)
> > Comment on attachment 616678 [details] [diff] [review]
> > patch to append build and symbol upload locations and channel names with
> > "-test"
> > 
> > Review of attachment 616678 [details] [diff] [review]:
> > -----------------------------------------------------------------
> > 
> > ::: mozilla-tests/thunderbird_production_config.py
> > @@ +27,5 @@
> > >      'comm-aurora': {
> > >          'tinderbox_tree': 'Thunderbird-Aurora',
> > >      },
> > >      'try-comm-central': {
> > >          'tinderbox_tree': 'ThunderbirdTry',
> > 
> > Do you really want to send results to the same trees as the current
> > production machines? That could be very confusing for developers.
> 
> I was assuming these results would be going to tbpl.mozilla.org not
> tinderbox.mozilla.org. Assuming that is the case, then the results are going
> to a different location, so it doesn't matter if the tree names are the same.
> 
> Btw, please make the try tree Thunderbird-Try (include the dash) as that's
> what I've just landed to tbpl.mozilla.org (not published yet).

The "tinderbox_tree" variable in buildbot-configs controls which tree on tinderbox.mozilla.org that builds report to. TBPL maintains its own list of trees and maps them to Tinderbox trees for the purpose of Tree Status. It uses the branch name in the buildapi JSON files (http://builddata.pub.build.mozilla.org/buildjson/) to map figure out which builds should be on each tree.

Setting these variables to be the same as the existing production machines *will* cause additional builds to show up on the tinderbox.mozilla.org trees of the same names. I don't know what, if anything, that will break but it seems like we should try to avoid it. Setting it to MozillaTest won't affect how TBPL sees it.

On a side note, we probably need to get a bug on file to get a bunch of new trees added to TBPL.
Attachment #616857 - Flags: review?(bhearsum) → review+
(In reply to Ben Hearsum [:bhearsum] from comment #12)
> Setting these variables to be the same as the existing production machines
> *will* cause additional builds to show up on the tinderbox.mozilla.org trees
> of the same names. I don't know what, if anything, that will break but it
> seems like we should try to avoid it. Setting it to MozillaTest won't affect
> how TBPL sees it.

Ah right, that's good.

> On a side note, we probably need to get a bug on file to get a bunch of new
> trees added to TBPL.

Already done, but it hasn't been deployed yet. It is possible to hook up a local copy to see those trees though.
(Assignee)

Comment 14

6 years ago
Created attachment 616942 [details] [diff] [review]
set mozilla-tests trees to MozillaTest

follow-up to buildbot-configs patch
Attachment #616942 - Flags: review?(bhearsum)
Attachment #616942 - Flags: review?(bhearsum) → review+
(Assignee)

Comment 15

6 years ago
Comment on attachment 616942 [details] [diff] [review]
set mozilla-tests trees to MozillaTest

Review of attachment 616942 [details] [diff] [review]:
-----------------------------------------------------------------

Landed in https://hg.mozilla.org/users/john.hopkins_mozillamessaging.com/buildbot-configs-stage-approved/rev/a3e29c10faa7
Attachment #616942 - Flags: checked-in+
(Assignee)

Comment 16

6 years ago
Comment on attachment 616857 [details] [diff] [review]
buildbot-configs patch

Review of attachment 616857 [details] [diff] [review]:
-----------------------------------------------------------------

Landed in https://hg.mozilla.org/users/john.hopkins_mozillamessaging.com/buildbot-configs-stage-approved/rev/5b6e4de2006c
Attachment #616857 - Flags: checked-in+
(Assignee)

Comment 17

6 years ago
https://hg.mozilla.org/users/john.hopkins_mozillamessaging.com/buildbot-configs-stage-approved/ has been 'hg pull'ed into buildbot-configs.
Depends on: 747856
Depends on: 747862
(Assignee)

Updated

6 years ago
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.