Drop support for universal Mac builds in Thunderbird 53

RESOLVED FIXED in Thunderbird 54.0

Status

Thunderbird
Build Config
RESOLVED FIXED
5 months ago
2 months ago

People

(Reporter: aleth, Assigned: aleth)

Tracking

unspecified
Thunderbird 54.0
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [Thunderbird-testfailure:B Mac opt, see comment #7])

Attachments

(3 attachments, 4 obsolete attachments)

(Assignee)

Description

5 months ago
+++ This bug was initially created as a clone of Bug #1295375 +++

Per bug 1183613 comment 16, we can drop universal mac builds in Firefox 53.

mozilla-central becomes Firefox 53 on November 7.
Created attachment 8821745 [details] [diff] [review]
dropUniversal.patch

Could this be the way to go? Or is more needed?
Attachment #8821745 - Flags: review?(aleth)
Created attachment 8821751 [details] [diff] [review]
buildbot change

I think this buildbot change is needed. But where is aurora, release etc?
Attachment #8821751 - Flags: feedback?(aleth)
(Assignee)

Comment 3

4 months ago
(In reply to Richard Marti (:Paenglab) from comment #1)
> Could this be the way to go? Or is more needed?

Unfortunately much more will be needed, with lots of moving parts also in the build system.
(Assignee)

Updated

4 months ago
Flags: needinfo?(aleth)

Comment 4

4 months ago
Comment on attachment 8821751 [details] [diff] [review]
buildbot change

:Paenglab, this is the same patch as the mozconfig one.
Created attachment 8825655 [details] [diff] [review]
buildbot change

Correct patch
Attachment #8821751 - Attachment is obsolete: true
Attachment #8821751 - Flags: feedback?(aleth)

Comment 6

4 months ago
Comment on attachment 8825655 [details] [diff] [review]
buildbot change

Review of attachment 8825655 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla/thunderbird_config.py
@@ +285,5 @@
>              'build_space': 12,
>              'upload_symbols': True,
>              'download_symbols': True,
>              'slaves': SLAVES['macosx64-lion'],
>              'platform_objdir': "%s/i386" % OBJDIR,

I think the "%s/i386" part should be changed to just OBJDIR.

Updated

3 months ago
Depends on: 1331297
(Assignee)

Comment 7

3 months ago
Bug 1331297 has now busted OS X Opt Dailies, as was always going to happen sooner or later. Note we still have full test coverage on OS X debug, and local OS X Opt builds are of course not affected as they are not universal in the first place.
Flags: needinfo?(aleth)
Comment hidden (mozreview-request)

Comment 9

3 months ago
Aleth, I also pushed this to try, https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=34ab1220161931cc7cdf379c05b39bd8beeed3b1

Changes compared to Richard's patch is that I removed the lightning universal stuff, in all the configs (including the l10n one)

Updated

3 months ago
Blocks: 1337350

Comment 10

3 months ago
build broke, https://treeherder.mozilla.org/#/jobs?repo=try-comm-central&revision=34ab1220161931cc7cdf379c05b39bd8beeed3b1&selectedJob=75134191, as it's using the old universal mozconfig. I wonder if we can do harm by landing this and the buildbotcustom changes, and then iterate if needed?

Or maybe at least the buildbotcustom one, so that we can push to try to test?
(Assignee)

Comment 11

3 months ago
(In reply to Axel Hecht from comment #10)
> build broke,
> https://treeherder.mozilla.org/#/jobs?repo=try-comm-
> central&revision=34ab1220161931cc7cdf379c05b39bd8beeed3b1&selectedJob=7513419
> 1, as it's using the old universal mozconfig. I wonder if we can do harm by
> landing this and the buildbotcustom changes, and then iterate if needed?

I don't think Firefox is using buildbot any more on dailies, so it can't get more broken ;)

Having said that, iirc there's a lot of os x build special casing in the buildbot code. I don't think the buildbot patch is correct as is, but if you want to r+ it and do it in stages, that's fine.

Updated

3 months ago
No longer blocks: 1337350
Duplicate of this bug: 1337350

Updated

3 months ago
Whiteboard: [Thunderbird-testfailure][see comment #7]

Updated

3 months ago
Whiteboard: [Thunderbird-testfailure][see comment #7] → [Thunderbird-testfailure:B Mac opt, see comment #7]
All the talk about updates and stuff in bug 1295375 concern me. Plus, I never understood buildbot-configs, so having to touch that would also be quite a bit out of my comfort zone.
(Assignee)

Updated

3 months ago
Assignee: nobody → aleth

Comment 14

2 months ago
I'm a little confused here. SM have landed:
https://hg.mozilla.org/comm-central/rev/becf847f864ca2765095ea5f22a4cd6f99d35729
which is not a big patch.

There are a couple of patches attached here and then Aleth took the bug. So what's going on? We've been without Mac coverage for a while now.
Comment on attachment 8825655 [details] [diff] [review]
buildbot change

Rail, could this work, when we have created the files in mail/config/mozconfigs/macosx64?
Attachment #8825655 - Flags: review?(rail)

Comment 16

2 months ago
Created attachment 8838590 [details] [diff] [review]
1322402-mac-build.patch (landed comment #17)

In bug 1339182 [Remove OSX universal support in the build system]
https://hg.mozilla.org/mozilla-central/rev/d88a5673b1f7b5e3c8c37383e43c5c1b599ed89c
M-C removed build/macosx/universal

So I'm getting build oranges since these files are still in C-C. This patch removes those directory, and also includes the combined patches of Richard and Pike to fix files in mail/config/mozconfigs/macosx-universal, in fact I remove the whole thing since there was only one file left after moving release and nightly.

With this patch I get a green debug build, the opt build still doesn't work due to a build bot problem:
cp: mail/config/mozconfigs/macosx-universal/nightly: No such file or directory
Attachment #8821745 - Attachment is obsolete: true
Attachment #8834390 - Attachment is obsolete: true
Attachment #8821745 - Flags: review?(aleth)
Attachment #8834390 - Flags: review?(aleth)

Comment 17

2 months ago
https://hg.mozilla.org/comm-central/rev/6ee27a7af6cd3e7bd25f392048fd40160910275a

Due to lack of movement here and the current build oranges, I had to land this as a bustage fix. However, reading the comments above, was the agreed process anyway.

Let's now move over to the build bot changes.
Comment on attachment 8825655 [details] [diff] [review]
buildbot change

Review of attachment 8825655 [details] [diff] [review]:
-----------------------------------------------------------------

I think :ewong is correct, let's drop i386 from platform_objdir
Attachment #8825655 - Flags: review?(rail) → review-
Created attachment 8838599 [details] [diff] [review]
macosx64-config.patch

Removed the i386 from platform_objdir.

Rail, if this patch should get r+, please can you land it then? Thank you.
Attachment #8825655 - Attachment is obsolete: true
Attachment #8838599 - Flags: review?(rail)
Comment on attachment 8838599 [details] [diff] [review]
macosx64-config.patch

Thank you!
Attachment #8838599 - Flags: review?(rail) → review+

Comment 21

2 months ago
Rail, could you please get this into production. Neither Richard nor myself are build specialists. The action here was triggered by the landing of bug 1339182 (see comment #16). So we hope we made the necessary changes in C-C and with some luck we'll get a build.
Flags: needinfo?(rail)
Will do in a bit.
Flags: needinfo?(rail)
Comment on attachment 8838599 [details] [diff] [review]
macosx64-config.patch

https://hg.mozilla.org/build/buildbot-configs/rev/a7218494e505ba92d75209c7aa226b3b80fe509e

I'll merge it to production in a bit
In production: https://hg.mozilla.org/build/buildbot-configs/rev/a7218494e505

Comment 25

2 months ago
https://hg.mozilla.org/comm-central/rev/6ee27a7af6cd3e7bd25f392048fd40160910275a (comment #17)
https://hg.mozilla.org/build/buildbot-configs/rev/a7218494e505 (comment #24)

Successful build:
https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=6ee27a7af6cd3e7bd25f392048fd40160910275a&selectedJob=78392016

Thanks to all involved. If you find more problems, reopen this bug or file another one.
Status: NEW → RESOLVED
Last Resolved: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 54.0
FWIW, I just triggered a new nightly from that revision, let's see what happens.
Updated to new Daily and it is working. :-)
When TB 54 will enter beta and later ESR59 the buildbot-config files need to be adapted.

DXR search where macosx-universal is referenced: https://dxr.mozilla.org/build-central/search?q=mail%2Fconfig%2Fmozconfigs%2Fmacosx-universal&redirect=false

Comment 29

2 months ago
There don't seem to be l10n repacks, I don't find any public traces of errors, sadly.

https://ftp.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-17-11-53-27-comm-central/ has a mac build, but https://ftp.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-17-11-53-27-comm-central-l10n/ doesn't

Comment 30

2 months ago
(In reply to Richard Marti (:Paenglab) from comment #28)
> When TB 54 will enter beta and later ESR59 the buildbot-config files need to
> be adapted.
Does that mean that we need to uplift this to Aurora since otherwise the changed build bot file will not allow an Aurora build?

(In reply to Axel Hecht from comment #29)
> There don't seem to be l10n repacks, I don't find any public traces of
> errors, sadly.
As I said, I'm not the author here. I merely fixed some orange builds and while I was there landed other people's patches which made it (semi-)work. Without those changes the files in mail/config/mozconfigs/macosx64/ would have referenced files I had to remove in build/macosx/universal/mozconfig (since FF removed them and we must keep the builds in sync).
(In reply to Jorg K (GMT+1) from comment #30)
> (In reply to Richard Marti (:Paenglab) from comment #28)
> > When TB 54 will enter beta and later ESR59 the buildbot-config files need to
> > be adapted.
> Does that mean that we need to uplift this to Aurora since otherwise the
> changed build bot file will not allow an Aurora build?

I see no aurora file. So I guess aurora uses the nightly config too. Only beta and esr have their own files.

Comment 32

2 months ago
So using the same build bot file, Aurora will be busted tomorrow unless this gets uplifted and then breaks l10n on Aurora.
When my assumption is correct, yes,

Comment 34

2 months ago
(In reply to Jorg K (GMT+1) from comment #32)
> So using the same build bot file, Aurora will be busted tomorrow unless this
> gets uplifted and then breaks l10n on Aurora.

wrt l10n..  is it working now?

Comment 35

2 months ago
(In reply to Edmund Wong (:ewong) from comment #34)
> wrt l10n..  is it working now?
Whether l11n is working on Aurora? According to the announcement
https://mail.mozilla.org/pipermail/tb-planning/2016-October/004954.html
and the evidence:
http://ftp.mozilla.org/pub/thunderbird/nightly/latest-comm-aurora-l10n/
Yes!

I'm still trying to see what Aleth did in October 2016 to get them working.

Th only thing I noticed that mail/config/mozconfigs/macosx-universal/l10n-mozconfig which I removed
https://hg.mozilla.org/comm-central/rev/6ee27a7af6cd3e7bd25f392048fd40160910275a#l5.2
didn't have |. $topsrcdir/build/mozconfig.common| but its replacement does:
https://dxr.mozilla.org/comm-central/source/mail/config/mozconfigs/macosx64/l10n-mozconfig

Would removing that line make a difference (for the better)?
Flags: needinfo?(ewong)

Comment 36

2 months ago
Could someone please compare
deleted file mode 100644
--- a/mail/config/mozconfigs/macosx-universal/l10n-mozconfig
with
https://dxr.mozilla.org/comm-central/source/mail/config/mozconfigs/macosx64/l10n-mozconfig

and tell me which of the marked differences
. $topsrcdir/build/mozconfig.common (**)
ac_add_options --with-l10n-base=../../l10n (**)

are likely to make it work again. Also note that for Windows, |disable-compile-environment| was removed here:
https://hg.mozilla.org/comm-central/rev/09ece7c086487cd738adb91669a43ad106a11443

For Mac it got added here:
https://hg.mozilla.org/comm-central/rev/6bbdff770fd6
(In reply to Jorg K (GMT+1) from comment #36)
> Could someone please compare
> deleted file mode 100644
> --- a/mail/config/mozconfigs/macosx-universal/l10n-mozconfig
> with
> https://dxr.mozilla.org/comm-central/source/mail/config/mozconfigs/macosx64/
> l10n-mozconfig
> 
> and tell me which of the marked differences
> . $topsrcdir/build/mozconfig.common (**)
> ac_add_options --with-l10n-base=../../l10n (**)

I would say first try to change 
ac_add_options --with-l10n-base=../../l10n
to ac_add_options --with-l10n-base=../../../l10n

Because this looks like a copy of FX and on C-C we have mozilla a directory deeper.

Comment 38

2 months ago
OK, Aurora just got busted, so let's uplift this and add Richards suggestion ;-)

Aurora (TB 53):
https://hg.mozilla.org/releases/comm-aurora/rev/6c3bbdce6b7b2a45f4afa44d3db3165c4cc239fb
https://hg.mozilla.org/releases/comm-aurora/rev/b2892e9b6a35fa8801218c511e3ead629508afa9

Note that in the file 'nightly' I added |ac_add_options --enable-profiling| to make it match the C-C file. No idea why that wasn't there in the first place.

Let's trigger a build on Aurora now.
Hmm, removing the build/macosx/universal/ files on aurora will likely give check-sync-dir oranges because they are only removed in M-C.

Comment 40

2 months ago
Richard, can you please try to trigger a N on Mac opt only on C-A. When I click "Add new jobs" I get "Error fetching runnable jobs".

Comment 41

2 months ago
I filed bug 1340787 for the problem in comment #40. Who should I CC this to?
I get the same error.

Comment 43

2 months ago
I tried to trigger a new nightly via https://secure.pub.build.mozilla.org/buildapi/self-serve/comm-aurora.
It said OK, and that works. I'll cancel the all but Mac.

Comment 44

2 months ago
Oops, grammar derailed: I'll cancel all but Mac.
BTW, did the same thing you did, that's why there are two now. Cancel one of the macs, too?

Comment 46

2 months ago
I cancelled the second lot completely.

Comment 47

2 months ago
(In reply to Richard Marti (:Paenglab) from comment #39)
> Hmm, removing the build/macosx/universal/ files on aurora will likely give
> check-sync-dir oranges because they are only removed in M-C.
The N is green. Mac binaries here:
https://archive.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-18-02-40-05-comm-aurora/
but not here:
https://archive.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-18-02-40-05-comm-aurora-l10n/
So that tweak
https://hg.mozilla.org/releases/comm-aurora/rev/b2892e9b6a35fa8801218c511e3ead629508afa9
didn't work.
Then | . $topsrcdir/build/mozconfig.common | probably needs to be removed too as the error says: "mozbuild.configure.options.InvalidOptionError: RUSTC is not available in this configuration" and mozconfig.common relates to | . "$topsrcdir/build/mozconfig.rust" |. And it seems it fails before it wants to go to l10n and we don't know which is the correct path. But I would leave the longer path in it at first because the old l10n file had it too.

Comment 49

2 months ago
OK, last try, Aleth will hate me for this trial-and-error approach:
https://hg.mozilla.org/releases/comm-aurora/rev/616d41945914fffcdcd57e87f5bd303544ade7e4

I'll trigger a build to see what that does.

Comment 50

2 months ago
https://archive.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-18-07-32-20-comm-aurora/
https://archive.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-18-07-32-20-comm-aurora-l10n/ didn't work.
ERROR: Invalid value --with-l10n-base, ../../../l10n doesn't exist

So I'll do one last one with |ac_add_options --with-l10n-base=../../l10n|

Comment 51

2 months ago
https://hg.mozilla.org/releases/comm-aurora/rev/19a4185737b9d245440592183c7217f8ee3304df
backout of
https://hg.mozilla.org/releases/comm-aurora/rev/b2892e9b6a35fa8801218c511e3ead629508afa9

Still active:
https://hg.mozilla.org/releases/comm-aurora/rev/616d41945914fffcdcd57e87f5bd303544ade7e4

Comment 52

2 months ago
https://archive.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-18-09-30-48-comm-aurora/
No l10n directory created. Time to stop, right?

Updated

2 months ago
Blocks: 1340829

Comment 53

2 months ago
https://hg.mozilla.org/releases/comm-aurora/rev/7216ccd49fb7d8f2d09d3e8f13d54f9a4f5b5063
backout of
https://hg.mozilla.org/releases/comm-aurora/rev/616d41945914fffcdcd57e87f5bd303544ade7e4

Summarising the landings here:
C-C: https://hg.mozilla.org/comm-central/rev/6ee27a7af6cd3e7bd25f392048fd40160910275a
C-A: https://hg.mozilla.org/releases/comm-aurora/rev/6c3bbdce6b7b2a45f4afa44d3db3165c4cc239fb

To be continued in bug 1340829.

Comment 54

2 months ago
Damn, I was too impatient here:
https://archive.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-18-09-30-48-comm-aurora/
https://archive.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-18-09-30-48-comm-aurora-l10n/
did get created. So summarising:
|ac_add_options --with-l10n-base=../../l10n| is correct and 
|. $topsrcdir/build/mozconfig.common| needs to go.

I'll do this quickly in bug 1340829.
Flags: needinfo?(ewong)

Comment 55

2 months ago
(In reply to Jorg K (GMT+1) from comment #54)
> Damn, I was too impatient here:
> https://archive.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-18-09-30-
> 48-comm-aurora/
> https://archive.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-18-09-30-
> 48-comm-aurora-l10n/
> did get created. So summarising:
> |ac_add_options --with-l10n-base=../../l10n| is correct and 
> |. $topsrcdir/build/mozconfig.common| needs to go.
> 
> I'll do this quickly in bug 1340829.

Yeah. sorry for the delay in response.

../../l10n is correct because the objdir no longer includes the
platform (otherwise, it'd require ../../../l10n).  

Also, why does |. $topsrcdir/build/mozconfig.common| need to go?

Comment 56

2 months ago
(In reply to Edmund Wong (:ewong) from comment #55)
> Also, why does |. $topsrcdir/build/mozconfig.common| need to go?
I really don't know. I tried with and without and without worked, see bug 1340829. Also comment #48 here.
The file I removed didn't have it:
https://hg.mozilla.org/comm-central/rev/6ee27a7af6cd3e7bd25f392048fd40160910275a#l5.2
Sorry, they let the apprentice (me) do the job and due to complete ignorance he could only do a trial-and-error job, which succeeded on the 4th try!

Comment 57

2 months ago
Beta (TB 52):
https://hg.mozilla.org/releases/comm-beta/rev/8da0792b305012c730d5c8e0dda8712338c4f5b3

Comment 58

2 months ago
ESR 45:
https://hg.mozilla.org/releases/comm-esr45/rev/f8478f0eaea4de365bf126c4805c3b31f001e8d0
Fingers crossed. Since the offending line |. $topsrcdir/build/mozconfig.common| wasn't in
mail/config/mozconfigs/macosx64/l10n-mozconfig to start with, the uplift from bug 1340829 wasn't necessary.
Why are you pushing this to TB 52 and 45? FX has it only in 53.

Comment 60

2 months ago
Because builds are failing there due to our build bot changes:
cp: mail/config/mozconfigs/macosx64/nightly: No such file or directory

Check for example

ESR 45
https://archive.mozilla.org/pub/thunderbird/nightly/2017/02/2017-02-18-03-02-11-comm-esr45/comm-esr45-macosx64-nightly-bm82-build1-build5.txt.gz

Beta TB 52:
https://archive.mozilla.org/pub/thunderbird/tinderbox-builds/comm-beta-macosx64/1487706688/comm-beta-macosx64-bm86-build1-build0.txt.gz

With the patch uplifted, it builds again. As I said, they called the apprentice to do the job :-(

Comment 61

2 months ago
Mac OS X Snow Leopard (10.6) and later will only run on Intel Macs and doesn't need universal/fat binaries according to https://en.wikipedia.org/wiki/Universal_binary.

Since according to https://www.mozilla.org/en-US/thunderbird/45.7.1/system-requirements/ TB 45.x only runs on 10.6 and later, it makes no sense to ship universal binaries for any of the versions we support.

Note that TB 52 (https://www.mozilla.org/en-US/thunderbird/52.0beta/system-requirements/) only supports Mac OS X 10.9 and later.

Anyway, I'm prepared to be wrong and leave it to the build peer to figure it out.
(In reply to Jorg K (GMT+1) from comment #61)
> Mac OS X Snow Leopard (10.6) and later will only run on Intel Macs and
> doesn't need universal/fat binaries according to
> https://en.wikipedia.org/wiki/Universal_binary.
> 
> Since according to
> https://www.mozilla.org/en-US/thunderbird/45.7.1/system-requirements/ TB
> 45.x only runs on 10.6 and later, it makes no sense to ship universal
> binaries for any of the versions we support.

But such changes shouldn't be done in a point release, especially on a stable ESR release.

What if you use the macosx-universal configs in the macos64 directories? then they call still the universal config ($topsrcdir/build/macosx/universal/mozconfig). maybe also the removed files in build/macosx/universal/ are needed again.

Comment 63

2 months ago
I won't touch it any more. The build failed anyway at the packaging stage:
https://archive.mozilla.org/pub/thunderbird/tinderbox-builds/comm-esr45-macosx64/1487873962/comm-esr45-macosx64-bm84-build1-build1.txt.gz
with the weirdest of errors.

Re. your suggestion: The build bot error is:
cp: mail/config/mozconfigs/macosx64/nightly: No such file or directory

So you're suggesting to copy the "universal" nightly, release and l10n files from macosx-universal to macosx64? Nice hack. We can back out what I landed and see whether an official build can still be done.

Comment 64

2 months ago
(In reply to Jorg K (GMT+1) from comment #61)
> Since according to
> https://www.mozilla.org/en-US/thunderbird/45.7.1/system-requirements/ TB
> 45.x only runs on 10.6 and later, it makes no sense to ship universal
> binaries for any of the versions we support.
>
The universal binaries currently built are not intel/ppc but 32-bit/64-bit binaries. IIRC the 32-bit ones are needed in order to run some of the supported plugins.

Comment 65

2 months ago
OK, the problem came from the build bot change in comment #24. Nick (:nthomas) has kindly offered to fix the mess we made.

I will then backout the beta (TB 52) and ESR 45 changesets. We can still build TB 52 beta 4 without those changes.
Created attachment 8840637 [details] [diff] [review]
[buildbotcustom] Swaps current beta, ESR45, and ESR52 back to universal

https://hg.mozilla.org/build/buildbot-configs/rev/a7218494e505ba92d75209c7aa226b3b80fe509e made a global change to all buildbot builds, and this brings it back to c-c and c-a.
Attachment #8840637 - Flags: review?(jlund)

Comment 67

2 months ago
Backout:

Beta (TB 52):
https://hg.mozilla.org/releases/comm-beta/rev/6afc02c2de5faec1edaf317b5c1d8a64c2fa6f8c

ESR 45:
https://hg.mozilla.org/releases/comm-esr45/rev/437b04a2d8c81cff35eab5e94771d7222501224a

Comment 68

2 months ago
Comment on attachment 8840637 [details] [diff] [review]
[buildbotcustom] Swaps current beta, ESR45, and ESR52 back to universal

should do the trick
Attachment #8840637 - Flags: review?(jlund) → review+
Comment on attachment 8840637 [details] [diff] [review]
[buildbotcustom] Swaps current beta, ESR45, and ESR52 back to universal

https://hg.mozilla.org/build/buildbot-configs/rev/f558856a142ed61c305388f16f1fbce99eae6b11
https://hg.mozilla.org/build/buildbot-configs/rev/83720de173380d5f54c6453a6791bdc8746f6969

It'll be live in about 40 minutes.

Comment 70

2 months ago
Pardon my ignorance..  if the macosx64's src_mozconfig is set to the nightly 
mozconfig,  where in the infra, during the release, would the src_mozconfig be
set to the release one?  (I do apologize if it's apparent.  I'm like a bit
overwhelmed with a lot of different fires, of which this mac-universal is 
one of 'em.)

Comment 71

2 months ago
(In reply to Edmund Wong (:ewong) from comment #70)
> Pardon my ignorance..  if the macosx64's src_mozconfig is set to the nightly 
> mozconfig,  where in the infra, during the release, would the src_mozconfig
> be
> set to the release one?  (I do apologize if it's apparent.  I'm like a bit
> overwhelmed with a lot of different fires, of which this mac-universal is 
> one of 'em.)

ah nevermind..  I found it.  it's in the release-thunderbird-*.py
You need to log in before you can comment on or make changes to this bug.