83589 - Doing a commercial build clobbers DefinesOptions.h, forcing an entire rebuild

Status: RESOLVED FIXED

(SeaMonkey :: Build Config, defect, P3)

Description

[Simon Fraser [no longer active]]

When you do a commercial build (whose scripts contain a different set of build 
options), we recreate DefinesOptions.h, which dirties the entire Mozilla tree.

This is very evil. We need to somehow ensure that the commercial scripts don't 
clobber the mozilla build options.

Comment 1

[Peter Van der Beken [:peterv]]

Huaarh. Let's fight the evil.

Comment 2

[J.J. Enser]

This seems quite recent to me. I remember seeing depend cycles being much shorter 
not that long ago. What happened and how hard is it to fix this ?

Comment 3

[Simon Fraser [no longer active]]

The simple fix is to just copy the build options from MozillaBuildFlags.txt into 
the commercial equivalent. The slightly more involved fix is to have the scripts 
grab MozillaBuildFlags.txt from the mozilla tree, and overlay any commercial-
specific options. The sripts should probably flag discrepancies between the 
options settings in a user's mozilla and commercial prefs.

Comment 4

[Simon Fraser [no longer active]]

Thinking some more, the real bug is this:
The commercial scripts (living in ns/build/mac) are regenerating the file 
mozilla/config/mac/DefinesOptions.h. This is wrong. They do this because 
BuildCore.pm's ConfigureBuildSystem() subroutine has:

    UpdateConfigHeader(":mozilla:config:mac:DefinesOptions.h");

We're gonna have to feed in that path from the top-level script, probably as a 
member of %inputfiles.

Comment 5

[Peter Van der Beken [:peterv]]

Attachment: Proposed fix

Comment 6

[Peter Van der Beken [:peterv]]

Haven't tried it but it should work. Anyone want to try it or review it?

Comment 7

[Simon Fraser [no longer active]]

Looks good. The only comment I have is that the BuildCore.pm module still 
contains a mozilla-relative path: ":mozilla:config:mac:" which, ideally, it 
should not. Maybe we should send in the entire path via 
filepaths{"optionsdefines"} ?

The patch also doesn't address the issue of sharing options between the netscape 
and mozilla builds, though I'm not sure what the correct approach is here.

Comment 8

[Peter Van der Beken [:peterv]]

Hmmm, ignore that previous patch, it doesn't work at all. I'd say I must have
smoking something but I don't smoke :/. Attaching a patch that seems to work.

Comment 9

[Peter Van der Beken [:peterv]]

Attachment: Non-smoking patch

Comment 10

[Simon Fraser [no longer active]]

sr=sfraser if you rename "$definesoption_file_name" to "$config_header_file_name" 
or something.

Comment 11

[Peter Van der Beken [:peterv]]

Just saw the commercial dep tinderbox finish in 40 minutes, down from about 2 hours.

Comment 12

[Jon Granrose]

cool.  Setting QA to jj for verification.

Comment 13

[J.J. Enser]

Yep, confirming that cycle time dropped: both NS depend tinderbox and morning 
depend verification build gained about 1 hour. Excellent job!

btw, I've been timing all the systems before the major hardware shuffle I'm about 
to operate!
NS depend trunk cycle = +/- 50mn (using a G4 450)
NS BRANCH depend cycle = +/- 1h50mn (G4 533)...

Which naturally leads me to the next question: Any chance this can land on the 
0.9.2 branch as well?

Comment 14

[Heikki Toivonen (remove -bugzilla when emailing directly)]

Build system change, been tested on trunk, looks good. nsBranch+ by me.

Comment 15

[Peter Van der Beken [:peterv]]

Checked in on the branch, commercial mac branch tinderbox now cycles happily in
50 mins, down from 2 hours. [PDT+] by Chris Hoffmann in mail.

Comment 16

[Simon Fraser [no longer active]]

Reopening. This fix prevents me from doing an optimized build over an existing 
debug build, without clobbering all the built library aliases.

Comment 17

[Simon Fraser [no longer active]]

Attachment: Dual build fix for mozilla/

Comment 18

[Simon Fraser [no longer active]]

Attachment: Dual build fix for ns/

Comment 19

[Simon Fraser [no longer active]]

Those two patches should, hopefully, get us back into a state where we can do opt 
and debug builds in the same tree. The changes I made are:

* Removing some old decrepit include files (MacConfig.h, ComponentConfig.h,
  MANIFEST_config)
* Adding

+#ifdef DEBUG
+#include "DefinesOptionsDebug.h"
+#else
+#include "DefinesOptions.h"
+#endif

  into NGLayoutConfigInclude.h (mozilla) and MacConfigInclude.h (ns)

* Standardize the auto-gen header names (DefinesOptions{Debug}.h)

* Adding a 'config' build step.

* Adding PrepareBuild() to the mozilla & ns build modules

* Moved UpdateConfigHeader() from BuildCore.pm into BuildUtils.pm

* Fixing UpdateConfigHeader() to be more flexible about matching, and to put
  a comment in the generated file.

Things to fix in future:
Clean up the mozilla and netscape include files; they are a mess. 
NGLayoutConfigInclude.h gets aliased into the tree with a different name, for no 
good reason.

Comment 20

[Peter Van der Beken [:peterv]]

Very nice! Thanks. r=peterv.

Comment 21

[Scott Collins]

sr=scc on the latest two patches above (i.e., 41864 and 41865)

Comment 22

[Simon Fraser [no longer active]]

Mine

Comment 23

[Simon Fraser [no longer active]]

Close to being in, but out of time.

Comment 24

[Simon Fraser [no longer active]]

Boing.

Comment 25

[Simon Fraser [no longer active]]

Changes checked in.