1297373 - Don't regenerate buildid for every build on developer machines

Status: RESOLVED DUPLICATE of bug 1298328

(Firefox Build System :: General, defect)

Description

[(not currently active) Ted Mielczarek]

Currently if you do a top-level build, even if it's a no-op, we generate a new build ID and then wind up re-linking the firefox binary because of that. This is important for automation (so that the generated binaries have a correct build ID), but not really for local builds. The FORCE rules are here:
https://dxr.mozilla.org/mozilla-central/rev/24763f58772d45279a935790f732d80851924b46/Makefile.in#34

When discussing this on IRC, glandium pointed out that we depend on this behavior to invalidate some caches (I have forgotten exactly what), but perhaps that's no longer true or if so we can find a smarter way to do this that doesn't involve doing this work for a no-op build.

Comment 1

[(not currently active) Ted Mielczarek]

Oh! We do already have a different workaround for this:
https://dxr.mozilla.org/mozilla-central/rev/24763f58772d45279a935790f732d80851924b46/config/rules.mk#1523

We touch dist/bin/.purgecaches, which is intended to have the same effect.

Comment 2

[Mike Hommey [:glandium]]

I'm not sure .purgecaches does all the necessary work.

Comment 3

[Mike Hommey [:glandium]]

Also, relinking the firefox binary isn't really /that/ time consuming. The other bugs blocking bug 1297386 take a lot more time.

Comment 4

[Gregory Szorc [:gps]]

I agree other bugs are more important from an impact perspective. But the principle of the matter is that if nothing changes the build should be a no-op. In fact, I believe Tup will reach this conclusion on its own and skip linking if none of the input files change. So we'll need to figure out cache purging eventually. Might as well do it in this bug.

Comment 5

[Mike Hommey [:glandium]]

I'll repeat what I've said repeatedly: no-ops are uninteresting. Nobody does a no-op. There is always something changing. The problem is it is very hard to know what change needs a buildid update. So we update it all the time. Ensuring Firefox doesn't require a buildid bump besides having the number updated is not a build system problem. It's a platform + Firefox problem.

Comment 6

[Gregory Szorc [:gps]]

If a Firefox developer changes a .jsm linked by a jar.mn, why do we need to relink firefox.exe? I contend we should never have to relink firefox.exe in that scenario.

Comment 7

[Mike Hommey [:glandium]]

We shouldn't, but if we don't, we can't guarantee it won't be broken. Again, providing that guarantee is a platform + Firefox problem, not a build system problem.

Comment 8

[(not currently active) Ted Mielczarek]

If .purgecaches doesn't work reliably we should fix it so it does.

Comment 9

[Mike Hommey [:glandium]]

Another option is to disable MOZ_APP_STATIC_INI on local builds.

Comment 10

[Michael Shal [:mshal]]

Tup itself doesn't track the clock as an input, so last time I built Firefox with it (~2013?), I generated the buildid / application.ini.h in mach before running tup. So as far as tup was concerned, some files had always changed even on a "no-op" build, and so it always ran the commands that depended on them. The nsBrowserApp.cpp / firefox chain takes less than a second, so I agree its fairly low on the priority list.

The main problem I ran into is that tup automatically picks up the dependency from buildid.h to libxul, which we currently lie about in the Makefile: https://dxr.mozilla.org/mozilla-central/rev/bd7645928990649c84609d3f531e803c2d41f269/toolkit/xre/Makefile.in#19

If you fix that dependency, you can see that libxul.so does indeed change on every build since the buildid used by nsAppRunner changes. Would it affect anything if we changed nsAppRunner's buildid for local builds to be something static? Should we grab someone knowledgeable from platform to see what other options we have?

Comment 11

[(not currently active) Ted Mielczarek]

This came up in discussion again today in re: tup. I posit that this *must* work OK, because artifact builds work, and those do not rebuild the firefox binary.

Comment 12

[Mike Hommey [:glandium]]

The point is that any binary change is accompanied with a buildid change.

Comment 13

[Chris Manchester (limited bugmail, email directly)]

This happened in bug 1298328