1211957 - mach build faster doesn't update resource:///modules/MigrationUtils.jsm

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[:Gijs (he/him)]

STR:

1. ./mach build
2. ./mach build faster
3. edit browser/components/migration/MigrationUtils.jsm
4. ./mach build faster
5. open resource:///modules/MigrationUtils.jsm in a tab

ER:
see updated version

AR:
see old version

Mike, any idea what's going on here?

Comment 1

[:Gijs (he/him)]

(in case it's relevant, this is on OS X 10.10, with hg 3.5.1)

Comment 2

[Nick Alexander :nalexander [he/him]]

Gijs: did you restart the browser?  I wonder if you're seeing caching.  When I was mucking with the Fennec bootstrapper, I had to basically disable the browser cache.

Comment 3

[:Gijs (he/him)]

(In reply to Nick Alexander :nalexander from comment #2)
> Gijs: did you restart the browser?  I wonder if you're seeing caching.  When
> I was mucking with the Fennec bootstrapper, I had to basically disable the
> browser cache.

Yes, the browser was not running when I ran ./mach build faster.

It works if I run:

./mach build browser/modules browser/components/migration


maybe a jar.mn/manifest thing is being missed as part of the jar.mn/moz.build/makefile DAG linearization?

Comment 4

[:Felipe Gomes (needinfo for replies!)]

I've verified that ./mach build faster does not create the .purgecaches file, which e.g. ./mach build browser/modules does.

Comment 5

[:Felipe Gomes (needinfo for replies!)]

Although touching the file manually didn't fix this problem.

I'm on OSX and did:
mach build faster
touch ../ff-opt/dist/Nightly.app/Contents/Resources/browser/.purgecaches
mach run

on launch, that file gets deleted as expected, but MigrationUtils was still not updated

Comment 6

[Mike Hommey [:glandium]]

If you apply the following patch, what does `mach build -X faster/install-dist/bin/browser` output?

diff --git a/python/mozbuild/mozpack/files.py b/python/mozbuild/mozpack/files.py
index 7203918..ddb0742 100644
--- a/python/mozbuild/mozpack/files.py
+++ b/python/mozbuild/mozpack/files.py
@@ -457,24 +457,26 @@ class PreprocessedFile(BaseFile):
             # assume that the preprocessor needs to be rerun. That will
             # regenerate the dependency file.
             if self.depfile and not os.path.exists(self.depfile):
                 skip = False
             else:
                 skip = not BaseFile.any_newer(dest.path, pp_deps)
 
         if skip:
+            print('skipping %s' % dest.path)
             return False
 
         deps_out = None
         if self.depfile:
             deps_out = FileAvoidWrite(self.depfile)
         pp = Preprocessor(defines=self.defines, marker=self.marker)
         pp.setSilenceDirectiveWarnings(self.silence_missing_directive_warnings)
 
+        print('preprocessing %s' % dest.path)
         with open(self.path, 'rU') as input:
             pp.processFile(input=input, output=dest, depfile=deps_out)
 
         dest.close()
         if self.depfile:
             deps_out.close()
 
         return True

Comment 7

[:Felipe Gomes (needinfo for replies!)]

Updated the tree and got:

> $ mach build -X faster/install-dist/bin/browser
>  0:00.19 /usr/bin/make -C /Users/felipe/moz/mozilla-central/ff-opt -j8 -s backend.RecursiveMakeBackend
>  0:00.44 /usr/bin/make -C faster -j8 -s install-dist/bin/browser
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/components/BrowserComponents.> manifest
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/components/BrowserProfileMigrators.> manifest
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/components/ChromeProfileMigrator.js
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/components/FeedWriter.js
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/components/SafariProfileMigrator.js
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/components/browsermodules.manifest
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/components/fuelApplication.js
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/components/nsBrowserGlue.js
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/defaults/preferences/firefox-> branding.js
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/defaults/preferences/firefox.js
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/defaults/preferences/webide-prefs.> js
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/modules/CustomizableUI.jsm
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/modules/CustomizableWidgets.jsm
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/modules/CustomizeMode.jsm
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/modules/MigrationUtils.jsm
>  0:00.84 preprocessing /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser/modules/PanelWideWidgetTracker.jsm
>  0:00.84 From /Users/felipe/moz/mozilla-central/ff-opt/dist/bin/browser: Kept 321 existing; Added/updated 16; > Removed 0 files and 0 directories.
>  0:00.90 Your build was successful!

So MigrationUtils is there in the preprocessing, but it doesn't get picked up

Comment 8

[:Felipe Gomes (needinfo for replies!)]

Attachment: zip of obj-dir/faster

Comment 9

[:Felipe Gomes (needinfo for replies!)]

I did a `mach build faster` and then a `mach build browser/app` and the changes get picked up. So is it just a matter of faster not rebuilding the .app package on OSX?

Comment 10

[Mike Hommey [:glandium]]

Ah yes, that would be it.

Comment 11

[Mike Hommey [:glandium]]

Attachment: Add the repackage step at the end of a faster build on Mac

Comment 12

[:Felipe Gomes (needinfo for replies!)]

Works for me with this patch

Comment 13

[Gregory Szorc [:gps]]

Comment on attachment 8670561 [details] [diff] [review]
Add the repackage step at the end of a faster build on Mac

Review of attachment 8670561 [details] [diff] [review]:
-----------------------------------------------------------------

I question the necessity of this build step now that we have install manifests. But that's for another bug.

Comment 14

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/993c6f004f39

Comment 15

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/993c6f004f39