1393817 - Enabling MOZ_AUTOMATION_L10N_CHECK breaks MinGW Build

Status: RESOLVED FIXED

(Core :: Internationalization: Localization, defect)

Description

[Tom Ritter [:tjr]]

> [task 2017-08-25T14:52:55.412154Z] 14:52:55     INFO -  gmake[7]: *** No rule to make target '/home/worker/workspace/build/src/obj-firefox/dist/install/sea/target.installer.exe', needed by 'repackage-win32-installer'.  Stop.
> [task 2017-08-25T14:52:55.412320Z] 14:52:55     INFO -  Makefile:161: recipe for target 'repackage-win32-installer-x-test' failed
> [task 2017-08-25T14:52:55.412482Z] 14:52:55     INFO -  gmake[6]: *** [repackage-win32-installer-x-test] Error 2
> [task 2017-08-25T14:52:55.412637Z] 14:52:55     INFO -  Makefile:215: recipe for target 'l10n-check' failed
> [task 2017-08-25T14:52:55.412796Z] 14:52:55     INFO -  gmake[5]: *** [l10n-check] Error 2
> [task 2017-08-25T14:52:55.412992Z] 14:52:55     INFO -  /home/worker/workspace/build/src/browser/build.mk:39: recipe for target 'l10n-check' failed

Comment 1

[Tom Ritter [:tjr]]

Try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7dc0c5a76bbc75cc720c13188d31a1f24eeb24e5

Comment 2

[Tom Ritter [:tjr]]

Attachment: Bug 1393817 Fix L10N check for MinGW build

With that fixed, Re-enable MOZ_AUTOMATION_L10N_CHECK for MinGW

Review commit: https://reviewboard.mozilla.org/r/194266/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/194266/

Comment 3

[(not currently active) Ted Mielczarek]

Comment on attachment 8923048 [details]
Bug 1393817 Fix L10N check for MinGW build

https://reviewboard.mozilla.org/r/194266/#review202558

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/25f21f22a34b
Use buildconfig to get the path for 7z rather than hardcoding it in exe_7z_archive.py r=ted

Comment 5

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/25f21f22a34b

Comment 6

[Dorel Luca [:dluca]]

Backed out 1 changesets (bug 1393817) for failing to repackage the nightly build on Windows a=backout

https://hg.mozilla.org/mozilla-central/rev/2203d23429f61f879753234b275ff141c50c11f0

https://treeherder.mozilla.org/logviewer.html#?job_id=143307017&repo=mozilla-central&lineNumber=450

Comment 7

[Justin Wood (:Callek)]

:tjr, to test this on try you can do `./mach try fuzzy --full` and select for example `repackage-win32-nightly`

Comment 8

[Tom Ritter [:tjr]]

Try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=89d9400ffdac179c1dc48278da62e4b6c0eba0e0

Comment 9

[Tom Ritter [:tjr]]

Comment on attachment 8923048 [details]
Bug 1393817 Fix L10N check for MinGW build

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/194266/diff/1-2/

Comment 10

[Tom Ritter [:tjr]]

Okay completely new version for review.

Comment 11

[(not currently active) Ted Mielczarek]

Related: bug 1415965.

Comment 12

[(not currently active) Ted Mielczarek]

Comment on attachment 8923048 [details]
Bug 1393817 Fix L10N check for MinGW build

https://reviewboard.mozilla.org/r/194266/#review210742

::: python/mozbuild/mozbuild/action/exe_7z_archive.py:24
(Diff revision 2)
>      try:
>          if pkg_dir:
>              shutil.move(pkg_dir, 'core')
>          subprocess.check_call(['upx', '--best', '-o', mozpath.join(tmpdir, '7zSD.sfx'), sfx_package])
>  
> -        subprocess.check_call(['7z', 'a', '-r', '-t7z', mozpath.join(tmpdir, 'app.7z'), '-mx', '-m0=BCJ2', '-m1=LZMA:d25', '-m2=LZMA:d19', '-m3=LZMA:d19', '-mb0:1', '-mb0s1:2', '-mb0s2:3'])
> +        subprocess.check_call([sevenz_command, 'a', '-r', '-t7z', mozpath.join(tmpdir, 'app.7z'), '-mx', '-m0=BCJ2', '-m1=LZMA:d25', '-m2=LZMA:d19', '-m3=LZMA:d19', '-mb0:1', '-mb0s1:2', '-mb0s2:3'])

If we're detecting 7zip in configure then you should be able to `import buildconfig` and just use `buildconfig.substs['7Z']` to get the binary name instead of passing it down from make.

Comment 13

[Tom Ritter [:tjr]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #12)
> Comment on attachment 8923048 [details]
> Bug 1393817 Fix L10N check for MinGW build
> 
> https://reviewboard.mozilla.org/r/194266/#review210742
> 
> ::: python/mozbuild/mozbuild/action/exe_7z_archive.py:24
> (Diff revision 2)
> >      try:
> >          if pkg_dir:
> >              shutil.move(pkg_dir, 'core')
> >          subprocess.check_call(['upx', '--best', '-o', mozpath.join(tmpdir, '7zSD.sfx'), sfx_package])
> >  
> > -        subprocess.check_call(['7z', 'a', '-r', '-t7z', mozpath.join(tmpdir, 'app.7z'), '-mx', '-m0=BCJ2', '-m1=LZMA:d25', '-m2=LZMA:d19', '-m3=LZMA:d19', '-mb0:1', '-mb0s1:2', '-mb0s2:3'])
> > +        subprocess.check_call([sevenz_command, 'a', '-r', '-t7z', mozpath.join(tmpdir, 'app.7z'), '-mx', '-m0=BCJ2', '-m1=LZMA:d25', '-m2=LZMA:d19', '-m3=LZMA:d19', '-mb0:1', '-mb0s1:2', '-mb0s2:3'])
> 
> If we're detecting 7zip in configure then you should be able to `import
> buildconfig` and just use `buildconfig.substs['7Z']` to get the binary name
> instead of passing it down from make.

That was the first version :)  But it broke the repackage-win32-nightly build because configure isn't run and buildconfig.substs isn't populated. After talking to you about it, I attempted to get configure to run, but it didn't work. (I forget the exact error, as it was a few weeks ago, but I might be able to dig it up.)

Comment 14

[Tom Ritter [:tjr]]

Forgot ni

Comment 15

[(not currently active) Ted Mielczarek]

OK, that sucks, but I think we can still do better by using that value when it's available, at least, and falling back to '7z' like the current code does when it's not. Something like:

```
import buildconfig
from mozbuild.base import BuildEnvironmentNotFoundException

try:
  sevenz = buildconfig.config.substs['7Z']
except BuildEnvironmentNotFoundException:
  # configure hasn't been run, just use the default
  sevenz = '7z'
```

You have to use `buildconfig.config.substs` instead of just `buildconfig.substs` for weird reasons related to recent changes to how things in the build system work, but that should reliably give you the value of `7Z` from configure when configure has been run.

You can test this by running `mach python` without a mozconfig available, or with a MOZ_OBJDIR set to some other place so that you don't have a configured objdir. That should be pretty close to the environment those repackage builds that don't run configure are using.

Comment 16

[Tom Ritter [:tjr]]

Comment on attachment 8923048 [details]
Bug 1393817 Fix L10N check for MinGW build

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/194266/diff/2-3/

Comment 17

[Tom Ritter [:tjr]]

Try runs: 
https://treeherder.mozilla.org/#/jobs?repo=try&revision=109b3acb7dfeeb9e5888e2f74c087139f3bf6db0
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4cf23d93ff3b1a2da7bd3a8287300c2d806b7e9e

Comment 18

[(not currently active) Ted Mielczarek]

Comment on attachment 8923048 [details]
Bug 1393817 Fix L10N check for MinGW build

https://reviewboard.mozilla.org/r/194266/#review212026

Thanks, that feels much better!

Comment 19

[Tom Ritter [:tjr]]

Thank you!

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/988a2837b8dc
Fix L10N check for MinGW build r=ted

Comment 21

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/988a2837b8dc