1313787 - Ensure packaged debug windows artifact builds are usable

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Chris Manchester (limited bugmail, email directly)]

There were found in bug 1305534 to fail in tests jobs. From testing them locally, I think we're missing some dlls from the package.

Comment 1

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Also certutil.exe? https://treeherder.mozilla.org/#/jobs?repo=try&revision=805d3f2a4af1&selectedJob=30325129

Comment 2

[James Graham [:jgraham]]

certutil.exe at least appears in the common.tests.zip file, so I think I was probably wrong about it being missing but it does seem to be erroring on use, so something isn't quite right.

Comment 3

[Intermittent Failures Robot]

12 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* try: 12

Platform breakdown:
* windows8-64: 4
* windows7-32-vm: 4
* windows10-64: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1313787&startday=2016-10-31&endday=2016-11-06&tree=all

Comment 4

[Chris Manchester (limited bugmail, email directly)]

Attachment: Bug 1313787 - Find relevant dlls on Windows when packaging an artifact build.

Review commit: https://reviewboard.mozilla.org/r/91488/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/91488/

Comment 5

[(not currently active) Ted Mielczarek]

Comment on attachment 8808762 [details]
Bug 1313787 - Find relevant dlls on Windows when packaging an artifact build.

https://reviewboard.mozilla.org/r/91488/#review91350

::: old-configure.in:183
(Diff revision 1)
>  dnl ========================================================
>  dnl Checks for compilers.
>  dnl ========================================================
>  
> +MSVC_C_RUNTIME_DLL_NAME=vcruntime140.dll
> +MSVC_CXX_RUNTIME_DLL_NAME=msvcp140.dll

This is going to break in the future if we update to a newer MSVC. It would be nice if we could figure out a better strategy for artifact builds longer-term without forcing them to run all of configure.

Comment 6

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8808762 [details]
Bug 1313787 - Find relevant dlls on Windows when packaging an artifact build.

https://reviewboard.mozilla.org/r/91488/#review91350

> This is going to break in the future if we update to a newer MSVC. It would be nice if we could figure out a better strategy for artifact builds longer-term without forcing them to run all of configure.

How so? We'll have to update these names as soon as we update, this patch makes these variables the only place they get set in the main old-configure.

Comment 7

[(not currently active) Ted Mielczarek]

Because there's always a period where we support >1 version of MSVC, which is why they were in the case block. Maybe we could just move that stuff into moz.configure and find a better way to use it from artifact builds? (In a separate bug, you don't have to fix that now.)

Comment 8

[Mike Hommey [:glandium]]

Comment on attachment 8808762 [details]
Bug 1313787 - Find relevant dlls on Windows when packaging an artifact build.

https://reviewboard.mozilla.org/r/91488/#review91438

I think this needs to be dealt with in some other way, because:
- there are other files involved that are not in the list you have here (specifically all the ucrt files, other d3d compilers (iirc we ship several))
- On top of ted's comment, in a close future where the next version of MSVC is released, the version used on automation will not necessarily match the one used locally.

I think a better approach would be to make use of what the artifacts build actually contains, not a hardcoded list of what we expect it contains.

Comment 9

[(not currently active) Ted Mielczarek]

It seems like the core problem here is that with a lot of things with artifact builds we're guessing or hardcoding a bunch of values because we aren't running configure, and we don't have the configure results from the actual build we're using available. What if we uploaded config.status or something like that, and used it to fill in the gaps of the configure tests we didn't run?

Comment 10

[Mike Hommey [:glandium]]

(In reply to Ted Mielczarek [:ted.mielczarek] from comment #9)
> It seems like the core problem here is that with a lot of things with
> artifact builds we're guessing or hardcoding a bunch of values because we
> aren't running configure

we are

> and we don't have the configure results from the
> actual build we're using available.

that, however is true.

> What if we uploaded config.status or
> something like that, and used it to fill in the gaps of the configure tests
> we didn't run?

we already have most if not all the relevant information in the build itself, in the form of the list of files it contains, as well as, for some specific things, buildconfig.html, platform.ini, etc.
config.status contains way too many things that are irrelevant.

Comment 11

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8808762 [details]
Bug 1313787 - Find relevant dlls on Windows when packaging an artifact build.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/91488/diff/1-2/

Comment 12

[Mike Hommey [:glandium]]

Comment on attachment 8808762 [details]
Bug 1313787 - Find relevant dlls on Windows when packaging an artifact build.

https://reviewboard.mozilla.org/r/91488/#review91834

::: browser/installer/package-manifest.in:117
(Diff revision 2)
> +; We don't have a complete view of which dlls to expect when doing an artifact
> +; build because we haven't run all of configure, so we trust what's in
> +; dist/bin, because everything there was extracted from our original build's
> +; package.
> +#ifdef MOZ_ARTIFACT_BUILDS
> +@BINPATH@/*.dll

Mmmm if this doesn't fail on non-Windows builds, there's a bug to file to make it so.

Comment 13

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8808762 [details]
Bug 1313787 - Find relevant dlls on Windows when packaging an artifact build.

https://reviewboard.mozilla.org/r/91488/#review91834

> Mmmm if this doesn't fail on non-Windows builds, there's a bug to file to make it so.

On a linux build we get:

Warning: /home/worker/workspace/build/src/browser/installer/package-manifest.in:117: Missing file(s): bin/*.dll

That seems about right, because we're unsetting `MOZ_PKG_FATAL_WARNINGS` (this was part of bug 1240134, which means we're no longer generating .xpt files during an artifact builds).

Comment 14

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8808762 [details]
Bug 1313787 - Find relevant dlls on Windows when packaging an artifact build.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/91488/diff/2-3/

Comment 15

[Mike Hommey [:glandium]]

Comment on attachment 8808762 [details]
Bug 1313787 - Find relevant dlls on Windows when packaging an artifact build.

https://reviewboard.mozilla.org/r/91488/#review92974

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7d6d40b8751d
Find relevant dlls on Windows when packaging an artifact build. r=glandium

Comment 17

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/7d6d40b8751d