1506016 - pdbs for utility programs in Windows ASAN builds aren't being copied correctly

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Take a look at a Windows 2012 x64 asan Bo build, and you'll find the following:

>  mozmake.EXE[4]: Circular ../../dist/bin/crashinject.pdb <- ../../dist/bin/crashinject.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../dist/bin/crashinject.pdb <- ../../dist/bin/crashinject.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../dist/bin/plugin-hang-ui.pdb <- ../../../../dist/bin/plugin-hang-ui.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../dist/bin/plugin-hang-ui.pdb <- ../../../../dist/bin/plugin-hang-ui.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/minidumpwriter.pdb <- ../../../dist/bin/minidumpwriter.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/minidumpwriter.pdb <- ../../../dist/bin/minidumpwriter.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/screenshot.pdb <- ../../../dist/bin/screenshot.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/screenshot.pdb <- ../../../dist/bin/screenshot.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/maintenanceservice.pdb <- ../../../dist/bin/maintenanceservice.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/maintenanceservice.pdb <- ../../../dist/bin/maintenanceservice.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../dist/bin/pingsender.pdb <- ../../../../dist/bin/pingsender.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../dist/bin/pingsender.pdb <- ../../../../dist/bin/pingsender.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../dist/bin/updater.pdb <- ../../../../dist/bin/updater.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../dist/bin/updater.pdb <- ../../../../dist/bin/updater.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../dist/bin/firefox.pdb <- ../../dist/bin/firefox.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../dist/bin/firefox.pdb <- ../../dist/bin/firefox.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/signmar.pdb <- ../../../dist/bin/signmar.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/signmar.pdb <- ../../../dist/bin/signmar.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../../dist/bin/pk12util.pdb <- ../../../../../dist/bin/pk12util.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../../dist/bin/pk12util.pdb <- ../../../../../dist/bin/pk12util.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../../dist/bin/shlibsign.pdb <- ../../../../../dist/bin/shlibsign.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../../dist/bin/shlibsign.pdb <- ../../../../../dist/bin/shlibsign.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/ssltunnel.pdb <- ../../../dist/bin/ssltunnel.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/ssltunnel.pdb <- ../../../dist/bin/ssltunnel.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../../dist/bin/certutil.pdb <- ../../../../../dist/bin/certutil.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../../dist/bin/certutil.pdb <- ../../../../../dist/bin/certutil.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../../dist/bin/modutil.pdb <- ../../../../../dist/bin/modutil.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../../dist/bin/modutil.pdb <- ../../../../../dist/bin/modutil.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/gdb-tests.pdb <- ../../../dist/bin/gdb-tests.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/gdb-tests.pdb <- ../../../dist/bin/gdb-tests.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/js.pdb <- ../../../dist/bin/js.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/js.pdb <- ../../../dist/bin/js.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/jsapi-tests.pdb <- ../../../dist/bin/jsapi-tests.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/jsapi-tests.pdb <- ../../../dist/bin/jsapi-tests.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../dist/bin/webrtc-gtest.pdb <- ../../../../dist/bin/webrtc-gtest.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../../dist/bin/webrtc-gtest.pdb <- ../../../../dist/bin/webrtc-gtest.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../dist/bin/plugin-container.pdb <- ../../dist/bin/plugin-container.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../dist/bin/plugin-container.pdb <- ../../dist/bin/plugin-container.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/xpcshell.pdb <- ../../../dist/bin/xpcshell.pdb dependency dropped.
>  mozmake.EXE[4]: Circular ../../../dist/bin/xpcshell.pdb <- ../../../dist/bin/xpcshell.pdb dependency dropped.

It seems like these things aren't being packaged anywhere, so this doesn't matter. But trying to package them (e.g. in Bug 1505936) does result in an error.

Comment 1

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Attachment: Bug 1506016 - If we're building with MOZ_COPY_PDBS, build the pdb next to the executable r?ted

The attached patch seems to work; I ran it in https://treeherder.mozilla.org/#/jobs?repo=try&revision=8b7713d0709d6d272c9cc623698f5514b0a8093f&selectedJob=210745223 

The pdbs show up in target.cppunittest.tests.tar.gz (with Bug 1505936's patch anyway) and target.zip (where they were before). Hasn't done anything to non-ASAN builds.

Strangely enough though this doesn't fix the "Circular foo.pdb <- foo.pdb dependency dropped." messages though. I'm not sure where they're coming from!

Comment 2

[(not currently active) Ted Mielczarek]

Comment on attachment 9023966 [details] [diff] [review]
Bug 1506016 - If we're building with MOZ_COPY_PDBS, build the pdb next to the executable r?ted

Review of attachment 9023966 [details] [diff] [review]:
-----------------------------------------------------------------

I think it would be less messy to just change the definition of `LINK_PDBFILE` when `MOZ_COPY_PDBS` is set:
https://dxr.mozilla.org/mozilla-central/rev/c2593a3058afdfeaac5c990e18794ee8257afe99/config/rules.mk#130

Also with that change (or something like it) you could presumably just drop `PROGRAMS` and `SIMPLE_PROGRAMS` from `DUMP_SYMS_TARGETS` when `MOZ_COPY_PDBS` is set to avoid that circular dependency issue:
https://dxr.mozilla.org/mozilla-central/rev/c2593a3058afdfeaac5c990e18794ee8257afe99/config/rules.mk#820

Comment 3

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Attachment: Bug 1506016 - If we're building with MOZ_COPY_PDBS, build the pdb next to the executable r?ted

Okay, I fiddled around a bit, and I came up with the following that works. (And a big comment.)

I couldn't find a difference between the change to DUMP_SYMS_TARGETS though...

Here's a run with
DUMP_SYMS_TARGETS := $(SHARED_LIBRARY)
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a555cae8a829566a645634fa2397bc79a995c71d

And a run with
DUMP_SYMS_TARGETS := $(SHARED_LIBRARY) $(PROGRAM) $(SIMPLE_PROGRAMS)
https://treeherder.mozilla.org/#/jobs?repo=try&revision=65d87cbc193847f8a2ce46ee673bf04958b66616

I compared target.cppunittest.tests.tar.gz for each and they were the same. I'm a little confused about what this is supposed to change so maybe you could double check or tell me what to double check?

Comment 4

[Tom Ritter [:tjr] (OOTO until 4/30 at least)]

Attachment: Bug 1506016 - If we're building with MOZ_COPY_PDBS, build the pdb next to the executable r?#build

This will allow us to copy the pdbs for test files into the resulting
test file archive, but doesn't do it automatically.

Bug 1505936 is for copying some of them.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5e3d6e5df827
If we're building with MOZ_COPY_PDBS, build the pdb next to the executable r=firefox-build-system-reviewers,mshal

Comment 6

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/5e3d6e5df827