`mach buildsymbols` hangs while gzip asks if I want to overwrite libclearkey.so.dbg.gz

RESOLVED FIXED in mozilla36

Status

()

Core
Audio/Video
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: jld, Assigned: mshal)

Tracking

Trunk
mozilla36
x86_64
Linux
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 1 obsolete attachment)

(Reporter)

Description

3 years ago
I'm not sure whose bug this is, but I'll start with Video/Audio because this seems to happen only to libclearkey and nothing else in the build.

STR: run "./mach buildsymbols" after doing a desktop build

At some point, gzip writes to the terminal as follows:

gzip: dist/crashreporter-symbols/libclearkey.so/FBD67515E0566373B3B48BC83385393C0/libclearkey.so.dbg.gz already exists; do you wish to overwrite (y or n)?

This gets buried under hundreds of lines of other output, because a lot of stuff is going on in parallel, so it's very much not obvious that gzip is sitting there waiting patiently for me to type something and blocking the build until that happens.

This also reproduced after deleting dist/crashreporter-symbols.
(Assignee)

Comment 1

3 years ago
Created attachment 8511218 [details] [diff] [review]
0001-Bug-1087088-don-t-connect-stdin-to-buildsymbols.patch

I hit this too - looks like fallout from bug 1044742. ted, should libclearkey.so be installed via an INSTALL_TARGET in the Makefile as well as listed as a SharedLibrary in moz.build? I'd guess the INSTALL_TARGET should go, but then the lib won't be available in dist/bin/gmp-clearkey/0.1. The problem is that buildsymbols tries to build both versions, so the second one waits for the user prompt:

16435: Submitting jobs for files: ('dist/bin/libclearkey.so',)
16435: Submitting jobs for files: ('dist/bin/gmp-clearkey/0.1/libclearkey.so',)

Attached is a patch that just disconnects stdin from buildsymbols, which just works around the problem by behaving more like it would in automation. We probably also want to fix the double-library situation.
Attachment #8511218 - Flags: review?(ted)
I assume something is looking for it in that location in dist/bin, so if we stop putting it there we'll need to fix whatever that is. I'm not sure if we can ask the build system to build it there directly instead of putting it in dist/bin and then copying it. glandium: is that a thing we could do?
(Assignee)

Updated

3 years ago
Flags: needinfo?(mh+mozilla)
Putting SHARED_LIBRARY in CLEARKEY_CDM_FILES in
https://hg.mozilla.org/mozilla-central/diff/3855948c6f30/media/gmp-clearkey/0.1/Makefile.in
causes the file to be copied to dist/bin/gmp-clearkey/0.1.

But SHARED_LIBRARY is also in SHARED_LIBRARY_FILES in
https://hg.mozilla.org/mozilla-central/file/3855948c6f30/config/makefiles/target_binaries.mk#l44
which causes the file to be copied to dist/bin.

Note the destination for the latter is $(FINAL_TARGET).

The right thing to do is to set FINAL_TARGET to 'dist/bin/gmp-clearkey/0.1' in media/gmp-clearkey/0.1/moz.build.
Flags: needinfo?(mh+mozilla)
(Assignee)

Comment 4

3 years ago
Created attachment 8514302 [details] [diff] [review]
0001-Bug-1087088-Don-t-install-libclearkey-to-dist-bin.patch

I guess we can leave buildsymbols as is for now to help catch any future occurrences.
Assignee: nobody → mshal
Attachment #8511218 - Attachment is obsolete: true
Attachment #8511218 - Flags: review?(ted)
Attachment #8514302 - Flags: review?(ted)
Comment on attachment 8514302 [details] [diff] [review]
0001-Bug-1087088-Don-t-install-libclearkey-to-dist-bin.patch

Review of attachment 8514302 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks! Ideally we'd fail with a clearer error here, maybe we ought to take your other patch anyway?
Attachment #8514302 - Flags: review?(ted) → review+
(Assignee)

Comment 6

3 years ago
Created attachment 8514335 [details] [diff] [review]
0001-Bug-1087088-buildsymbols-should-fail-if-there-are-du.patch

How's this? The error message can be buried back in the log, but at least searching for "Error" will help reveal the problem.
Attachment #8514335 - Flags: review?(ted)
(Assignee)

Comment 7

3 years ago
Comment on attachment 8514335 [details] [diff] [review]
0001-Bug-1087088-buildsymbols-should-fail-if-there-are-du.patch

Hmm, actually this fails on OSX because libjsctypes-test.dylib is installed into two locations as well. I'm not quite sure why the same thing doesn't fail on Linux, since it is installed twice there as well. Only OSX processes them in symbolstore.py, though.

Plus it made my try push timeout rather than just fail, which is odd. It didn't do that locally.
Attachment #8514335 - Flags: review?(ted)
(Assignee)

Comment 8

3 years ago
inbound: https://hg.mozilla.org/integration/mozilla-inbound/rev/3b7bdfbb0988
https://hg.mozilla.org/mozilla-central/rev/3b7bdfbb0988
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla36
You need to log in before you can comment on or make changes to this bug.