Closed Bug 1499915 Opened 11 months ago Closed 11 months ago

no libxul symbols for Android xpcshell crashes

Categories

(Firefox Build System :: Android Studio and Gradle Integration, defect, major)

defect
Not set
major

Tracking

(firefox63 wontfix, firefox64 fixed, firefox65 fixed)

RESOLVED FIXED
mozilla65
Tracking Status
firefox63 --- wontfix
firefox64 --- fixed
firefox65 --- fixed

People

(Reporter: gbrown, Assigned: glandium)

References

(Blocks 1 open bug)

Details

(Keywords: regression)

Attachments

(3 files)

+++ This bug was initially created as a clone of Bug #1389805 +++

In bug 1389805 we saw no symbols in Android xpcshell crash reports and addressed that with 'elfhack -r'; that is no longer working.

https://treeherder.mozilla.org/logviewer.html#?job_id=206214577&repo=try&lineNumber=1756

[task 2018-10-17T20:53:21.173Z] 20:53:21     INFO -   0  libxul.so + 0x282082
[task 2018-10-17T20:53:21.174Z] 20:53:21     INFO -       r0 = 0x00000180    r1 = 0x42e6ee40    r2 = 0x00000000    r3 = 0x00000000
[task 2018-10-17T20:53:21.174Z] 20:53:21     INFO -       r4 = 0x4581d800    r5 = 0x00000000    r6 = 0x00020001    r7 = 0xbec27930
[task 2018-10-17T20:53:21.175Z] 20:53:21     INFO -       r8 = 0x00000000    r9 = 0x5103c450   r10 = 0x00000000   r12 = 0x00000000
[task 2018-10-17T20:53:21.175Z] 20:53:21     INFO -       fp = 0x458d8200    sp = 0xbec27918    lr = 0x00000000    pc = 0x40922082
[task 2018-10-17T20:53:21.176Z] 20:53:21     INFO -      Found by: given as instruction pointer in context
[task 2018-10-17T20:53:21.176Z] 20:53:21     INFO -   1  libxul.so + 0x24bba5
[task 2018-10-17T20:53:21.177Z] 20:53:21     INFO -       sp = 0xbec27938    pc = 0x408ebba7
[task 2018-10-17T20:53:21.177Z] 20:53:21     INFO -      Found by: stack scanning
[task 2018-10-17T20:53:21.178Z] 20:53:21     INFO -   2  libxul.so + 0x24bb1f
[task 2018-10-17T20:53:21.178Z] 20:53:21     INFO -       sp = 0xbec2796c    pc = 0x408ebb21
[task 2018-10-17T20:53:21.179Z] 20:53:21     INFO -      Found by: stack scanning
We run elfhack just before pushing, so these "Not elfhacked" messages are likely coming from elfhack:

https://treeherder.mozilla.org/logviewer.html#?job_id=206214577&repo=try&lineNumber=1289

[task 2018-10-17T20:46:28.210Z] 20:46:28     INFO -  Pushing assets/armeabi-v7a/libfreebl3.so..
[task 2018-10-17T20:46:28.230Z] 20:46:28     INFO -  /tmp/tmp6Wf7nr/assets/armeabi-v7a/libfreebl3.so: Not elfhacked. Skipping
[task 2018-10-17T20:46:28.536Z] 20:46:28     INFO -  Pushing assets/armeabi-v7a/liblgpllibs.so..
[task 2018-10-17T20:46:28.544Z] 20:46:28     INFO -  /tmp/tmp6Wf7nr/assets/armeabi-v7a/liblgpllibs.so: Not elfhacked. Skipping
[task 2018-10-17T20:46:28.751Z] 20:46:28     INFO -  Pushing assets/armeabi-v7a/libmozavcodec.so..
[task 2018-10-17T20:46:28.762Z] 20:46:28     INFO -  /tmp/tmp6Wf7nr/assets/armeabi-v7a/libmozavcodec.so: .elfhack.data.v0 and .elfhack.text.v0 not in the same segment. Skipping
[task 2018-10-17T20:46:28.969Z] 20:46:28     INFO -  Pushing assets/armeabi-v7a/libmozavutil.so..
[task 2018-10-17T20:46:28.981Z] 20:46:28     INFO -  /tmp/tmp6Wf7nr/assets/armeabi-v7a/libmozavutil.so: Not elfhacked. Skipping
[task 2018-10-17T20:46:29.187Z] 20:46:29     INFO -  Pushing assets/armeabi-v7a/libnss3.so..
[task 2018-10-17T20:46:29.253Z] 20:46:29     INFO -  /tmp/tmp6Wf7nr/assets/armeabi-v7a/libnss3.so: .elfhack.data.v0 and .elfhack.text.v0 not in the same segment. Skipping
[task 2018-10-17T20:46:29.860Z] 20:46:29     INFO -  Pushing assets/armeabi-v7a/libnssckbi.so..
[task 2018-10-17T20:46:29.881Z] 20:46:29     INFO -  /tmp/tmp6Wf7nr/assets/armeabi-v7a/libnssckbi.so: Not elfhacked. Skipping
[task 2018-10-17T20:46:30.188Z] 20:46:30     INFO -  Pushing assets/armeabi-v7a/libsoftokn3.so..
[task 2018-10-17T20:46:30.201Z] 20:46:30     INFO -  /tmp/tmp6Wf7nr/assets/armeabi-v7a/libsoftokn3.so: Not elfhacked. Skipping
[task 2018-10-17T20:46:30.407Z] 20:46:30     INFO -  Pushing assets/armeabi-v7a/libxul.so..
[task 2018-10-17T20:46:32.444Z] 20:46:32     INFO -  /tmp/tmp6Wf7nr/assets/armeabi-v7a/libxul.so: .elfhack.data.v0 and .elfhack.text.v0 not in the same segment. Skipping
[task 2018-10-17T20:46:56.902Z] 20:46:56     INFO -  Pushing lib/armeabi-v7a/libmozglue.so..
[task 2018-10-17T20:46:57.621Z] 20:46:57     INFO -  Pushing lib/armeabi-v7a/libplugin-container.so..
Whiteboard: [stockwell fixed:other]
Blocks: 1423822
See Also: → 1499454
Assignee: nobody → mh+mozilla
This effectively backs out bug 822584, which worked around a similar
problem to what we are facing with Android xpcshell, being that the
crash reporter doesn't handle the address space "fragmentation" induced
by elfhack. The work around worked, at the expense of some added
complexity.

It was used for B2G only, and has effectively been unused since B2G was
retired.
This is some sort of followup to bug 1423813, providing a minimalistic
way to undo elfhack when the elfhack sections are in separate segments,
which has been the case since bug 1385783 but didn't cause problems
on Android builds until bug 1423822.

Depends on D9622
Android xpcshell tests run `elfhack -r`, and do so by using an
elfhack binary from an archive downloaded via tooltool (which contains
other things).

Instead, we just put the elfhack binary in the
target.common.tests.tar.gz archive, and make the xpcshell harness
use it from there instead, allowing to pick in-tree changes to elfhack.

Depends on D9623
Pushed by mh@glandium.org:
https://hg.mozilla.org/integration/autoland/rev/0988f87f6bd2
Remove support for the elfhack filler segment r=froydnj
https://hg.mozilla.org/integration/autoland/rev/550a527e90ee
Support undoing elfhack when the elfhacked sections are in separate segments r=froydnj
https://hg.mozilla.org/integration/autoland/rev/fbe393405a4a
Use elfhack from the build for Android xpcshell tests r=froydnj
https://hg.mozilla.org/mozilla-central/rev/0988f87f6bd2
https://hg.mozilla.org/mozilla-central/rev/550a527e90ee
https://hg.mozilla.org/mozilla-central/rev/fbe393405a4a
Status: NEW → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 65
Is this low-enough risk to consider uplifting to Beta? If so, please nominate.
Flags: needinfo?(mh+mozilla)
Comment on attachment 9019583 [details]
Bug 1499915 - Remove support for the elfhack filler segment

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: Bug 1423822

User impact if declined: Automation doesn't get symbolication on xpcshell crashes

Is this code covered by automated tests?: Yes

Has the fix been verified in Nightly?: Yes

Needs manual test from QE?: No

If yes, steps to reproduce: 

List of other uplifts needed: None

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): This patch removes support for an unused flag of elfhack.

String changes made/needed: N/A
Flags: needinfo?(mh+mozilla)
Attachment #9019583 - Flags: approval-mozilla-beta?
Comment on attachment 9019584 [details]
Bug 1499915 - Support undoing elfhack when the elfhacked sections are in separate segments

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: Bug 1423822

User impact if declined: Automation doesn't get symbolication on xpcshell crashes

Is this code covered by automated tests?: Yes

Has the fix been verified in Nightly?: Yes

Needs manual test from QE?: No

If yes, steps to reproduce: 

List of other uplifts needed: None

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): This changes a code path that is only exercised when running xpcshell tests.

String changes made/needed: N/A
Attachment #9019584 - Flags: approval-mozilla-beta?
Comment on attachment 9019585 [details]
Bug 1499915 - Use elfhack from the build for Android xpcshell tests

[Beta/Release Uplift Approval Request]

Feature/Bug causing the regression: Bug 1423822

User impact if declined: Automation doesn't get symbolication on xpcshell crashes

Is this code covered by automated tests?: Yes

Has the fix been verified in Nightly?: Yes

Needs manual test from QE?: No

If yes, steps to reproduce: 

List of other uplifts needed: None

Risk to taking this patch: Low

Why is the change risky/not risky? (and alternatives if risky): This makes Android xcpshell tests use elfhack from the Android build rather than one that was built n months ago.

String changes made/needed:
Attachment #9019585 - Flags: approval-mozilla-beta?
Comment on attachment 9019583 [details]
Bug 1499915 - Remove support for the elfhack filler segment

[Triage Comment]
Allows us to get usable stacks from Android xpcshell crashes in automation. Approved for 64.0b6.
Attachment #9019583 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Attachment #9019584 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Attachment #9019585 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Product: Firefox for Android → Firefox Build System
Target Milestone: Firefox 65 → mozilla65
You need to log in before you can comment on or make changes to this bug.