Crash report submission broken on Android

RESOLVED FIXED in Firefox 55

Status

()

Toolkit
Crash Reporting
--
blocker
RESOLVED FIXED
9 months ago
9 months ago

People

(Reporter: Jukka Jylänki, Assigned: jchen)

Tracking

({regression})

unspecified
mozilla55
ARM
Android
regression
Points:
---

Firefox Tracking Flags

(firefox-esr52 unaffected, firefox53 unaffected, firefox54 unaffected, firefox55+ fixed)

Details

Attachments

(4 attachments)

(Reporter)

Description

9 months ago
Created attachment 8869973 [details]
Screenshot_20170522-123233.png

Running WebGL conformance test suite on Firefox on an Android device (https://www.khronos.org/registry/webgl/sdk/tests/webgl-conformance-tests.html) and it is crashing the browser.

I am unable to access the crash reports however:
  - in Firefox settings, crash reporting, telemetry and Firefox health report are checked.
  - visiting about:crashes shows the crash report as unsubmitted
  - clicking on the link does nothing (no navigation, no user feedback)
  - there is no button to submit the report

I presume perhaps the crash report would get eventually submitted? I can't wait for that however, I need to figure out at the same sitting which URL crashed so that I can get on with work, i.e. I need to disable those crashing URLs and rerun the suite, but without being able to access the crash report to find the crashing URL makes this impossible (there are thousands of tests in the suite, and the suite takes hours to run, so I don't know exactly which URL crashed)

See the attachment for how the about:crashes window looks like.
(Reporter)

Comment 1

9 months ago
Created attachment 8870766 [details]
Pending unsubmitted crash reports are not being sent after a few days have passed
(Reporter)

Comment 2

9 months ago
Created attachment 8870767 [details]
Device settings for submitting data
(Reporter)

Comment 3

9 months ago
After waiting for a few of days, the crash reports are not getting submitted, and there is no ribbon/banner/popup being displayed like there is on desktop Firefox which asks whether to submit crash reports. This is becoming a major blocker.

I've tested this on the following Android devices, which all have the problem:

 - LG Nexus 5 - Android 6.0.1, Kernel 3.4.0-gcf10b7e (hammerhead M4B30Z), Fennec 55.0a1 (2017-05-19) - Qualcomm Adreno 330, OpenGL ES 3.0 V@127.9 AU@(GIT@I98aee987eb)

 - Google Pixel XL - Android 7.1.2, Kernel 3.18.31-g416bf43 (marlin N2G47O), Fennec 55.0a1 (2017-05-19) - Qualcomm Adreno 530, OpenGL ES 3.2 V@145.0 (GIT@Idb2b4cb785)

 - Samsung Galaxy S7 Edge SM-G935F - Android 7.0, Kernel 3.18.14-11104523 (NRD90M), Fennec 55.0a1 (2017-05-19), ARM Mali-T880, OpenGL ES 3.2 v1.r12p1-03dev0.228ab63cced004f840e7dd47b762a1d0

 - Huawei P10 Plus - Android 7.0, Kernel 4.1.18-gfd75bbb (VKY-L29), Fennec 55.0a1 (2017-05-19) - ARM Mali-G71, OpenGL ES 3.2 v1.r2p0-02dev0.f7269486f3e0e3b308edf85872e361f4

Attempting to click the unsubmitted crash reports does nothing. Is there a way to somehow force sending of unsubmitted crash reports on Android?

To get Firefox Nightly to crash, visit the following page:

  https://www.khronos.org/registry/webgl/sdk/tests/webgl-conformance-tests.html?version=2.0.1

and run all the tests. From what I'm seeing, this suggests that crash reports are not getting though on Android at all. (Or perhaps there is some heuristic that is affecting all the reports my test devices are generating)
Severity: normal → blocker
Hello Ted - Can you help us find an owner for this bug?
Flags: needinfo?(ted)
snorp: this sounds Android-specific. Can you help or pass this off to someone else with Android expertise?

Jukka: when the browser crashes, do you see the crash reporter dialog? It should look like this: http://imgur.com/a/MHl4H

If the crashreporter dialog isn't showing, and tapping on crashes in about:crashes doesn't work, sounds like we have two bugs.
Flags: needinfo?(ted) → needinfo?(snorp)
FWIW, I just tried reproducing the crash on Nightly using a Nexus 6 running Android 7. I did get a crash report, but when I clicked on the link it said the report was throttled - and I do see the report listed in "Unsubmitted Crash Reports."

When the browser crashed, I did get the crash reporter dialog.
Submission from about:crashes being broken is already filed as bug 1262812.
OK, I can reproduce the failure to submit locally. In logcat I see:
```
05-24 11:08:51.849 28714 28762 I GeckoCrashReporter: server url: https://crash-reports.mozilla.com/submit?id={aa3c5121-dab2-40e2-81ca-7ea25febc110}&version=55.0a1&buildid=20170510123633
05-24 11:08:51.853 28714 28762 E GeckoCrashReporter: exception during new URI: 
05-24 11:08:51.853 28714 28762 E GeckoCrashReporter: java.net.URISyntaxException: Illegal character in query at index 44: https://crash-reports.mozilla.com/submit?id={aa3c5121-dab2-40e2-81ca-7ea25febc110}&version=55.0a1&buildid=20170510123633
05-24 11:08:51.853 28714 28762 E GeckoCrashReporter: 	at java.net.URI$Parser.fail(URI.java:2855)
05-24 11:08:51.853 28714 28762 E GeckoCrashReporter: 	at java.net.URI$Parser.checkChars(URI.java:3028)
05-24 11:08:51.853 28714 28762 E GeckoCrashReporter: 	at java.net.URI$Parser.parseHierarchical(URI.java:3118)
05-24 11:08:51.853 28714 28762 E GeckoCrashReporter: 	at java.net.URI$Parser.parse(URI.java:3060)
05-24 11:08:51.853 28714 28762 E GeckoCrashReporter: 	at java.net.URI.<init>(URI.java:589)
05-24 11:08:51.853 28714 28762 E GeckoCrashReporter: 	at org.mozilla.gecko.CrashReporter.sendReport(CrashReporter.java:413)
05-24 11:08:51.853 28714 28762 E GeckoCrashReporter: 	at org.mozilla.gecko.CrashReporter.access$300(CrashReporter.java:48)
05-24 11:08:51.853 28714 28762 E GeckoCrashReporter: 	at org.mozilla.gecko.CrashReporter$7.run(CrashReporter.java:244)
05-24 11:08:51.853 28714 28762 E GeckoCrashReporter: 	at java.lang.Thread.run(Thread.java:761)
```

Looking at the line where the error occurs, it seems to have been changed in bug 1357997, so I believe this is a regression from that bug:
https://dxr.mozilla.org/mozilla-central/rev/96e18bec9fc8a5ce623c16167c12756bbe190d73/mobile/android/base/java/org/mozilla/gecko/CrashReporter.java#413

Specifically, that patch changed:
https://hg.mozilla.org/mozilla-central/rev/676720d93349#l1.39
-            URL url = new URL(spec);
-            HttpURLConnection conn = (HttpURLConnection)url.openConnection();
+            URI uri = new URI(spec);
+            HttpURLConnection conn = (HttpURLConnection)ProxySelector.openConnectionWithProxy(uri);

So I guess our URL (which includes {}) parsed as a URL but not a URI? Exciting! I am sad that nobody noticed this until now. Given that we don't have automated testing for crash report submission, we should at least be doing targeted manual QA when we change the crashreporter code.
Blocks: 1357997
status-firefox55: --- → affected
tracking-firefox55: --- → ?
I would guess the simplest fix here would be to encode the URL somehow before passing it to the URI constructor, but my Java is extremely rusty.
Summary: Cannot access crash reports on an android device → Crash report submission broken on Android
(Reporter)

Comment 10

9 months ago
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #5)
> Jukka: when the browser crashes, do you see the crash reporter dialog? It
> should look like this: http://imgur.com/a/MHl4H
> 
> If the crashreporter dialog isn't showing, and tapping on crashes in
> about:crashes doesn't work, sounds like we have two bugs.

Yeah, when the crashes do occur, the crashreporter dialog does come up, and I leave the checkboxes ticked and then choose "Restart Nightly".
Jim, can you take a look at the crash reporter issue in comment #8, looks like it should be a relatively simple fix.

Jukka, you'll have better luck debugging crashes with a local build. You can hook up gdb, etc. Use 'mach run --debug' after building.
Flags: needinfo?(snorp) → needinfo?(nchen)
Tracking 55+ for this issue - important to have it working.
tracking-firefox55: ? → +
(Assignee)

Updated

9 months ago
Assignee: nobody → nchen
Status: NEW → ASSIGNED
Flags: needinfo?(nchen)
Keywords: regression
(Assignee)

Comment 13

9 months ago
Created attachment 8870962 [details] [diff] [review]
Encode crash submission URI correctly (v1)

Correctly encode the crash submission URI so it's valid.
Attachment #8870962 - Flags: review?(snorp)
Attachment #8870962 - Flags: review?(snorp) → review+

Comment 14

9 months ago
Pushed by nchen@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/60e0d98a2ae6
Encode crash submission URI correctly; r=snorp

Comment 15

9 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/60e0d98a2ae6
Status: ASSIGNED → RESOLVED
Last Resolved: 9 months ago
status-firefox55: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
status-firefox53: --- → unaffected
status-firefox54: --- → unaffected
status-firefox-esr52: --- → unaffected
You need to log in before you can comment on or make changes to this bug.