Autophone - Mochitest Skia failures

RESOLVED FIXED

Status

()

Firefox for Android
General
RESOLVED FIXED
2 years ago
3 months ago

People

(Reporter: bc, Unassigned)

Tracking

(Blocks: 2 bugs, {leave-open})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(4 attachments)

(Reporter)

Comment 1

2 years ago
Created attachment 8752491 [details]
Android 4.4 / Nexus 5 Errors
(Reporter)

Comment 2

2 years ago
Created attachment 8752492 [details]
Android 6.0.1 / Nexus 6p Errors
(Reporter)

Comment 3

2 years ago
snorp: If we aren't able to fix these issues with Msk or if it is not a priority, I'll remove Msk from the non-try production runs so we will not be in a perma-orange state. Let me know how you'd like me to handle these.
Flags: needinfo?(snorp)
Milan, do you think you could get someone to look at these failures?
Flags: needinfo?(snorp) → needinfo?(milan)
A bit of background here - these used to run, but started failing, or never ran, or we updated the version of Android and then it started failing, or...?
George, can you find out some of the history on these?
Flags: needinfo?(milan) → needinfo?(gwright)
(Reporter)

Comment 6

2 years ago
Snorp asked for these ages ago where they have been available on try but we never got them in shape to run regularly partly due to perma-orange.

In an attempt to find out which tests we could run without failures I added them to fx-team mozilla-central mozilla-beta mozilla-aurora mozilla-release in March but removed them from anything but mozilla-central in April since they were causing load issues due to timeouts/perma-orange.

If we don't care about these I intend to remove them from mozilla-central before the end of June and just keep them available on try.
I'm going to assume we care or :snorp wouldn't have tagged us, so, George, let's see what more we can find out about these failures.
For some reason I never knew this existed in the first place. As far as I can tell, it looks like these have been around for nearly 3 years? The original bug is bug 893611. It's just a normal mochitest canvas test run on Android but with skiagl preffed on. What puzzles me is that the prefs used for this are the same as the default on android anyway (https://dxr.mozilla.org/mozilla-central/source/mobile/android/app/mobile.js#819), and we don't have any perma-orange on our normal mochitest runs, so why is this failing?

Looking at the log, there are only three actual test failures:

452 INFO TEST-UNEXPECTED-FAIL | dom/canvas/test/test_imagebitmap_cropping.html | pixel 100,100 of  is 255,255,255,247; expected 255,255,255,255 +/- 5 
207 INFO TEST-UNEXPECTED-FAIL | dom/canvas/test/test_imagebitmap.html | pixel 384,0 of  is 4,251,0,255; expected 0,255,0,255 +/- 1 
538 INFO TEST-UNEXPECTED-FAIL | dom/canvas/test/test_offscreencanvas_dynamic_fallback.html | after dynamic fallback, screenshots should be the same 

But we do get an EGL error:

05-13 18:21:23.220 E/libEGL  ( 3540): eglMakeCurrent:775 error 3002 (EGL_BAD_ACCESS)
05-13 18:21:23.220 W/Adreno-EGL( 3540): <qeglDrvAPI_eglMakeCurrent:2919>: EGL_BAD_ACCESS

This may be harmless though.

Bob, are there any differences between our "normal" android mochitests run on m-c, and the autophone-based Mochitest-Skia here? Are the devices themselves different?
Flags: needinfo?(gwright) → needinfo?(bob)
(Reporter)

Comment 9

2 years ago
gbrown can provide more detail I think, but my understanding is the "normal" android mochitests run on emulators while the autophone tests run on actual devices.

If you click on the job symbol, you will see the device name in the left panel. We currently have Nexus 4, Nexus 5, Nexus 6, Nexus 6P, Nexus 9 and a smattering of Nexus S which are being phased out as Android 2.3 support leaves the station.

I would guess that the different screen sizes might come into play here.
Flags: needinfo?(bob) → needinfo?(gbrown)
The "normal" tier-1 mochitests on treeherder run on Android 4.3.1 (slight variant of JLS36I), on the android emulator with a screen size of 800 wide x 1280 high. This bug is about autophone, running on various real devices.
Flags: needinfo?(gbrown)
tracking-fennec: ? → 49+
Created attachment 8779044 [details] [diff] [review]
0001-Bug-1272878-Fuzz-test_bitmaprenderer.html-because-of.patch

bitmaprenderer uses an ImageLayer instead of a CanvasLayer, which takes a totally different codepath to render images than canvas2d, and result in some anti-aliasing differences at the edge of the image.

Both codepaths apply some anti-aliasing (canvas2d in this case has 254,255,254 as the edge pixels, and bitmaprenderer has 240,255,240), so it's not a case of AA vs. non-AA, which I think would be problematic.

It seems reasonable to fuzz this as AA differences when using different codepaths isn't something we've traditionally been fussed about.
Attachment #8779044 - Flags: review?(mtseng)
leave open for now, as there are a couple of other issues that need resolving.
Keywords: leave-open
Comment on attachment 8779044 [details] [diff] [review]
0001-Bug-1272878-Fuzz-test_bitmaprenderer.html-because-of.patch

Review of attachment 8779044 [details] [diff] [review]:
-----------------------------------------------------------------

Great, thanks for fix.
Attachment #8779044 - Flags: review?(mtseng) → review+

Comment 14

2 years ago
Pushed by gwright@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e2d1e50ec9f5
Fuzz test_bitmaprenderer.html because of differences in our two codepaths with antialiasing r=Morris
Bob, once this hits central we can probably enable Msk on Android 5 and 6:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9916bcddfd85&exclusion_profile=false
Flags: needinfo?(bob)
(Reporter)

Comment 16

2 years ago
https://github.com/mozilla/autophone/commit/b19e073039d9e50db74f7d065ac5e8b698ea7b94

deployed 2016-08-10 ~00:00 PDT

I expect the next mozilla-central build will have the merge. /me crosses fingers.
Flags: needinfo?(bob)
(Reporter)

Comment 18

2 years ago
I messed up and enabled Nexus 5 which of course is still failing. When the current push on m-c finishes, I'll update the manifest to disable it again. We'll be running Msk on Nexus 6P only.
(Reporter)

Comment 19

2 years ago
disable Nexus 5 on Msk
https://github.com/mozilla/autophone/commit/3a2e6bd5f2261d20e59917b3afbf272cad072870
deployed 2016-08-10 ~08:55:00 PDT.
(Reporter)

Comment 20

a year ago
filed bug 1294491 for the assertion errors.
Blocks: 1281640
tracking-fennec: 49+ → ---
(Reporter)

Updated

a year ago
Depends on: 1299847
(Reporter)

Updated

3 months ago
No longer blocks: 1281640
(Reporter)

Updated

3 months ago
Blocks: 1281640
(Reporter)

Comment 21

3 months ago
-> fixed
Status: NEW → RESOLVED
Last Resolved: 3 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.