Closed Bug 781792 Opened 8 years ago Closed 5 years ago

test_webgl_conformance_test_suite.html on panda board with fennec yield OOM

Categories

(Core :: Canvas: WebGL, defect)

ARM
Android
defect
Not set
normal

Tracking

()

RESOLVED WONTFIX

People

(Reporter: jmaher, Unassigned)

References

Details

(Keywords: intermittent-failure, Whiteboard: [leave open])

Attachments

(1 file)

I am trying to verify all the existing tests run on the panda boards and I found that I could reproduce an OOM with webgl_conformance_test_suite.html on my panda board (which has 1GB of RAM) inside this test conformance/renderbuffers/framebuffer-object-attachment.html.

I don't know if this is the test that causes it or if it happens to be that something in the suite before this eats up enough memory and this is the test that breaks the pandas back.

Lets figure out how to debug this so we can figure out how to fix this or determine if we need to disable some part of this suite or not.
Does the OOM reproduce by running this framebuffer-object-attachment.html page standalone?

It's here:
https://www.khronos.org/registry/webgl/conformance-suites/1.0.1/conformance/renderbuffers/framebuffer-object-attachment.html

Do you know if it OOMs at a consistent location, either in Gecko's code (get a backtrace in JimDB) or in the test's code (as a starting point, what does adb logcat say?)
Yeah, the issue might be that we're just not releasing GL resources fast enough so by the time we get here we end up OOM'ing even though each individual test is fine.  Though bug 617453 should have helped with that case...
here is what I see in logcat while running just that webl_conformance test suite:
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: readPixels: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 63}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: clear: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 61}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: readPixels: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 63}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: Exceeded 2 live WebGL contexts for this principal, losing the least recently used one." {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/resources/webgl-test.js" line: 49}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: clear: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 61}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: readPixels: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 63}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: clear: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 61}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: readPixels: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 63}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: clear: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 61}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: readPixels: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 63}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: clear: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 61}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: readPixels: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 63}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: clear: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 61}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: readPixels: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 63}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: clear: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 61}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: readPixels: incomplete framebuffer" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 63}]
E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: renderbufferStorage generated error OUT_OF_MEMORY" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 343}]
I/SUTAgentAndroid( 1891): 192.168.1.64 : ps
I/SUTAgentAndroid( 1891): 192.168.1.64 : pull /mnt/sdcard/tests/logs/mochitest.log
W/NetworkManagementSocketTagger( 1412): setKernelCountSet(10046, 0) failed with errno -2
I/WindowManager( 1412): WIN DEATH: Window{4139f6b8 SurfaceView paused=false}
I/ActivityManager( 1412): Process org.mozilla.fennec_jmaher (pid 2237) has died.
W/NetworkManagementSocketTagger( 1412): setKernelCountSet(10044, 1) failed with errno -2
W/ActivityManager( 1412): Force removing ActivityRecord{413b6a88 org.mozilla.fennec_jmaher/.App}: app died, no saved state
I/WindowManager( 1412): WIN DEATH: Window{4156f3c8 org.mozilla.fennec_jmaher/org.mozilla.fennec_jmaher.App paused=false}
This is the one:

E/GeckoConsole( 2237): [JavaScript Warning: "Error: WebGL: renderbufferStorage generated error OUT_OF_MEMORY" {file: "http://mochi.test:8888/tests/content/canvas/test/webgl/conformance/renderbuffers/framebuffer-object-attachment.html" line: 343}]

What's line 343? It is:

    gl.renderbufferStorage(gl.RENDERBUFFER, gl.RGBA4, 0, 0);

So we seem to have a driver bug there, right? And we could easily enough work around it by overriding 0 dimensions as, say, 1.
Summary: test_webgl_conformance_test_suite.html on panda board with fennec yield OOM → test_webgl_conformance_test_suite.html on panda board with fennec yield OOM because driver OOMs on 0-size renderBufferStorage
let me know what to try to ensure it works for us.
Can you try out this patch? It may introduce other failures, I've not checked at all. The questions is: does it fix this particular OOM, as identified in adb logcat having this OUT_OF_MEMORY message pointing to line 343 in that test?
unfortunately that didn't help :(  Same error, same file and line number.
Well, this is fun. It sounds like we need to use 1x1 instead, and pretend it's 0x0 when the user queries it with getRenderbufferParameter.

Luckily, for a framebuffer to be 'complete', it needs to have non-zero width and height, and all attachments must match. We have to assure that attaching a 1x1 texture with a '0x0' (actually 1x1) renderbuffer must not be considered complete.
Blocks: 438871
(In reply to Joel Maher (:jmaher) from comment #7)
> unfortunately that didn't help :(  Same error, same file and line number.

That suggests that my theory was wrong then.

We could disable this particular test page on Android, but then we lose coverage.
Summary: test_webgl_conformance_test_suite.html on panda board with fennec yield OOM because driver OOMs on 0-size renderBufferStorage → test_webgl_conformance_test_suite.html on panda board with fennec yield OOM
I am trying to get a pandaboard to reproduce at the moment.
I got a pandaboard all set up. But Fennec (m-c) can't get to any https page. All I get is "Untrusted Connection" and the browser freezes. Known bug / should I file? Pretty much blocks me. Also, I'm away tomorrow and friday.
Have you made any progress on this?  I don't believe there are issues with the https connection.  I would want to make sure you have the profile setup correctly as well is the proper OS installed that we have for running tests, the image is here:
http://people.mozilla.org/~jmaher/panda8gb.img.bz2
Had been busy with B2G bugs. Getting back to it today.
Check the clock on the pandaboard -- if it's too far out of whack, SSL connections will fail.
Blocks: 803685
on the pandaboard, I have a log of tests which pass but are marked as failing for android (presumably on the tegras).
conformance/extensions/oes-standard-derivatives.html
conformance/glsl/functions/glsl-function-ceil.html
conformance/glsl/functions/glsl-function-clamp-float.html
conformance/glsl/functions/glsl-function-clamp-gentype.html
conformance/glsl/functions/glsl-function-cross.html
conformance/glsl/functions/glsl-function-distance.html
conformance/glsl/functions/glsl-function-dot.html
conformance/glsl/functions/glsl-function-floor.html
conformance/glsl/functions/glsl-function-fract.html
conformance/glsl/functions/glsl-function.html
conformance/glsl/functions/glsl-function-length.html
conformance/glsl/functions/glsl-function-max-float.html
conformance/glsl/functions/glsl-function-max-gentype.html
conformance/glsl/functions/glsl-function-min-float.html
conformance/glsl/functions/glsl-function-min-gentype.html
conformance/glsl/functions/glsl-function-mix-float.html
conformance/glsl/functions/glsl-function-mix-gentype.html
conformance/glsl/functions/glsl-function-mod-float.html
conformance/glsl/functions/glsl-function-mod-gentype.html
conformance/glsl/functions/glsl-function-normalize.html
conformance/glsl/functions/glsl-function-reflect.html
conformance/glsl/functions/glsl-function-smoothstep-float.html
conformance/glsl/functions/glsl-function-smoothstep-gentype.html
conformance/glsl/misc/shared.html
conformance/misc/null-object-behaviour.html
conformance/renderbuffers/framebuffer-object-attachment.html


With those tests removed from the manifest file, this is what remains as failures:
2287 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/extensions/oes-texture-float.html] Test failed - gl.checkFramebufferStatus(gl.FRAMEBUFFER) should be 36053. Was 36054.
2289 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/extensions/oes-texture-float.html] (WebGL test error) 1 failure(s) and 0 timeout(s)
3135 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-smoothstep-float.html] Test failed - images are different
3138 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-smoothstep-float.html] Test failed - images are different
3141 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-smoothstep-float.html] Test failed - images are different
3144 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-smoothstep-float.html] Test failed - images are different
3145 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/glsl/functions/glsl-function-smoothstep-float.html] (WebGL test error) 4 failure(s) and 0 timeout(s)
3758 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/limits/gl-max-texture-dimensions.html] Test failed - Should be 0, 255, 128, 255
3759 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/limits/gl-max-texture-dimensions.html] Test failed - Should be 0, 255, 128, 255
3761 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/limits/gl-max-texture-dimensions.html] (WebGL test error) 2 failure(s) and 0 timeout(s)
3763 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/limits/gl-min-textures.html] Test failed - getError expected: NO_ERROR. Was INVALID_VALUE : Should be no errors from setup.
3764 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/limits/gl-min-textures.html] Test failed - Should render using all texture units
3765 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | [conformance/limits/gl-min-textures.html] (WebGL test error) 2 failure(s) and 0 timeout(s)
jmaher@jmaher-MacBookPro:~/mozilla/src/objdir-droid/dist/test-package-stage/mochitest$ 


There is no OOM, just some failures to figure out.
Bug 810943 adds the ability to skip WebGL test pages on android just by listing them in a file. Once this lands you'll be able to disable WebGL pages on android super easily.
the problem is I have two different versions of android and two different hardware platforms.  How could I differentiate between the two?  Also I would wonder if we should attempt to fix or investigate the remaining 4 webgl tests that report failure(s)?
(In reply to Joel Maher (:jmaher) from comment #17)
> the problem is I have two different versions of android and two different
> hardware platforms.  How could I differentiate between the two?

Can you differentiate them in the same way that we differentiate between windows, android, etc in that mochitest?

Really as long as there is any way that privileged JS code can differentiate these platforms/devices, you can use that in that mochitest. You could then have separate variants of skipped_tests_android.txt for each -- see the patch in bgu 810943.

>  Also I
> would wonder if we should attempt to fix or investigate the remaining 4
> webgl tests that report failure(s)?

To make progress on this, it's very important that we keep separate things separate:
 - adding new test hardware
 - understanding WebGL test failures on each class of test hardware

The way we've managed to keep these things separate so far was by having separate lists of known-to-fail webgl pages on each OS, e.g. failing_tests_android.txt. So, once we've figured how to differentiate these devices in the mochitest, we can have separate variants of failing_tests_android.txt for each of them. That will allow to turn on tests without being blocked on investigating the failures.
gbrown/kats/blassey: do you guys know how we could detect the panda board or the newer android os from inside of firefox?  right now we have webgl tests that fail on tegras, but pass on the panda boards, and some tests which pass on the tegras and fail on the pandas.

If we can differentiate between these test runs, we can successfully test the working webgl stuff on each respective platform.
Well, from inside of _firefox_ of course we can --- the information what we use for blacklisting (GfxInfo) on Android allows to identify devices.

So in fact, you could certainly differentiate the devices from the mochitest, using nsIGfxInfo. I can make you the code for that, but i need to know how the about:support looks for each of these devices (the graphics section and the android version).
Whiteboard: [orange]
Whiteboard: [leave open]
Bug last touched in 2012, and reported against pandas to begin with which are now dead.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.