Closed Bug 688052 Opened 14 years ago Closed 2 years ago

Perma-orange Android TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityEventUtils.html | Test timed out. (followed by >100 more failures)

Categories

(Testing :: Mochitest, defect)

Product:
Testing
Component:
Mochitest
Platform:
ARM
Android
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 1704064

People

(Reporter: mbrubeck, Unassigned)

References

Details

(Whiteboard: [mobile_unittests][test disabled on Android])

Attachments

(1 file, 1 obsolete file)

Disable 3 broken Android tests.
1.02 KB, patch
mbrubeck
: review+
Details | Diff | Splinter Review
https://tbpl.mozilla.org/php/getParsedLog.php?id=6474086&tree=Mozilla-Inbound and https://tbpl.mozilla.org/php/getParsedLog.php?id=6472144&tree=Mozilla-Inbound Android Tegra 250 mozilla-inbound opt test mochitest-7 on 2011-09-20 13:33:19 PDT for push 1e41259daf67 35 INFO TEST-END | /tests/Harness_sanity/test_sanity.html | finished in 342ms 36 INFO TEST-START | /tests/Harness_sanity/test_sanityEventUtils.html 37 INFO TEST-INFO | /tests/Harness_sanity/test_sanityEventUtils.html | Profile::EventUtilsLoadTime: 18 38 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityEventUtils.html | Test timed out. 39 INFO TEST-END | /tests/Harness_sanity/test_sanityEventUtils.html | finished in 325158ms 40 INFO TEST-START | /tests/Harness_sanity/test_sanityException.html 41 INFO TEST-PASS | /tests/Harness_sanity/test_sanityException.html | a call to ok 42 INFO TEST-PASS | /tests/Harness_sanity/test_sanityException.html | [SimpleTest/SimpleTest.js, window.onerror] - An error occurred: uncaught exception: an uncaught exception at :0 43 INFO TEST-END | /tests/Harness_sanity/test_sanityException.html | finished in 137ms 44 INFO TEST-START | /tests/Harness_sanity/test_sanityException2.html 45 INFO TEST-PASS | /tests/Harness_sanity/test_sanityException2.html | a call to ok 46 INFO TEST-PASS | /tests/Harness_sanity/test_sanityException2.html | [SimpleTest/SimpleTest.js, window.onerror] - An error occurred: uncaught exception: an uncaught exception at :0 47 INFO TEST-END | /tests/Harness_sanity/test_sanityException2.html | finished in 85ms 48 INFO TEST-START | /tests/Harness_sanity/test_sanityPluginUtils.html 49 INFO TEST-INFO | /tests/Harness_sanity/test_sanityPluginUtils.html | Profile::PluginUtilsLoadTime: 19 50 INFO TEST-KNOWN-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Need a test plugin on this platform 51 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Test plugin should be found 52 INFO TEST-KNOWN-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Need a test plugin on this platform 53 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Test plugin should be found 54 INFO TEST-KNOWN-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Need a test plugin on this platform 55 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Test plugin should be found 56 INFO TEST-KNOWN-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Need a test plugin on this platform 57 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Test plugin should be found 58 INFO TEST-KNOWN-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Need a test plugin on this platform 59 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Test plugin should be found 60 INFO TEST-KNOWN-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Need a test plugin on this platform 61 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Test plugin should be found 62 INFO TEST-KNOWN-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Need a test plugin on this platform 63 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Test plugin should be found 64 INFO TEST-KNOWN-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Need a test plugin on this platform 65 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Test plugin should be found 66 INFO TEST-KNOWN-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Need a test plugin on this platform 67 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Test plugin should be found 68 INFO TEST-KNOWN-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Need a test plugin on this platform [... continues for ~200 lines ...] 247 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Test plugin should be found 248 INFO TEST-KNOWN-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Need a test plugin on this platform 249 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityPluginUtils.html | Test plugin should be found 250 INFO TEST-INFO | /tests/Harness_sanity/test_sanityPluginUtils.html | Profile::PluginUtilsRunTime: 74 251 INFO TEST-END | /tests/Harness_sanity/test_sanityPluginUtils.html | finished in 220ms 252 INFO TEST-START | /tests/Harness_sanity/test_sanitySimpletest.html 253 INFO TEST-INFO | /tests/Harness_sanity/test_sanitySimpletest.html | Profile::SimpleTestLoadTime: 29 254 ERROR TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanitySimpletest.html | Test timed out. 255 INFO TEST-END | /tests/Harness_sanity/test_sanitySimpletest.html | finished in 329535ms
This is looking perma-orange on mozilla-inbound. The first occurence of this that I can find is here: https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=ab0a6fdf0144 https://tbpl.mozilla.org/php/getParsedLog.php?id=6454938&tree=Mozilla-Inbound though the tests were not really running before that for other reasons, so it's hard to know exactly when it started.
Summary: Intermittent Android TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityEventUtils.html | Test timed out. (followed by >100 more failures) → Perma-orange Android TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityEventUtils.html | Test timed out. (followed by >100 more failures)
Here's a regression range between the last green changeset and first instance of this bug on mozilla-inbound: https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=9ae43404500d&tochange=ab0a6fdf0144 It might be possible to narrow the range by looking at results from mozilla-central, and/or retriggering the tests a bunch of times.
Oh, it looks like these tests are new from bug 677626 and have never been green on Android. We should disable them on Android and then fix them.
Blocks: 677626
Attached patch Disable test (obsolete) — — Splinter Review
Requesting review since I didn't know whether your IRC comment counted as rs+, plus not quite sure if commenting out in the Makefile is what you wanted or else using todo_is()? Also not sure if test_sanityPluginUtils.html needed disabling at the same time, given comment 0.
Comment on attachment 561369 [details] [diff] [review] Disable test We should disable this on Android only (#ifndef ANDROID). I'm also not sure exactly which tests we need to disable; we could push some different options to Try to figure it out. I'm going to sleep soon, but I'll try this in the morning if no one else gets to it first.
Attachment #561369 - Flags: review-
(In reply to Matt Brubeck (:mbrubeck) from comment #7) > We should disable this on Android only (#ifndef ANDROID). I'm also not sure > exactly which tests we need to disable; we could push some different options > to Try to figure it out. I'm going to sleep soon, but I'll try this in the > morning if no one else gets to it first. Whoops that was a bit retarded on my part; was somewhat hurrying to get to sleep :-)
This time with |ifneq ($(OS_TARGET),Android)|, sent to try to see what else needs disabling: https://tbpl.mozilla.org/?tree=Try&usebuildbot=1&rev=6e1ff2427a04
(In reply to Ed Morley [:edmorley] from comment #6) > Created attachment 561369 [details] [diff] [review] > Disable test > > Requesting review since I didn't know whether your IRC comment counted as > rs+, plus not quite sure if commenting out in the Makefile is what you > wanted or else using todo_is()? Also not sure if test_sanityPluginUtils.html > needed disabling at the same time, given comment 0. This is the right thing to do, but as Matt mentioned in comment 7 these tests should only be disabled for Android, so you can #ifdef them in the makefile. Are you planning to get to this today? Let me know if you need help, happy to jump in. We can't let Android tests sit perma-orange for a long time.
(In reply to Clint Talbert ( :ctalbert ) from comment #16) > This is the right thing to do, but as Matt mentioned in comment 7 these > tests should only be disabled for Android, so you can #ifdef them in the > makefile. Are you planning to get to this today? Let me know if you need > help, happy to jump in. We can't let Android tests sit perma-orange for a > long time. I pushed a try run with updated patch (see comment 12, somewhat lost in the TBPL robot replies), to work out which tests needed disabling in addition to test_sanityEventUtils.html, but results haven't come back yet (presume due to tree closure). Feel free to take over if you have a better idea of which to disable, I only started doing it after mbrubeck asked if I wanted to last night - but in retrospect maybe I should have declined, since I haven't touched Mochitests before, so I'm not going to be the quickest person to do this :-)
That sounds good. Let us know what the try server comes back with. The code changes are simple, we can help walk you through that.
Next in the pending queue (with a wait time of 9:25:28 woot!) https://tbpl.mozilla.org/?tree=Try&usebuildbot=1&rev=6e1ff2427a04
Thanks for your help, Ed! Looking at the errors and the code, we'll need to kick out test_sanitySimpletest.html and test_sanityPluginUtils.html from Android tests as well. test_sanitySimpletest.html and test_sanityEventUtils.html both make calls to netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect"), instead of using SpecialPowers, which ends up breaking in Android. test_sanityPluginUtils.html won't work in Android since plugins aren't supported. I just kicked off a try run with these tests removed for Android: https://tbpl.mozilla.org/?tree=Try&usebuildbot=1&rev=2a76d3dffe11 I'll try to migrate the calls to SpecialPowers as soon as possible so we can re-enable the non-plugin tests.
Comment on attachment 561608 [details] [diff] [review] Disable 3 broken Android tests. (In reply to Malini Das from comment #23) > I just kicked off a try run with these tests removed for Android: > > https://tbpl.mozilla.org/?tree=Try&usebuildbot=1&rev=2a76d3dffe11 Passed try. M7 isn't showing on TBPL for some reason, but you can see here: https://build.mozilla.org/buildapi/self-serve/try/rev/2a76d3dffe11
Attachment #561608 - Flags: review?(mbrubeck)
Attachment #561608 - Flags: review?(mbrubeck) → review+
Unfortunately, I don't have permissions to submit patches to mozilla-central/mozilla-inbound. Is it possible for one of you to land this patch?
Attachment #561369 - Attachment is obsolete: true
Comment on attachment 561608 [details] [diff] [review] Disable 3 broken Android tests. Thanks, pushed to inbound: https://hg.mozilla.org/integration/mozilla-inbound/rev/376187bc07dc
Whiteboard: [orange][mobile_unittests] → [orange][mobile_unittests] [on merging, leave open for remaining work]
Whiteboard: [orange][mobile_unittests] [on merging, leave open for remaining work] → [orange][mobile_unittests][test disabled on Android]
Whiteboard: [orange][mobile_unittests][test disabled on Android] → [mobile_unittests][test disabled on Android]
OS: Android → All
Hardware: ARM → All
Summary: Perma-orange Android TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityEventUtils.html | Test timed out. (followed by >100 more failures) → Intermittent TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityEventUtils.html | Test timed out. (followed by >100 more failures)
Since this bug was very much about these tests being permaorange on Android when they were introduced and disabling them on Android at that time, in 2011, if someone has introduced a new Mac regression in them, please get a new bug for it.
OS: All → Android
Hardware: All → ARM
Summary: Intermittent TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityEventUtils.html | Test timed out. (followed by >100 more failures) → Perma-orange Android TEST-UNEXPECTED-FAIL | /tests/Harness_sanity/test_sanityEventUtils.html | Test timed out. (followed by >100 more failures)
Severity: normal → S3
Status: NEW → RESOLVED
Closed: 2 years ago
Duplicate of bug: 1704064
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: