Closed Bug 870453 Opened 7 years ago Closed 2 years ago

[intermittent] Figure out Windows 7 on iX intermittent test failures

Categories

(Testing :: General, defect)

x86
Windows 7
defect
Not set

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: armenzg, Unassigned)

References

Details

Attachments

(1 file)

So far we have these which all seem intermittent:
https://tbpl.mozilla.org/?tree=Cedar&rev=df6b18c4a857&jobname=Windows%207%2032-bit&showall=1

1) mochitest-browser-chrome
Bug 820738 - Intermittent browser_newtab_block.js | grid status = 0,1,2,3,6,7,8,9, - Got 0,1,2,3,6,7,8,9,http://mochi.test:8888/browser/browser/base/content/test/test_wyciwyg_copying.html, expected 0,1,2,3,6,7,8,9, (and another)

2) reftests (INTERMITTENT FAILURE)
https://tbpl.mozilla.org/php/getParsedLog.php?id=22776627#error0
Two pixels on two corners of a rectangle have different shades of gray (see attachment)

3) xpcshells
https://tbpl.mozilla.org/php/getParsedLog.php?id=22778872&tree=Cedar&full=1#error0
09:23:03     INFO -  TEST-INFO | C:\slave\test\build\tests\xpcshell\tests\browser\components\migration\tests\unit\test_IE_bookmarks.js | running test ...
09:23:04  WARNING -  TEST-UNEXPECTED-FAIL | C:\slave\test\build\tests\xpcshell\tests\browser\components\migration\tests\unit\test_IE_bookmarks.js | test failed (with xpcshell return code: 0), see following log:
09:23:04     INFO -  >>>>>>>
09:23:04     INFO -  TEST-INFO | (xpcshell/head.js) | test MAIN run_test pending (1)
09:23:04     INFO -  TEST-INFO | (xpcshell/head.js) | test pending (2)
09:23:04     INFO -  TEST-PASS | C:/slave/test/build/tests/xpcshell/tests/browser/components/migration/tests/unit/test_IE_bookmarks.js | [run_test : 11] true == true
09:23:04     INFO -  TEST-PASS | C:/slave/test/build/tests/xpcshell/tests/browser/components/migration/tests/unit/test_IE_bookmarks.js | [run_test : 15] true == true
09:23:04     INFO -  TEST-INFO | (xpcshell/head.js) | test MAIN run_test finished (2)
09:23:04     INFO -  TEST-INFO | (xpcshell/head.js) | running event loop
09:23:04     INFO -  TEST-PASS | C:/slave/test/build/tests/xpcshell/tests/browser/components/migration/tests/unit/test_IE_bookmarks.js | [onItemAdded : 30] 0 != -1
09:23:04  WARNING -  TEST-UNEXPECTED-FAIL | C:/slave/test/build/tests/xpcshell/tests/browser/components/migration/tests/unit/test_IE_bookmarks.js | 1 == 0 - See following stack:
09:23:04     INFO -  JS frame :: C:\slave\test\build\tests\xpcshell\head.js :: do_throw :: line 473
09:23:04     INFO -  JS frame :: C:\slave\test\build\tests\xpcshell\head.js :: do_report_result :: line 575
09:23:04     INFO -  JS frame :: C:\slave\test\build\tests\xpcshell\head.js :: _do_check_eq :: line 585
09:23:04     INFO -  JS frame :: C:\slave\test\build\tests\xpcshell\head.js :: do_check_eq :: line 592
09:23:04     INFO -  JS frame :: C:/slave/test/build/tests/xpcshell/tests/browser/components/migration/tests/unit/test_IE_bookmarks.js :: onMigrationEnded :: line 49
09:23:04     INFO -  JS frame :: resource:///modules/MigrationUtils.jsm :: <TOP_LEVEL> :: line 213
09:23:04     INFO -  JS frame :: resource:///modules/MigrationUtils.jsm :: <TOP_LEVEL> :: line 235
09:23:04     INFO -  JS frame :: resource://app/components/IEProfileMigrator.js :: migrateBatched :: line 215
09:23:04     INFO -  JS frame :: resource://app/components/IEProfileMigrator.js :: B_migrate :: line 217
09:23:04     INFO -  JS frame :: resource:///modules/MigrationUtils.jsm :: <TOP_LEVEL> :: line 244
09:23:04     INFO -  native frame :: <unknown filename> :: <TOP_LEVEL> :: line 0
09:23:04     INFO -  TEST-INFO | (xpcshell/head.js) | exiting test
09:23:04     INFO -  <<<<<<<
It seems that all test failures are intermittent but to be honest too many happened in just one commit:

4) mochitest-browser-chrome
https://tbpl.mozilla.org/php/getParsedLog.php?id=22784647&tree=Cedar&full=1
11:13:50 WARNING - TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/base/content/test/browser_bookmark_titles.js | Found the expected bookmark
Depends on: 870485
Depends on: 870488
Depends on: 870490
Depends on: 820738
No longer depends on: 870485, 870488, 870490
We got a clean run:
https://tbpl.mozilla.org/?tree=Cedar&jobname=Windows%207%2032-bit%20cedar&rev=884de34c1481
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WORKSFORME
Depends on: 870485
It seems that they happen enough that RyanVM pinged me about it.
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Summary: Green out Windows 7 on iX test failures → [intermittent] Figure out Windows 7 on iX intermittent test failures
Win7 dromaeo_css is perma-fail on PGO builds. Filing now.
s/dromaeo_css/dromaeojs
Depends on: 872748
Depends on: 872753
Depends on: 872760
Depends on: 872788
Depends on: 872865
Depends on: 872996
I would like to know which of these bugs should really block us from switching over from rev3 minis to iX hardware and who can help us to close them.
Unless I have owners for intermittent failures, it would be hard for me not to go ahead and switch over.

* bug 820738 -  Intermittent browser_newtab_block.js | grid status = 0,1,2,3,6,7,8,9, - Got 0,1,2,3,6,7,8,9,http://mochi.test:8888/browser/browser/base/content/test/test_wyciwyg_copying.html, expected 0,1,2,3,6,7,8,9, (and another)
* bug 870485 - Intermittent reftest failure on Windows 7 on iX machines
* bug 872748 - Intermittent Win7 iX content/media/test/test_mozLoadFrom.html | Test timed out, sometimes followed by other failures
* bug 872753 -  dom/imptests/html/webgl/test_bufferSubData.html | bufferSubData: Failure | test_compressedTexImage2D.html | compressedTexImage2D: Failure | compressedTexSubImage2D: Failure | test_uniformMatrixNfv.html | Should not throw for N: Failure
* bug 872760 - Intermittent Win7 iX content/media/test/test_new_audio.html | uncaught exception - NS_ERROR_FAILURE: Failure at http://mochi.test:8888/tests/content/media/test/test_new_audio.html:34
* bug 872788 - Frequent Windows 7 iX PROCESS-CRASH | dromaeo_dom | application crashed [Unknown top frame][@ mozalloc_abort(char const * const)] followed by talosError: 'timeout exceeded'
* bug 872865 -  Intermittent test_acceleration.html | Acceleration enabled on Windows XP or newer - didn't expect 0, but got it | Direct2D enabled on Windows Vista or newer | DirectWrite enabled on Windows Vista or newer
* bug 872996 - Intermittent Win7 iX content/media/test/test_mozLoadFrom.html,test_reset_src.html | Exited with code -2147483645 during test run | application crashed [Unknown top frame]

From an email thread, it seems that dromaeojs perma fails on PGO:
https://tbpl.mozilla.org/?jobname=Windows%207%2032-bit%20mozilla-central%20pgo%20talos%20dromaeojs&showall=1

* bug 872788
Depends on: 859571
I want to disable jobs on Rev3 and only run them on the iX machines.

Which jobs should I keep running on the Rev3 machines?

Should I keep the following jobs running on both platforms?
* bug 859571 - bimodal issue --> dirtypaint, other
* bug 872788 - dromaeo_dom intermittently crashes --> dromaeojs

Or should we just live with them and hence making it more important for developers to fix it?

Do you guys have any suggestions on how to raise these two issues for developers to help?
Are you guys fine if I ask today if we can disable the rev3 jobs on Thursday morning?
I would keep dirtypaint and other running side-by-side if you guys believe it to be important until we can fix bug 859571.
Flags: needinfo?(ryanvm)
Flags: needinfo?(philringnalda)
Flags: needinfo?(emorley)
I'd say go for it now (given that worrying about bug 859571 first means presuming we pay attention to talos...) :-)
Flags: needinfo?(emorley)
Doesn't matter to me - I'm having to double-star since I star &showall=1, and I generally retrigger on any fatal talos errors, so I won't miss the r3s, and we aren't actually losing data (other than when I forget to retrigger) while I'm awake anyway.
Flags: needinfo?(philringnalda)
We still have a worrying number of iX-only failures, but they're starrable at least.
Flags: needinfo?(ryanvm)
No longer blocks: win7-ix-releng
Depends on: win7-ix-releng
Mass closing bugs with no activity in 2+ years. If this bug is important to you, please re-open.
Status: REOPENED → RESOLVED
Closed: 7 years ago2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.