550843 - MozMill Composition tests randomly failing and timing out.

Status: RESOLVED WORKSFORME

(Thunderbird :: Message Compose Window, defect)

Description

[Mark Banner (:standard8)]

When bug 518215 landed a set of composition tests we got various random failures. I've therefore disabled the composition tests for the time being:

http://hg.mozilla.org/comm-central/rev/006e4b91354c

(note I only removed "composition" from the mozmilltests.list file, so they can still be run/patched using make mozmill-one.

On ThunderbirdTest these were seen on all platforms, obviously production boxes only have Windows enabled at the moment (they were less persistant than Mac/Linux).

I'm wondering if some of this is related to the issues with notification bars and things that we saw in bug 526733 - asuth?



Here's a few of them:

http://tinderbox.mozilla.org/showlog.cgi?tree=Thunderbird&errorparser=unittest&logfile=1268031520.1268044849.3559.gz&buildtime=1268031520&buildname=WINNT%205.2%20comm-central%20check&fulltext=1#err0

INFO | (runtestlist.py) | Running directory: composition
['python', 'runtest.py', '-t', 'e:/buildbot/win32-comm-central-check/build/mail/test/mozmill\\composition', '--binary', '../../.././mozilla/dist/bin/thunderbird.exe', '--symbols-path', 'e:/buildbot/win32-comm-central-check/build/objdir/mozilla/dist/crashreporter-symbols']
mozmill.startRunner :: True
mozmill.setState :: 'currentState'
mozmill.setModule :: 'currentModule'
mozmill.setState :: 'currentState'
mozmill.setTest :: {'name': 'setupModule', 'filename': 'e:\\buildbot\\win32-comm-central-check\\build\\mail\\test\\mozmill\\composition\\test-attachment-reminder.js'}
mozmill.setState :: 'currentState'
mozmill.setTest :: {'name': 'testAttachmentReminderPopsUpWhenItShould', 'filename': 'e:\\buildbot\\win32-comm-central-check\\build\\mail\\test\\mozmill\\composition\\test-attachment-reminder.js'}
Test Failure: {'exception': {'message': 'could not find element Elem instance.', 'lineNumber': 256, 'stack': 'Error("could not find element Elem instance.")@:0\n([object Object])@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/controller.js:256\ntestAttachmentReminderPopsUpWhenItShould()@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js -> file:///e:/buildbot/win32-comm-central-check/build/mail/test/mozmill/composition/test-attachment-reminder.js:91\n(testAttachmentReminderPopsUpWhenItShould)@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js:468\n([object Object])@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js:520\n([object Object])@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js:562\n("e:\\\\buildbot\\\\win32-comm-central-check\\\\build\\\\mail\\\\test\\\\mozmill\\\\composition")@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js:411\n("e:\\\\buildbot\\\\win32-comm-central-check\\\\build\\\\mail\\\\test\\\\mozmill\\\\composition")@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js:568\n((function (dir, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestDirectory(dir);runner.end();return true;}),[object Array])@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/server.js:164\n("b1351693-2a9e-11df-bf0a-00505600801d",(function (dir, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestDirectory(dir);runner.end();return true;}),[object Array])@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/server.js:168\n@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/server.js:244\n', 'fileName': 'file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/controller.js'}}
Test Failed : testAttachmentReminderPopsUpWhenItShould in e:\buildbot\win32-comm-central-check\build\mail\test\mozmill\composition\test-attachment-reminder.js
mozmill.setState :: 'currentState'
mozmill.setTest :: {'name': 'testAttachmentReminderDismissal', 'filename': 'e:\\buildbot\\win32-comm-central-check\\build\\mail\\test\\mozmill\\composition\\test-attachment-reminder.js'}
mozmill.startRunner :: True
mozmill.startRunner :: True
mozmill.setState :: 'currentState'
mozmill.setTest :: {'name': 'testAttachmentReminderAggressivePref', 'filename': 'e:\\buildbot\\win32-comm-central-check\\build\\mail\\test\\mozmill\\composition\\test-attachment-reminder.js'}
Test Failure: {'exception': {'message': 'No attachment notification shown with aggressive pref.', 'lineNumber': 145, 'stack': 'Error("No attachment notification shown with aggressive pref.")@:0\ntestAttachmentReminderAggressivePref()@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js -> file:///e:/buildbot/win32-comm-central-check/build/mail/test/mozmill/composition/test-attachment-reminder.js:145\n(testAttachmentReminderAggressivePref)@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js:468\n([object Object])@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js:520\n([object Object])@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js:562\n("e:\\\\buildbot\\\\win32-comm-central-check\\\\build\\\\mail\\\\test\\\\mozmill\\\\composition")@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js:411\n("e:\\\\buildbot\\\\win32-comm-central-check\\\\build\\\\mail\\\\test\\\\mozmill\\\\composition")@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js:568\n((function (dir, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestDirectory(dir);runner.end();return true;}),[object Array])@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/server.js:164\n("b1351693-2a9e-11df-bf0a-00505600801d",(function (dir, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestDirectory(dir);runner.end();return true;}),[object Array])@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/server.js:168\n@file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/server.js:244\n', 'fileName': 'file:///e:/buildbot/win32-comm-central-check/build/objdir/mozilla/_tests/mozmill/mozmillprofile/extensions/[e-mail]/resource/modules/frame.js -> file:///e:/buildbot/win32-comm-central-check/build/mail/test/mozmill/composition/test-attachment-reminder.js'}}
Test Failed : testAttachmentReminderAggressivePref in e:\buildbot\win32-comm-central-check\build\mail\test\mozmill\composition\test-attachment-reminder.js
mozmill.setState :: 'currentState'
mozmill.setTest :: {'name': 'testNoSendNowSends', 'filename': 'e:\\buildbot\\win32-comm-central-check\\build\\mail\\test\\mozmill\\composition\\test-attachment-reminder.js'}
mozmill.startRunner :: True
mozmill.startRunner :: True
mozmill.setState :: 'currentState'
mozmill.setTest :: {'name': 'teardownModule', 'filename': 'e:\\buildbot\\win32-comm-central-check\\build\\mail\\test\\mozmill\\composition\\test-attachment-reminder.js'}
mozmill.persist :: {}
Passed 4 :: Failed 2 :: Skipped 0
mozmill.endRunner :: True
TEST-PASS |  setupModule
NEXT ERROR TEST-UNEXPECTED-FAIL |  testAttachmentReminderPopsUpWhenItShould
  EXCEPTION: could not find element Elem instance.
    at: controller.js line 256
       Error("could not find element Elem instance.")  0
            controller.js 256
       testAttachmentReminderPopsUpWhenItShould() test-attachment-reminder.js 91
            frame.js 468
            frame.js 520
            frame.js 562
            frame.js 411
            frame.js 568
            server.js 164
            server.js 168
TEST-PASS |  testAttachmentReminderDismissal
TEST-UNEXPECTED-FAIL |  testAttachmentReminderAggressivePref
  EXCEPTION: No attachment notification shown with aggressive pref.
    at: test-attachment-reminder.js line 145
       Error("No attachment notification shown with aggressive pref.")  0
       testAttachmentReminderAggressivePref() test-attachment-reminder.js 145
            frame.js 468
            frame.js 520
            frame.js 562
            frame.js 411
            frame.js 568
            server.js 164
            server.js 168
TEST-PASS |  testNoSendNowSends
TEST-PASS |  teardownModule
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
INFO | (runtestlist.py) | composition: 4 passed, 2 failed

http://tinderbox.mozilla.org/showlog.cgi?log=Thunderbird/1268017754.1268021642.18361.gz

INFO | (runtestlist.py) | Running directory: composition
['python', 'runtest.py', '-t', 'e:/buildbot/win32-comm-central-check/build/mail/test/mozmill\\composition', '--binary', '../../.././mozilla/dist/bin/thunderbird.exe', '--symbols-path', 'e:/buildbot/win32-comm-central-check/build/objdir/mozilla/dist/crashreporter-symbols']
mozmill.startRunner :: True
mozmill.setState :: 'currentState'
mozmill.setModule :: 'currentModule'
mozmill.setState :: 'currentState'
mozmill.setTest :: {'name': 'setupModule', 'filename': 'e:\\buildbot\\win32-comm-central-check\\build\\mail\\test\\mozmill\\composition\\test-attachment-reminder.js'}
mozmill.setState :: 'currentState'
mozmill.setTest :: {'name': 'testAttachmentReminderPopsUpWhenItShould', 'filename': 'e:\\buildbot\\win32-comm-central-check\\build\\mail\\test\\mozmill\\composition\\test-attachment-reminder.js'}
mozmill.startRunner :: True
mozmill.setState :: 'currentState'
mozmill.setTest :: {'name': 'testAttachmentReminderDismissal', 'filename': 'e:\\buildbot\\win32-comm-central-check\\build\\mail\\test\\mozmill\\composition\\test-attachment-reminder.js'}
mozmill.startRunner :: True
mozmill.startRunner :: True
mozmill.setState :: 'currentState'
mozmill.setTest :: {'name': 'testAttachmentReminderAggressivePref', 'filename': 'e:\\buildbot\\win32-comm-central-check\\build\\mail\\test\\mozmill\\composition\\test-attachment-reminder.js'}
mozmill.startRunner :: True
mozmill.setState :: 'currentState'
mozmill.setTest :: {'name': 'testNoSendNowSends', 'filename': 'e:\\buildbot\\win32-comm-central-check\\build\\mail\\test\\mozmill\\composition\\test-attachment-reminder.js'}
mozmill.startRunner :: True

command timed out: 300 seconds without output
SIGKILL failed to kill process

http://tinderbox.mozilla.org/showlog.cgi?log=ThunderbirdTest/1268042723.1268044495.2672.gz#err0

TEST-PASS |  setupModule
TEST-PASS |  testAttachmentReminderPopsUpWhenItShould
NEXT ERROR TEST-UNEXPECTED-FAIL |  testAttachmentReminderDismissal
  EXCEPTION: Not a send error dialog; title=
    at: test-attachment-reminder.js line 195
       Error("Not a send error dialog; title=")  0
       clickOkOnSendError([object Object]) test-attachment-reminder.js 195
            frame.js 463
       WindowWatcher_notify([object XPCWrappedNative_NoHelper]) test-window-helpers.js 261
NEXT ERROR TEST-UNEXPECTED-FAIL |  testAttachmentReminderAggressivePref
  EXCEPTION: Not a send error dialog; title=
    at: test-attachment-reminder.js line 195
       Error("Not a send error dialog; title=")  0
       clickOkOnSendError([object Object]) test-attachment-reminder.js 195
            frame.js 463
       WindowWatcher_notify([object XPCWrappedNative_NoHelper]) test-window-helpers.js 261
TEST-UNEXPECTED-FAIL |  testNoSendNowSends
  EXCEPTION: Not a send error dialog; title=
    at: test-attachment-reminder.js line 195
       Error("Not a send error dialog; title=")  0
       clickOkOnSendError([object Object]) test-attachment-reminder.js 195
            frame.js 463
       WindowWatcher_notify([object XPCWrappedNative_NoHelper]) test-window-helpers.js 261
TEST-PASS |  teardownModule
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run

Comment 1

[Mark Banner (:standard8)]

These got re-enabled at some stage.