Last Comment Bug 881361 - Permanent Orange: TEST-UNEXPECTED-FAIL | test-retest-config.js::test_re_test_config
: Permanent Orange: TEST-UNEXPECTED-FAIL | test-retest-config.js::test_re_test_...
Status: RESOLVED FIXED
: intermittent-failure
Product: Thunderbird
Classification: Client Software
Component: Account Manager (show other bugs)
: Trunk
: All All
: -- normal (vote)
: Thunderbird 25.0
Assigned To: Mark Banner (:standard8)
:
Mentors:
Depends on:
Blocks: 889000
  Show dependency treegraph
 
Reported: 2013-06-10 11:14 PDT by Mark Banner (:standard8)
Modified: 2013-07-02 02:52 PDT (History)
6 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
+
fixed


Attachments
WIP patch (3.55 KB, patch)
2013-06-11 12:21 PDT, :aceman
no flags Details | Diff | Review
patch v2 (3.60 KB, patch)
2013-06-22 10:00 PDT, :aceman
no flags Details | Diff | Review
The fix (789 bytes, patch)
2013-07-01 03:00 PDT, Mark Banner (:standard8)
acelists: review+
standard8: approval‑comm‑aurora+
Details | Diff | Review

Description Mark Banner (:standard8) 2013-06-10 11:14:59 PDT
Since some time ago, we have been getting this on trunk:

https://tbpl.mozilla.org/php/getParsedLog.php?id=23957929&tree=Thunderbird-Trunk

TEST-START | /Users/cltbld/talos-slave/test/build/mozmill/account/test-retest-config.js | test_re_test_config
Step Pass: {"function": "controller.click()"}
Test Failure: Timeout waiting for modal dialog to open.
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/account/test-retest-config.js | test-retest-config.js::test_re_test_config

I've not yet found a way to reproduce it - it works fine for me locally, even when I try a packaged tests debug build.
Comment 1 :aceman 2013-06-11 12:21:25 PDT
Created attachment 761101 [details] [diff] [review]
WIP patch

Can you please run this on the try servers?
Comment 2 :aceman 2013-06-11 12:34:14 PDT
Mozmill tests are enough.
Comment 3 Mark Banner (:standard8) 2013-06-11 12:56:43 PDT
Comment on attachment 761101 [details] [diff] [review]
WIP patch

https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=29103dea3376
Comment 4 :aceman 2013-06-12 11:15:44 PDT
For reference, the failure starts first at checkin https://tbpl.mozilla.org/?tree=Thunderbird-Trunk&rev=72bf76257a34 , together with other failures. Looks like something changed in mozilla-central. I fixed the other ones (customize dialog), as the problem was that it looks like it no longer is possible to wait_for_new_window() via a window element ID, but windowType must be used. Maybe something similar needs to be done here too.
Comment 5 Mark Banner (:standard8) 2013-06-12 14:22:13 PDT
That would give a regression range of approximately:

http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=c21ef3664c67&tochange=00b264c7cced

(the nightly builds tend to have the logs kept for longer).

Note there are a couple of checkins relating to modal window changes at the bottom of that block.

A bit strange about the window id, not sure why I wouldn't be able to reproduce that locally.
Comment 6 :aceman 2013-06-12 14:32:51 PDT
And the window is not modal on any platform so I changed to wait_for_new_window() in the patch but still no success.
Comment 7 Treeherder Robot 2013-06-13 03:48:04 PDT
mkmelin
https://tbpl.mozilla.org/php/getParsedLog.php?id=24062964&tree=Thunderbird-Trunk
TB Rev4 MacOSX Lion 10.7 comm-central debug test mozmill on 2013-06-12 09:35:53
revision: 188ea3049daf
slave: talos-r4-lion-071

TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/account/test-retest-config.js | test-retest-config.js::test_re_test_config
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_appears_properly
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_dismissal
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_aggressive_pref
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_no_send_now_sends
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/content-tabs/test-plugin-blocked.js | test-plugin-blocked.js::test_blocklisted_plugin_notification
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | Disconnect Error: Application unexpectedly closed
OSError: [Errno 2] No such file or directory
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/folder-widget/test-message-filters.js | test-message-filters.js::test_customize_toolbar_doesnt_double_get_mail_menu
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
OSError: [Errno 2] No such file or directory
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
Comment 8 Treeherder Robot 2013-06-13 03:48:52 PDT
mkmelin
https://tbpl.mozilla.org/php/getParsedLog.php?id=24062724&tree=Thunderbird-Trunk
TB Rev3 WINNT 6.1 comm-central opt test mozmill on 2013-06-12 09:47:44
revision: 188ea3049daf
slave: talos-r3-w7-103

TEST-UNEXPECTED-FAIL | C:\talos-slave\test\build\mozmill\account\test-retest-config.js | test-retest-config.js::test_re_test_config
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | C:\talos-slave\test\build\mozmill\composition\test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_appears_properly
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | C:\talos-slave\test\build\mozmill\content-tabs\test-plugin-blocked.js | test-plugin-blocked.js::test_blocklisted_plugin_notification
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | Disconnect Error: Application unexpectedly closed
WindowsError: [Error 2] The system cannot find the file specified
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
Comment 9 Mark Banner (:standard8) 2013-06-13 04:26:54 PDT
(please keep the intermittent-failure annotation, even for permanent oranges, as this is required so that tbpl can pick it up and offer it as an orange option).
Comment 10 Treeherder Robot 2013-06-13 11:23:32 PDT
mkmelin
https://tbpl.mozilla.org/php/getParsedLog.php?id=24103608&tree=Thunderbird-Trunk
TB Rev4 MacOSX Snow Leopard 10.6 comm-central opt test mozmill on 2013-06-13 05:41:44
revision: 47d08a243bde
slave: talos-r4-snow-057

TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/account/test-retest-config.js | test-retest-config.js::test_re_test_config
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_appears_properly
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_dismissal
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_aggressive_pref
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_no_send_now_sends
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/content-tabs/test-plugin-blocked.js | test-plugin-blocked.js::test_blocklisted_plugin_notification
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | Disconnect Error: Application unexpectedly closed
OSError: [Errno 2] No such file or directory
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/folder-widget/test-message-filters.js | test-message-filters.js::test_customize_toolbar_doesnt_double_get_mail_menu
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
Comment 11 Treeherder Robot 2013-06-13 11:24:00 PDT
mkmelin
https://tbpl.mozilla.org/php/getParsedLog.php?id=24102864&tree=Thunderbird-Trunk
TB Rev4 MacOSX Snow Leopard 10.6 comm-central debug test mozmill on 2013-06-13 04:49:55
revision: 47d08a243bde
slave: talos-r4-snow-080

TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/account/test-retest-config.js | test-retest-config.js::test_re_test_config
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_appears_properly
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_dismissal
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_aggressive_pref
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_no_send_now_sends
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/content-tabs/test-plugin-blocked.js | test-plugin-blocked.js::test_blocklisted_plugin_notification
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | Disconnect Error: Application unexpectedly closed
OSError: [Errno 2] No such file or directory
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/folder-widget/test-message-filters.js | test-message-filters.js::test_customize_toolbar_doesnt_double_get_mail_menu
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
Comment 12 Treeherder Robot 2013-06-13 11:24:10 PDT
mkmelin
https://tbpl.mozilla.org/php/getParsedLog.php?id=24103654&tree=Thunderbird-Trunk
TB Rev4 MacOSX Lion 10.7 comm-central opt test mozmill on 2013-06-13 05:41:58
revision: 47d08a243bde
slave: talos-r4-lion-002

TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/account/test-retest-config.js | test-retest-config.js::test_re_test_config
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_appears_properly
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_dismissal
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_aggressive_pref
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_no_send_now_sends
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/content-tabs/test-plugin-blocked.js | test-plugin-blocked.js::test_blocklisted_plugin_notification
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | Disconnect Error: Application unexpectedly closed
OSError: [Errno 2] No such file or directory
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/folder-widget/test-message-filters.js | test-message-filters.js::test_customize_toolbar_doesnt_double_get_mail_menu
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
Comment 13 Treeherder Robot 2013-06-13 11:24:14 PDT
mkmelin
https://tbpl.mozilla.org/php/getParsedLog.php?id=24103032&tree=Thunderbird-Trunk
TB Rev4 MacOSX Lion 10.7 comm-central debug test mozmill on 2013-06-13 04:49:44
revision: 47d08a243bde
slave: talos-r4-lion-077

TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/account/test-retest-config.js | test-retest-config.js::test_re_test_config
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_appears_properly
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_dismissal
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_aggressive_pref
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/composition/test-attachment-reminder.js | test-attachment-reminder.js::test_no_send_now_sends
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/content-tabs/test-plugin-blocked.js | test-plugin-blocked.js::test_blocklisted_plugin_notification
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | Disconnect Error: Application unexpectedly closed
OSError: [Errno 2] No such file or directory
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | /Users/cltbld/talos-slave/test/build/mozmill/folder-widget/test-message-filters.js | test-message-filters.js::test_customize_toolbar_doesnt_double_get_mail_menu
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
OSError: [Errno 2] No such file or directory
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
Comment 14 Treeherder Robot 2013-06-13 11:25:07 PDT
mkmelin
https://tbpl.mozilla.org/php/getParsedLog.php?id=24102934&tree=Thunderbird-Trunk
TB Rev3 WINNT 6.1 comm-central opt test mozmill on 2013-06-13 05:05:44
revision: 47d08a243bde
slave: talos-r3-w7-122

TEST-UNEXPECTED-FAIL | C:\talos-slave\test\build\mozmill\account\test-retest-config.js | test-retest-config.js::test_re_test_config
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | C:\talos-slave\test\build\mozmill\composition\test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_appears_properly
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | C:\talos-slave\test\build\mozmill\content-tabs\test-plugin-blocked.js | test-plugin-blocked.js::test_blocklisted_plugin_notification
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | Disconnect Error: Application unexpectedly closed
WindowsError: [Error 2] The system cannot find the file specified
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
Comment 15 Treeherder Robot 2013-06-13 11:25:34 PDT
mkmelin
https://tbpl.mozilla.org/php/getParsedLog.php?id=24102834&tree=Thunderbird-Trunk
TB Rev3 WINNT 5.1 comm-central opt test mozmill on 2013-06-13 05:05:06
revision: 47d08a243bde
slave: talos-r3-xp-088

TEST-UNEXPECTED-FAIL | C:\talos-slave\test\build\mozmill\account\test-retest-config.js | test-retest-config.js::test_re_test_config
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | C:\talos-slave\test\build\mozmill\composition\test-attachment-reminder.js | test-attachment-reminder.js::test_attachment_reminder_appears_properly
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | C:\talos-slave\test\build\mozmill\content-tabs\test-plugin-blocked.js | test-plugin-blocked.js::test_blocklisted_plugin_notification
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
TEST-UNEXPECTED-FAIL | Disconnect Error: Application unexpectedly closed
WindowsError: [Error 2] The system cannot find the file specified
TEST-UNEXPECTED-FAIL | (runtestlist.py) | Exited with code 1 during directory run
Comment 16 :aceman 2013-06-22 10:00:59 PDT
Created attachment 766320 [details] [diff] [review]
patch v2

This would need a try run again.
Comment 17 Josiah Bruner [:JosiahOne] (needinfo for responses) 2013-06-22 14:19:05 PDT
https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=6a41642b8fd9
Comment 18 Mark Banner (:standard8) 2013-07-01 02:29:47 PDT
Oh interesting, I've just been running with this patch. Standalone the test passes, but run in alongside the rest of them in the directory, it fails.
Comment 19 Mark Banner (:standard8) 2013-07-01 03:00:18 PDT
Created attachment 769620 [details] [diff] [review]
The fix

Ok, now I can reproduce it, that makes it easier.

What I found was that stopping test-mail-account-setup-wizard.js running 'fixed' the test.

In test-mail-account-setup-wizard.js it is opening up the account wizard as well. There's also some mc.sleep(0) statements in there.

I think what is happening is when the dialog closes, the stack isn't getting a chance to unwind before we try to open it again. Hence, this adds an additional sleep to test-retest-config.js which fixes the issue locally for me.

I've also pushed this to try server:

https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=39a46ba37b01

Aceman: I think this is simple enough that you can review it.
Comment 20 :aceman 2013-07-01 04:03:45 PDT
So do we leave the modal dialog waiting in (even thought it is not modal) and just add this sleep?
Comment 21 Mark Banner (:standard8) 2013-07-01 06:31:41 PDT
The sleep is enough to fix the tests (as demonstrated by try :-) ). We could fix it to be non-modal, but we should fix the other file at the same time, and I think that would probably be best done in a separate bug so that its clear what the fix for this bug is.
Comment 22 :aceman 2013-07-01 11:33:33 PDT
Comment on attachment 769620 [details] [diff] [review]
The fix

Review of attachment 769620 [details] [diff] [review]:
-----------------------------------------------------------------

I got the failure randomly on my machine but I didn't get it since applying the patch. So let's try to land this.
Comment 23 Mark Banner (:standard8) 2013-07-02 00:31:22 PDT
https://hg.mozilla.org/comm-central/rev/f878ff74a85a
Comment 24 Mark Banner (:standard8) 2013-07-02 02:47:21 PDT
Comment on attachment 769620 [details] [diff] [review]
The fix

[Triage Comment]
All passing on trunk now, so lets get this on aurora.
Comment 25 Mark Banner (:standard8) 2013-07-02 02:52:50 PDT
https://hg.mozilla.org/releases/comm-aurora/rev/a7b8008b37c7

Note You need to log in before you can comment on or make changes to this bug.