Closed Bug 1414533 Opened 7 years ago Closed 7 years ago

Mozmill bustage on 2017-11-04: 5 plugin related tests and two other tests fail

Categories

(Thunderbird :: General, defect)

Product:
Thunderbird
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
Thunderbird 58.0

People

(Reporter: jorgk-bmo, Assigned: aceman)

Details

(Whiteboard: [Thunderbird-testfailure: Z all])

Attachments

(1 file)

1414533.patch
11.33 KB, patch
jorgk-bmo
: review+
Details | Diff | Splinter Review 
TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/mozmill/content-policy/test-plugins-policy.js | test-plugins-policy.js::setupModule 
TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/mozmill/content-tabs/test-plugin-outdated.js | test-plugin-outdated.js::setupModule 
TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/mozmill/content-tabs/test-plugin-crashing.js | test-plugin-crashing.js::setupModule 
TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/mozmill/content-tabs/test-plugin-blocked.js | test-plugin-blocked.js::test_blocklisted_plugin_notification 
TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/mozmill/override-main-menu-collapse/test-override-mainmenu-collapse.js | test-override-mainmenu-collapse.js::test_main_menu_not_collapsed 
TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/mozmill/pref-window/test-font-chooser.js | test-font-chooser.js::test_font_name_not_present 
TEST-UNEXPECTED-FAIL | /builds/slave/test/build/tests/mozmill/testTodayPane.js | testTodayPane.js::testTodayPane 

First seen Sat Nov 4, 2017, 11:41:10
https://treeherder.mozilla.org/#/jobs?repo=comm-central&revision=75d12f0aeb238a399919f1e3a911597b850934bd

Log
https://archive.mozilla.org/pub/thunderbird/tinderbox-builds/comm-central-linux64/1509792082/comm-central_ubuntu64_vm_test-mozmill-bm54-tests1-linux64-build478.txt.gz
says:
INFO -    EXCEPTION: plugin is null
for the failing plugin tests.

test-override-mainmenu-collapse.js::test_main_menu_not_collapsed says:
INFO -    EXCEPTION: The main menu should not have the autohide attribute.
INFO -      at: test-folder-display-helpers.js line 2924
INFO -         assert_false test-folder-display-helpers.js:2924 11
INFO -         test_main_menu_not_collapsed test-override-mainmenu-collapse.js:46 3

testTodayPane.js::testTodayPane says:
INFO -  SUMMARY-UNEXPECTED-FAIL | testTodayPane.js | testTodayPane.js::testTodayPane
INFO -    EXCEPTION: could not validate element ID: datevalue-label with value 4
INFO -      at: controller.js line 951
INFO -         MozMillController.prototype.assertValue controller.js:951 9
INFO -         testTodayPane testTodayPane.js:145 5
INFO -         Runner.prototype.wrapper frame.js:589 9

test-font-chooser.js::test_font_name_displayed says:
INFO -  SUMMARY-PASS | test-font-chooser.js::test_font_name_displayed
INFO -  SUMMARY-UNEXPECTED-FAIL | test-font-chooser.js | test-font-chooser.js::test_font_name_not_present
INFO -    EXCEPTION: font.name-list.serif.x-western should have exactly two fonts, but it is serif.
INFO -      at: test-font-chooser.js line 151
INFO -         test_font_name_not_present test-font-chooser.js:151 13

So this looks like a variety of issues.

M-C last good: 4e6df5159df3e64c3c453c0db0a2e1d126
M-C first bad: 14fd26761bc4d10c5334abe50d7b6f3a59

https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=4e6df5159df3e64c3c453c0db0a2e1d126&tochange=14fd26761bc4d10c5334abe50d7b6f3a59

Aceman, please take a look.
Flags: needinfo?(acelists)
Maybe not related to bug 1414513 as I first though.
Summary: Mozmill bustage on 2017-11-04: 7 plugin related tests fail → Mozmill bustage on 2017-11-04: 5 plugin related tests and two other tests fail
mozmake -C calendar/test/mozmill SOLO_TEST=testTodayPane.js mozmill-one
gives
### Error loading backend: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsIPrefBranch.getBoolPref]"  nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)"  location: "JS frame :: file:///C:/mozilla-source/comm-central/obj-i686-pc-mingw32/dist/bin/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/components/calBackendLoader.js :: loadBackend :: line 41"  data: no]

So that's bug 1414398.
(In reply to Jorg K (GMT+2) from comment #0)
> test-font-chooser.js::test_font_name_displayed says:
> INFO -  SUMMARY-PASS | test-font-chooser.js::test_font_name_displayed
> INFO -  SUMMARY-UNEXPECTED-FAIL | test-font-chooser.js |
> test-font-chooser.js::test_font_name_not_present
> INFO -    EXCEPTION: font.name-list.serif.x-western should have exactly two
> fonts, but it is serif.
> INFO -      at: test-font-chooser.js line 151
> INFO -         test_font_name_not_present test-font-chooser.js:151 13

I'd guess the file at https://dxr.mozilla.org/comm-central/source/mail/test/mozmill/pref-window/linux-prefs.js containing the default prefs is not loaded due to bug 1414398, see https://dxr.mozilla.org/comm-central/rev/952762964f408e33d060dd1110cac0626ae5fa51/mail/test/mozmill/pref-window/wrapper.py#21 .

> test-override-mainmenu-collapse.js::test_main_menu_not_collapsed says:
> INFO -    EXCEPTION: The main menu should not have the autohide attribute.
> INFO -      at: test-folder-display-helpers.js line 2924
> INFO -         assert_false test-folder-display-helpers.js:2924 11
> INFO -         test_main_menu_not_collapsed
> test-override-mainmenu-collapse.js:46 3

This could be the same, also uses the python script to set up default prefs.
Flags: needinfo?(acelists)
So summarising:
- test-font-chooser.js::test_font_name_displayed
- test-override-mainmenu-collapse.js::test_main_menu_not_collapsed
- testTodayPane.js::testTodayPane
are likely caused by bug 1414398, which leave the five plugin test failures.

I was going to add 
// Backend to use. false: libical, true: ical.js
#ifdef NIGHTLY_BUILD
pref("calendar.icaljs", true);
#else
pref("calendar.icaljs", false);
#endif
to all-thunderbird.js to see whether it clears the test failure.

Which other preferences do we need for the other two tests?
Flags: needinfo?(acelists)
It seems to me for test-font-chooser.js::test_font_name_displayed we do not really need to set the prefs as defaults before TB starts. I set them in setupModule and the test passes for me.
Flags: needinfo?(acelists)
Hmm, adding pref "calendar.icaljs" does not make the test pass.

How many bustages are we looking at here?
(In reply to :aceman from comment #5)
> It seems to me for test-font-chooser.js::test_font_name_displayed we do not
> really need to set the prefs as defaults before TB starts. I set them in
> setupModule and the test passes for me.
OK, any patch welcome that will reduce the amount of bustage we have here.
There is also bug 1414521 for calendar. This fixed one issue but the today pane is still broken.
I also got override-main-menu-collapse working by using a user.js file instead of the "undocumented feature" of placinf default prefs in profile/preferences.
Perfect. Patch(es) please.
It seems the content-tab tests also set up the initial prefs, so the cause may be the same. I'm now changing the python script in all test folders, and will push it to try.
Nice. So that will fix the plug-in issues as well.

Since Win and Mac are busted, it wasn't strictly necessary to try on those platforms ;-)
Looks like the failure of
  mozmake -C calendar/test/mozmill SOLO_TEST=testTodayPane.js mozmill-one
is caused by bug 1414521.

So I guess we land this patch Aceman has prepared and we're done here.
Attached patch 1414533.patch — — Splinter Review 
This implements what I describe in previous comments.
The try run seems successful.
Attachment #8925291 - Flags: review?(jorgk)
Comment on attachment 8925291 [details] [diff] [review]
1414533.patch

Review of attachment 8925291 [details] [diff] [review]:
-----------------------------------------------------------------

Hmm, we're becoming Python programmers :-) or should that be :-(

If you want to practise landing, please land this patch with the mac/windows-prefs.js files fixed as well.

Should we move test-override-mainmenu-collapse.js::test_main_menu_not_collapsed to another bug?

::: mail/test/mozmill/pref-window/linux-prefs.js
@@ +1,2 @@
>  // Default preferences used for test-font-chooser.js on Linux. The UUIDs here
> +// should be kept in sync with kFakeFonts in test-font-chooser.js.

Please fix mac-prefs.js and windows-prefs.js as well.

::: mail/test/mozmill/pref-window/test-font-chooser.js
@@ +20,5 @@
>  
>  var gFontEnumerator;
>  
>  // We'll test with Western. Unicode has issues on Windows (bug 550443).
> +const kLanguage = "x-western";

These clean-up changes seem unrelated, but anyway.
Attachment #8925291 - Flags: review?(jorgk) → review+
(In reply to Jorg K (GMT+2) from comment #16)
> Should we move
> test-override-mainmenu-collapse.js::test_main_menu_not_collapsed to another
> bug?

Why? The failure was listed in this bug and is fixed by the patch.
Did you mean test-content-tab.js | test-content-tab.js::test_spellcheck_in_content_tabs that is still left in the try run?
 
> ::: mail/test/mozmill/pref-window/linux-prefs.js
> @@ +1,2 @@
> >  // Default preferences used for test-font-chooser.js on Linux. The UUIDs here
> > +// should be kept in sync with kFakeFonts in test-font-chooser.js.
> 
> Please fix mac-prefs.js and windows-prefs.js as well.

Yes, thanks. The invalid variable reference made the hunting harder ;)
Looks like a different kind of beast:
TEST-START | mail/test/mozmill/content-tabs/test-content-tab.js | test_spellcheck_in_content_tabs
JavaScript error: resource://gre/modules/InlineSpellChecker.jsm, line 208: TypeError: spellchecker is null
JavaScript error: chrome://messenger/content/mailContextMenus.js, line 52: TypeError: gContextMenu is undefined
(In reply to :aceman from comment #17)
> (In reply to Jorg K (GMT+2) from comment #16)
> > Should we move
> > test-override-mainmenu-collapse.js::test_main_menu_not_collapsed to another
> > bug?
> Why? The failure was listed in this bug and is fixed by the patch.

19:45:13 - aceman: it didn't fix the test-message-header.js::test_toolbar_collapse_and_expand, but that is a different problem

Sorry, got confused, different "collapse" issue. My mistake.
Pushed by acelists@atlas.sk:
https://hg.mozilla.org/comm-central/rev/21ba31e50f44
use user.js way instead of default prefs in profile/preferences to set prefs before mozmill tests start. r=jorgk CLOSED TREE
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Thanks!
Assignee: nobody → acelists
Target Milestone: --- → Thunderbird 58.0
(In reply to :aceman from comment #18)
> Looks like a different kind of beast:
> TEST-START | mail/test/mozmill/content-tabs/test-content-tab.js |
> test_spellcheck_in_content_tabs
Filed bug 1414563 for that (in case anyone is watching).
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: