1976110 - browser_multiselect_tabs_using_keyboard.js sets the sidebar.verticalTabs preference back to false in the middle of the M-vt test suite

Status: NEW

(Firefox :: Sidebar, task, P1)

Description

[Florian Quèze [:florian]]

sidebar.verticalTabs is set to false during browser/components/tabbrowser/test/browser/tabs/browser_multiselect_tabs_using_keyboard.js with this stack:

promise callback []
AsyncFunctionNext [self-hosted:800:27]
InterpretGeneratorResume [self-hosted:1332:34]
nextTest [chrome://mochikit/content/browser-test.js:775:17]
(root scope) chrome://mochitests/content/browser/browser/components/tabbrowser/test/browser/tabs/browser_multiselect_tabs_using_keyboard.js [chrome://mochitests/content/browser/browser/components/tabbrowser/test/browser/tabs/browser_multiselect_tabs_using_keyboard.js:28:27]
reset [resource:///modules/CustomizableUI.sys.mjs:6397:8]
reset [resource:///modules/CustomizableUI.sys.mjs:4436:8]
nsIPrefBranch.setBoolPref []

It's done by the CustomizableUI.reset(); call at https://searchfox.org/mozilla-central/rev/584b344830aa2558985675b99aaadbb915402caa/browser/components/tabbrowser/test/browser/tabs/browser_multiselect_tabs_using_keyboard.js#29

Profile showing the behavior: https://share.firefox.dev/408L1sC

If the sidebar.verticalTabs pref is reset to true automatically at the end of the test by the patch stack in bug 1975980, there are a few tests that perma fail. Example of failure: https://treeherder.mozilla.org/jobs?repo=try&selectedTaskRun=XDW2CFtqSj2AbuDEbgswjA.0&revision=e0f0525226885f39abaf9aaec7174833fd816920

Comment 1

[Florian Quèze [:florian]]

In https://phabricator.services.mozilla.com/D256365 I'm annotating the 3 tests that are perma fail when actually running with vertical tabs. Interestingly, browser_multiselect_tabs_using_keyboard.js (which sets the tabs back to horizontal) is run-if = ["os != 'mac'"], and the 3 perma fail tests were all already annotated with skip-if "os == 'mac' && vertical_tab".

This indicates that these 3 tests were perma-fail on vertical tabs for a while, but also that the rest of the vertical-tab test suite ran on Mac, so we did have some test coverage on at least one platform, which is a bit reassuring.

Comment 2

[Sam Foster [:sfoster] (he/him)]

Marking this as P1 as we really want these tests to run and pass to give us the confidence needed as we scale out vertical tabs.

Comment 3

[Florian Quèze [:florian]]

(In reply to Florian Quèze [:florian] from comment #1)

In https://phabricator.services.mozilla.com/D256365 I'm annotating the 3 tests that are perma fail when actually running with vertical tabs. Interestingly, browser_multiselect_tabs_using_keyboard.js (which sets the tabs back to horizontal) is run-if = ["os != 'mac'"], and the 3 perma fail tests were all already annotated with skip-if "os == 'mac' && vertical_tab".

This indicates that these 3 tests were perma-fail on vertical tabs

There's actually a forth test that fails on vertical tab: browser_window_open_modifiers.js. It was hidden before because browser_tab_tooltips.js fails with a timeout that caused a browser shutdown.