1009726 - scrollbar tests in mochitest-chrome fail on OSX 10.9

Status: NEW

(Core :: Widget: Cocoa, defect)

Description

[Jonathan Griffin (:jgriffin)]

There are a couple of scrollbar-related tests in mochitest-chrome that fail when run on OSX 10.9 which may be related to different scrollbar behavior there.

3263 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/layout/xul/test/test_bug159346.xul | scrollbar didn't change curpos by mousedown #1
3264 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/layout/xul/test/test_bug159346.xul | scrollbar didn't change curpos by auto repeat #1
3268 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/layout/xul/test/test_bug159346.xul | scrollbar didn't change curpos by mousedown #2
3272 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/layout/xul/test/test_bug159346.xul | scrollbar didn't change curpos by mousemove after cursor is back on the scrollbar button #2
3273 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/layout/xul/test/test_bug159346.xul | scrollbar didn't change curpos by mousedown #3
3274 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/layout/xul/test/test_bug159346.xul | scrollbar didn't change curpos by auto repeat when cursor is outside of scrollbar button #3
3275 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/layout/xul/test/test_bug159346.xul | scrollbar didn't change curpos by mousemove after cursor is back on the scrollbar button #3
3314 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/layout/xul/test/test_bug703150.xul | scrollbar thumb hasn't been dragged
4042 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/toolkit/content/tests/chrome/test_scrollbar.xul | Dragging the scrollbar thumb should work.
4047 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/toolkit/content/tests/chrome/test_scrollbar.xul | Dragging the scrollbar thumb should work.
4052 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/toolkit/content/tests/chrome/test_scrollbar.xul | Dragging the scrollbar thumb should work.
4057 INFO TEST-UNEXPECTED-FAIL | chrome://mochitests/content/chrome/toolkit/content/tests/chrome/test_scrollbar.xul | Dragging the scrollbar thumb should work.

full log: https://tbpl.mozilla.org/php/getParsedLog.php?id=39527956&tree=Cedar

Comment 1

[Stephen A Pohl [:spohl]]

I can confirm that the failures in test_bug159346.xul don't occur if the system preferences are set to always display scrollbars. I'm a bit surprised that we weren't seeing these failures on 10.7 and 10.8 however...

Comment 2

[Stephen A Pohl [:spohl]]

This is failing on my 10.7 system as well. Now I'm curious, is this passing on 10.7 and 10.8, but failing on 10.9 on Cedar?

Comment 3

[Jonathan Griffin (:jgriffin)]

These tests do pass on cedar on 10.7 and 10.8, but fail on 10.9.

Comment 4

[Stephen A Pohl [:spohl]]

This makes it even stranger... Is there another difference between the 10.7/10.8 and 10.9 systems? For example, do the 10.7/10.8 systems have an external mouse attached? Or were the System Preferences manually changed to always display scrollbars?

Comment 5

[Jonathan Griffin (:jgriffin)]

I don't know; needinfo'ing kmoir who might be able to tell us.   Kim, see comment #4.

Comment 6

[Timothy Nikkel (:tnikkel)]

I think the 10.7 and 10.8 machines have the always display scroll bars pref set and 10.9 do not if my memory serves me correctly from last time I looked into it.

Comment 7

[Kim Moir [:kmoir] ET]

The default behaviour in 10.9 for scrollbars it to display automatically based on mac or trackpad. They are not always on.  You can change this preference to Always if you want to have them on all the time but the slaves you're testing don't have this change enabled by default.

Comment 8

[Stephen A Pohl [:spohl]]

Kim, the default behavior that you're describing for 10.9 is also the default behavior for 10.7 and 10.8. Still, there appears to be a difference in the way these systems are set up. Would you happen to know if 10.7 and 10.8 were manually set to always display scrollbars?

Timothy, when you say "display scroll bars pref", do you mean the pref in System Preferences? Or a pref in the Firefox profile?

Comment 9

[Kim Moir [:kmoir] ET]

No 10.7 and 10.8 were not manually set to always display scrollbars.

Comment 10

[Timothy Nikkel (:tnikkel)]

(In reply to Stephen Pohl [:spohl] from comment #8)
> Timothy, when you say "display scroll bars pref", do you mean the pref in
> System Preferences? Or a pref in the Firefox profile?

I meant the system pref. But what Kim says contradicts what I thought, so I don't know.

Comment 11

[Stephen A Pohl [:spohl]]

Kim, could you double-check the System Preferences on one of our 10.7 or 10.8 slaves? Or could you tell me how I can do that myself? Thanks!

Comment 12

[Kim Moir [:kmoir] ET]

I did look at at 10.8 slave this morning the the system preference was "display automatically based on mac or trackpad" Not always.

Comment 13

[Steven Michaud [:smichaud] (Retired)]

With the "automatically based on mouse or trackpad" setting, scrollbars will always be displayed if the machine thinks a mouse is attached.

Comment 14

[Stephen A Pohl [:spohl]]

kmoir and I debugged this and we've found that the 10.8 slaves display permanently visible scrollbars while the 10.9 slaves do not, even though there don't seem to be any differences in the hardware used. It turns out that OSX 10.9 most likely changed the way scrollbars are displayed on headless mac minis[1]. To work around this, we'll have to manually set the System Preferences to always display scrollbars on our 10.9 slaves.

[1] https://discussions.apple.com/thread/5467890

Comment 15

[Timothy Nikkel (:tnikkel)]

We want a configuration where we test overlay scrollbars. They are the default, so this is users see. We also have no test coverage of overlay scrollbars on desktop currently. Overlay scrollbars do (and have, don't have the bug numbers handy) cause regressions and problems so we need test coverage of them, we can't just keep disabling them for test. We need to fix the tests.

Comment 16

[Steven Michaud [:smichaud] (Retired)]

Testing with overlay scrollbars will always be tricky -- they fade in and out on timers.

So reftests, for example, generally won't work if overlay scrollbars are enabled.

Comment 17

[Stephen A Pohl [:spohl]]

Timothy, the idea here was to make the 10.9 test slaves match the 10.7 and 10.8 test slaves. Updating all the tests to work nicely with overlay scrollbars is the ultimate goal, but will require significantly more work. Are you saying that you would rather delay enabling 10.9 test slaves than have them match the 10.7 and 10.8 ones for now?

Comment 18

[Timothy Nikkel (:tnikkel)]

(In reply to Steven Michaud from comment #16)
> Testing with overlay scrollbars will always be tricky -- they fade in and
> out on timers.
> 
> So reftests, for example, generally won't work if overlay scrollbars are
> enabled.

We have the layout.testing.overlay-scrollbars.always-visible pref to disable the fading in-out which we currently use on reftests (ie on b2g where we do get overlay scrollbars).

(In reply to Stephen Pohl [:spohl] from comment #17)
> Timothy, the idea here was to make the 10.9 test slaves match the 10.7 and
> 10.8 test slaves. Updating all the tests to work nicely with overlay
> scrollbars is the ultimate goal, but will require significantly more work.
> Are you saying that you would rather delay enabling 10.9 test slaves than
> have them match the 10.7 and 10.8 ones for now?

No, I'm just saying that we really should be testing overlay scrollbars. Hopefully we can get that sometime soon.

Comment 19

[Stephen A Pohl [:spohl]]

(In reply to Timothy Nikkel (:tn) from comment #18)
> I'm just saying that we really should be testing overlay scrollbars.
> Hopefully we can get that sometime soon.

Understood. I reopened bug 1033650 and filed bug 1035274 to update the tests to make them aware of overlay scrollbars. Then, we should be able to change the default for our test slaves to overlay scrollbars and get better coverage that way. It should be possible to switch back to permanently visible scrollbars on a test-by-test basis when a test is specifically testing permanently visible scrollbars.