858203 - mark more reftests as needs-focus

Status: RESOLVED FIXED

(Core :: General, defect)

Comment 1

[Nathan Froyd [:froydnj]]

Attachment: mark more reftests as needs-focus

In service of bug 813742, this patch attempts to identify all the
reftests that need focus and aren't marked as such.  Testing
methodology:

1. Start headless X server.  Start basic window manager on it.
2. Run reftests with said X server.  Take note of failing tests.
3. Disable all reftests that need focus:

   find $topsrcdir -name '*.list' | xargs sed -i -e '/needs-focus/s/^/#/'

4. Make reftest.js:Focus never focus the window and always return true.
5. Start reftests with said X server again.  Once the tests begin
   running, start a xterm on the same X server.  The window manager will
   focus this xterm and will not return focus to the browser.
6. Compare now-failing tests with baseline.  Mark all such tests as
   needs-focus.

Comments on specific tests:

editor/reftests/
- spellcheck-input-ref.html and spellcheck-textarea-ref.html focus
  input elements and don't work right without the window having focus.
- spellcheck-hyphen-invalid-ref.html and
  spellcheck-hyphen-multiple-invalid-ref.html use @autofocus on
  textarea.
- 674212-spellcheck.html, 338427-2.html, and
  462758-grabbers-resizers.html all wait for onfocus for div elements.

layout/reftests/bugs/
- 652301-1b.html focuses a bogus element with an onfocus handler.

layout/reftests/selection/
- pseudo-element-of-native-anonymous-ref.html .select()s an <input>.

Comment 2

[Daniel Holbert [:dholbert]]

Comment on attachment 733498 [details] [diff] [review]
mark more reftests as needs-focus

Methodology seems sound.  I didn't look at all of the flagged /editor tests, but I buy that they need focus, given that they're using <input> elements and given your comments.

Comment 3

[Daniel Holbert [:dholbert]]

(Technically Testing|Reftest is for bugs in the reftest *harness*, not in tests/manifests.  Moving this to Core|General.)

Comment 4

[(not currently active) Ted Mielczarek]

It would be cool if we could add something to the reftest harness to explicitly *not* focus tests that aren't marked as needs-focus, to prevent these from creeping in.

Comment 5

[Nathan Froyd [:froydnj]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/282de3fcc0cd

Comment 6

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/282de3fcc0cd