832180 - Test failure "Unexpectedly found element ID: footer-right" in testToolbar/testStopReloadButtons.js

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect, P2)

Description

[Andreea Matei [:AndreeaMatei]]

This is occurring on all Windows for aurora branch, fr and en-US locales.

In the test, this ID is checked twice, first it should not exist, then it should.

I believe we fail here:

  // Even an element at the top of a page shouldn't exist when we hit the stop
  // button extremely fast
  var footer = new elementslib.ID(controller.tabs.activeTab, "footer-right");
  controller.assertNodeNotExist(footer);

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Somehow regressed by yesterdays checkins? I think about bug 793092. Can you reproduce the issue locally?

Comment 2

[Daniela Petrovici]

I cannot reproduce this locally: 
http://mozmill-crowd.blargon7.com/#/functional/report/9e41582ed5e806fa373351d9d33de88c

Comment 3

[Daniela Petrovici]

The test does not reproduce locally by running it alone in a loop 100 times.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I would assume that this is a somewhat fallout from bug 809431 but I cannot explain why. We don't see that for Nightly where it also has been landed. We should get this reproduced on one of the affected machines. I will try to do it later today. If I can reproduce it will be easy enough to figure out if the before mentioned bug is related or not.

Comment 5

[Andreea Matei [:AndreeaMatei]]

We have this on Nightly as well under this error:

http://mozmill-ci.blargon7.com/#/functional/report/5dfc8c8f9af0c35c0952dab5302dcf42
http://mozmill-ci.blargon7.com/#/functional/report/5dfc8c8f9af0c35c0952dab5302daf6e

Happened over the weekend.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This is very intermittent. I just tried to reproduce it on the machines we have seen this failure but I wasn't successful. Everything passed. I wonder if that's somewhat related to the network. We should wait a bit and get a clear picture on which factors it depends on.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We got a massive amount of failures today across various branches but only on Windows. Any idea why it doesn't happen Linux and OS X?

Comment 8

[mario garbi]

Happened again today on Windows NT 6.0.6002 Service Pack 2 (x86) - Firefox 21.0a1 :
http://mozmill-ci.blargon7.com/#/functional/report/1641bc2f6438595b86015bd90153549a

Comment 9

[Andreea Matei [:AndreeaMatei]]

I reproduced on win-xp-32-3 and it seems the click over stop works because stopButton.getNode().disabled is true after click as it should (the reload button appears).
The only thing left is that the page loads quickly, despite our listener. I'm searching for another event/listener to add and fix this.

Comment 10

[Andreea Matei [:AndreeaMatei]]

Attachment: patch v1

Tested this on mm-win-xp-32-3 and on one heavy loaded OS X to check it's not affecting those slow machines.

Checking more often will prevent from having the test page load until the next check. Also we don't need 5s timeout, I tested with just 1s.

See reports here, those with 7 tests:
http://mozmill-crowd.blargon7.com/#/functional/reports?branch=21.0&platform=Mac&from=2013-02-12&to=2013-02-12

And from the win XP, since 13:39 PM:
http://mozmill-crowd.blargon7.com/#/functional/reports?branch=21.0&platform=Windows NT&from=2013-02-12&to=2013-02-12

Comment 11

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 712918 [details] [diff] [review]
patch v1

Review of attachment 712918 [details] [diff] [review]:
-----------------------------------------------------------------

So the page is loading too fast? This would be a great candidate for a screenshot on failure being useful! The change looks good to me, and loading pages 'too quickly' is a good problem to have! :)

Landed as:
http://hg.mozilla.org/qa/mozmill-tests/rev/a986e21aad97 (default)

Comment 12

[Andreea Matei [:AndreeaMatei]]

Heh, yeah :)
I'll wait for results before checking backporting, cause this failure is on all branches.

Comment 13

[u279076]

Failure happened today during 19.0b6 testing:
> Vista: http://mozmill-ci.blargon7.com/#/functional/report/a83c700664548dba07298b74bf1025b6
> WinXP: http://mozmill-ci.blargon7.com/#/functional/report/a83c700664548dba07298b74bf101561

Comment 14

[Andreea Matei [:AndreeaMatei]]

No more failures on Nightly, patch applies cleanly to all other branches.

Comment 15

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Transplanted as:
http://hg.mozilla.org/qa/mozmill-tests/rev/20b99eff72cf (mozilla-aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/096e2b3d0958 (mozilla-beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/47ef70c6a596 (mozilla-release)
http://hg.mozilla.org/qa/mozmill-tests/rev/c10644d186bf (mozilla-esr17)

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Great investigation Andreea. This fix is totally understandable. Thanks!

Comment 17

[Daniela Petrovici]

Happened again on Windows 8 x64 with Nightly:
http://mozmill-ci.blargon7.com/#/functional/report/90a0b225333a4e0c868fcb2b440a4125

Comment 18

[Daniela Petrovici]

Happened again on Windows 8 with Nightly FR in 5/25:
http://mozmill-ci.blargon7.com/#/functional/report/90a0b225333a4e0c868fcb2b44ae8b60

Comment 19

[Daniela Petrovici]

Happened again on Windows 8 x86 with Nightly FR:
http://mozmill-ci.blargon7.com/#/functional/report/8aa237803fd4b0e7348adc8bbedb6677

Comment 20

[Andrei Eftimie]

This hasn't failed since June

Comment 21

[Andreea Matei [:AndreeaMatei]]

Not sure if WFM is the appropriate state here since this had actually a fix. After 3 months since the fix we had again some failure with this test, which should actually have been a different bug since we don't reopen fixed bugs after a while.