Last Comment Bug 731158 - Mozmill test failure /testPreferences/testPreferredLanguage.js | could not find element Link: Gruppi
: Mozmill test failure /testPreferences/testPreferredLanguage.js | could not fi...
Status: RESOLVED FIXED
[mozmill-test-failure][qa-]
:
Product: Mozilla QA
Classification: Other
Component: Mozmill Tests (show other bugs)
: unspecified
: All All
: -- normal (vote)
: ---
Assigned To: Maniac Vlad Florin (:vladmaniac)
:
Mentors:
: 731690 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-02-28 02:43 PST by Maniac Vlad Florin (:vladmaniac)
Modified: 2012-08-14 14:55 PDT (History)
4 users (show)
vlad.mozbugs: in‑litmus+
See Also:
QA Whiteboard:
Iteration: ---
Points: ---
fixed
fixed
fixed
fixed
fixed


Attachments
disable v1.0 [landed] (2.31 KB, patch)
2012-02-28 02:53 PST, Maniac Vlad Florin (:vladmaniac)
anthony.s.hughes: review+
Details | Diff | Review
disable v1.0 (esr) [landed] (2.26 KB, patch)
2012-03-05 02:01 PST, Maniac Vlad Florin (:vladmaniac)
anthony.s.hughes: review+
Details | Diff | Review
fix v1.0 [landed:default] (2.64 KB, patch)
2012-03-12 07:58 PDT, Maniac Vlad Florin (:vladmaniac)
anthony.s.hughes: review+
hskupin: feedback+
Details | Diff | Review
unskip patch (1.29 KB, patch)
2012-07-05 21:44 PDT, Henrik Skupin (:whimboo)
anthony.s.hughes: review+
Details | Diff | Review

Description Maniac Vlad Florin (:vladmaniac) 2012-02-28 02:43:28 PST
Build ID:
Build identifier: Mozilla/5.0 (X11; Linux i686; rv:13.0) Gecko/20120227 Firefox/13.0a1
---------------------------------------------------------------------------------------
Platform: All Firefox branches
---------------------------------------------------------------------------------------
Mozmill Version: Mozmill 1.5.9
---------------------------------------------------------------------------------------
Error:
([object Object])@resource://mozmill/modules/controller.js:877 ()@resource://mozmill/modules/frame.js -> file:///c:/users/mozilla/appdata/local/temp/tmp6vp5zh.mozmill-tests/tests/functional/testPreferences/testPreferredLanguage.js:79 ((function () {controller.open("about:blank");prefs.openPreferencesDialog(controller, prefDialogCallback);controller.open("http://www.google.com/");controller.waitForPageLoad();if (browserLocale == "it") {controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Zaloguj"));controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Grupy"));controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Szukanie zaawansowane"));} else {controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Accedi"));controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Gruppi"));controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Ricerca avanzata"));}}))@resource://mozmill/modules/frame.js:557 ([object Object])@resource://mozmill/modules/frame.js:626 ([object Object])@resource://mozmill/modules/frame.js:669 ("c:\\users\\mozilla\\appdata\\local\\temp\\tmp6vp5zh.mozmill-tests\\tests\\functional\\testPreferences\\testPreferredLanguage.js")@resource://mozmill/modules/frame.js:506 ("c:\\users\\mozilla\\appdata\\local\\temp\\tmp6vp5zh.mozmill-tests\\tests\\functional\\testPreferences\\testPreferredLanguage.js")@resource://mozmill/modules/frame.js:681 ((function (filename, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestFile(filename);runner.end();return true;}),[object Proxy])@resource://jsbridge/modules/server.js:179 ("5e20e624-615e-11e1-b7d8-005056f2858a",(function (filename, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestFile(filename);runner.end();return true;}),[object Proxy])@resource://jsbridge/modules/server.js:183 
---------------------------------------------------------------------------------------
First failure date: 2012-02-23
---------------------------------------------------------------------------------------
Results dashboard:
http://mozmill-release.blargon7.com/#/functional/failure?branch=13.0&platform=All&from=2012-02-23&to=2012-02-28&test=%2FtestPreferences%2FtestPreferredLanguage.js&func=testPreferredLanguage.js%3A%3AtestSetLanguages
Comment 1 Maniac Vlad Florin (:vladmaniac) 2012-02-28 02:53:19 PST
Created attachment 601227 [details] [diff] [review]
disable v1.0 [landed]

This test should use a local test page - its failing sometimes because of the google page. 

The failure can be manually reproduced but I cannot find a fix for it, test related. 

The test fails on all Firefox default and aurora branches since 2012/02/23, but it also can be reproduced on beta and release branch. 
I recommend disabling this test and decide whether we should have our own test page and this test refactored. 

Please debate your ideas on the bug
Comment 2 Henrik Skupin (:whimboo) 2012-02-28 03:29:45 PST
Are you sure the link element still exists? Has its name been changed, or replaced by another element?
Comment 3 Maniac Vlad Florin (:vladmaniac) 2012-02-28 03:55:55 PST
(In reply to Henrik Skupin (:whimboo) [away 02/17 - 02/26] from comment #2)
> Are you sure the link element still exists? Has its name been changed, or
> replaced by another element?

As I said, the failure is intermittent so the link exists sometimes, and sometimes it cannot be found.
Comment 4 Henrik Skupin (:whimboo) 2012-02-28 03:59:34 PST
Then we should use waitForElement before asserting that the element exists.
Comment 5 Maniac Vlad Florin (:vladmaniac) 2012-02-28 04:21:12 PST
(In reply to Henrik Skupin (:whimboo) [away 02/17 - 02/26] from comment #4)
> Then we should use waitForElement before asserting that the element exists.

It does not help at all. There will be a timeout failure in waiting for that specific element. 

http://pastebin.mozilla.org/1494559
Comment 6 Henrik Skupin (:whimboo) 2012-02-29 12:09:43 PST
*** Bug 731690 has been marked as a duplicate of this bug. ***
Comment 7 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-02-29 12:49:44 PST
Comment on attachment 601227 [details] [diff] [review]
disable v1.0 [landed]

Patch is fine however I am going to hold off check-in so we can use it as a training check-in for Vlad's commit access tomorrow.
Comment 8 Henrik Skupin (:whimboo) 2012-03-02 01:02:31 PST
Can we please get this checked-in? We don't really want to wait until someone has commit privs. We have to disable broken tests immediately. Thanks.
Comment 9 Maniac Vlad Florin (:vladmaniac) 2012-03-02 01:11:32 PST
(In reply to Henrik Skupin (:whimboo) from comment #8)
> Can we please get this checked-in? We don't really want to wait until
> someone has commit privs. We have to disable broken tests immediately.
> Thanks.

Anthony is not around at this time, so I guess no one besides you can check it in at this moment
Comment 10 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-02 07:59:49 PST
Comment on attachment 601227 [details] [diff] [review]
disable v1.0 [landed]

Agreed that we can't wait any longer for check-ins. Vlad, once you get your commit access squared away we will practice using whatever is in the current queue.

Patch has now been landed where it fails:
http://hg.mozilla.org/qa/mozmill-tests/rev/36e75bd8fbe9 (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/0210192ecf02 (mozilla-aurora)
Comment 11 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-03 07:30:00 PST
Test also fails for 10.0.3 ESR today:
http://mozmill-release.blargon7.com/#/functional/report/e438d6e3916b2b636037d77445184f64

Patch does not apply cleanly, please provide an ESR skip patch.
Comment 12 Maniac Vlad Florin (:vladmaniac) 2012-03-05 02:01:49 PST
Created attachment 602831 [details] [diff] [review]
disable v1.0 (esr) [landed]

Adding skip test patch on esr branch - I don't know why it did not apply cleanly, locally it did for me

This new patch is for mozilla-esr10 branch exclusively, as requested
Comment 13 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-05 14:59:25 PST
Comment on attachment 602831 [details] [diff] [review]
disable v1.0 (esr) [landed]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/236966250704 (mozilla-esr10)
Comment 14 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-06 11:02:03 PST
Comment on attachment 601227 [details] [diff] [review]
disable v1.0 [landed]

Disabled the test on remaining branches as this failed today with Beta. It's not disabled everywhere.

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/aca4c2481e87 (mozilla-beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/4262ee1a05c3 (mozilla-release)
Comment 15 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-06 12:15:04 PST
Comment on attachment 601227 [details] [diff] [review]
disable v1.0 [landed]

Also on mozilla-1.9.2...

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/272a63fb3dd4 (mozilla-1.9.2)
Comment 16 Maniac Vlad Florin (:vladmaniac) 2012-03-12 03:21:08 PDT
What can we do to fix this test judging by the fact that this is a Google issue? 
We'd want to fix it as soon as possible as part of our target to provide fixes for skipped tests.
Comment 17 Henrik Skupin (:whimboo) 2012-03-12 03:53:35 PDT
Some questions:
1. In such a case does Google load the Italian version of the page?
2. Are the other links we are referencing present?
Comment 18 Maniac Vlad Florin (:vladmaniac) 2012-03-12 05:02:03 PDT
(In reply to Henrik Skupin (:whimboo) from comment #17)
> Some questions:
> 1. In such a case does Google load the Italian version of the page?
Yes, the Italian version is loaded, but sometimes the page looks different, probably a bad redirect  
> 2. Are the other links we are referencing present?
The other links are available afaik 

Are you suggesting to just test for another link ?
Comment 19 Henrik Skupin (:whimboo) 2012-03-12 07:28:44 PDT
Yes, just use another link which will indeed be there all the time.
Comment 20 Maniac Vlad Florin (:vladmaniac) 2012-03-12 07:58:37 PDT
Created attachment 604937 [details] [diff] [review]
fix v1.0 [landed:default]

Proposed initial fix, which passes for all branches. 
Since google changed, we changed the test to search for a link which is always present
Comment 21 Henrik Skupin (:whimboo) 2012-03-12 08:11:02 PDT
Comment on attachment 604937 [details] [diff] [review]
fix v1.0 [landed:default]

Looks better.
Comment 22 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-12 11:16:05 PDT
Comment on attachment 604937 [details] [diff] [review]
fix v1.0 [landed:default]

Looks fine to me.
Comment 23 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-12 11:18:09 PDT
Comment on attachment 604937 [details] [diff] [review]
fix v1.0 [landed:default]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/af518eba0d7b (default)
Comment 24 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-12 11:18:42 PDT
If this passes on default, I'll go ahead and land on other branches.
Comment 25 Dave Hunt (:davehunt) 2012-07-05 15:24:34 PDT
It appears that this is still disabled on mozilla-aurora, mozilla-beta, mozilla-release, and mozilla-esr10
Comment 26 Henrik Skupin (:whimboo) 2012-07-05 21:38:18 PDT
(In reply to Dave Hunt (:davehunt) from comment #25)
> It appears that this is still disabled on mozilla-aurora, mozilla-beta,
> mozilla-release, and mozilla-esr10

How can this be? The fix landed in March this year and should have been settled down to probably release meanwhile. Anthony, haven't you merged those skip changes whenever we had a version bump? If yes, there should be more than this single one probably we should fix. Dave, have you seen something else?
Comment 27 Henrik Skupin (:whimboo) 2012-07-05 21:42:39 PDT
Not sure what actually happened here, but the code changes have been merged except that the skipped lines have been removed. We just have to unskip those tests on older branches and have to find out why that happened.
Comment 28 Henrik Skupin (:whimboo) 2012-07-05 21:44:41 PDT
Created attachment 639581 [details] [diff] [review]
unskip patch

Dave or Anthony, whoever comes first please review this patch.
Comment 29 Henrik Skupin (:whimboo) 2012-07-05 21:50:58 PDT
So the merge went wrong from default to mozilla-aurora and the skip lines have been re-added:

http://hg.mozilla.org/qa/mozmill-tests/rev/e070324ab322

Anthony, when you do merges and you are unclear please always consult the bug and appropriate dashboard results. Usually those unskip patches have to be forwarded to the next branch. The only exception are remote tests for the discovery pane so far.
Comment 30 Henrik Skupin (:whimboo) 2012-07-05 21:52:09 PDT
Transplanted original patch to ESR10:
http://hg.mozilla.org/qa/mozmill-tests/rev/299061be6cdf
Comment 31 Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-07-05 22:01:05 PDT
Comment on attachment 639581 [details] [diff] [review]
unskip patch

Patch looks fine to me.

Note that with skipped tests, I err on the side of caution. If a test is skipped in Firefox 15 Nightly, I assume it needs to continue to be disabled when it merges, unless the comment specifically indicates the test is fine for Aurora. It's far better, in my opinion, to have to re-enable unintentionally skipped tests then to have to disable unintentionally enabled tests which are failing.

Perhaps we need a more descriptive commenting scheme for skipped blocks so that it is more evident.
Comment 32 Henrik Skupin (:whimboo) 2012-07-05 22:07:10 PDT
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #31)
> Note that with skipped tests, I err on the side of caution. If a test is
> skipped in Firefox 15 Nightly, I assume it needs to continue to be disabled
> when it merges, unless the comment specifically indicates the test is fine
> for Aurora. It's far better, in my opinion, to have to re-enable

That's true but a different scenario. In this case the test wasn't marked as skip anymore. The final patch removed the skip lines. See your comment 23 with the changeset landed. 

Pushed to remaining branches:
http://hg.mozilla.org/qa/mozmill-tests/rev/0381a4a8ff08 (aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/9448e8b6ca1f (beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/e4d1ddec0276 (release)

Vlad, please check all the Litmus tests and re-enable those for mozmill if necessary.
Comment 33 Maniac Vlad Florin (:vladmaniac) 2012-07-06 01:05:00 PDT
There is no Litmus work to be done here as the test is no longer available. 

I did: 
1. Browsed 'manage testcases' section in Litmus and found no test under Options/Preferences to match this bug
2. Browsed our mozmill repo, took the litmus id from the map and searched - no results 

https://litmus.mozilla.org/single_result.cgi?id=8322 

In conclusion, dismissing the litmus flag
Comment 34 Maniac Vlad Florin (:vladmaniac) 2012-07-06 01:21:56 PDT
It seems that we have the test for 3.6 branch 

https://litmus.mozilla.org/show_test.cgi?id=8322
Comment 35 Maniac Vlad Florin (:vladmaniac) 2012-07-06 01:27:13 PDT
Following Litmus tests are not disabled 

aurora: https://litmus.mozilla.org/show_test.cgi?id=15786
release: https://litmus.mozilla.org/show_test.cgi?id=55779
beta: https://litmus.mozilla.org/show_test.cgi?id=64133

We need to re-enable for esr
Comment 36 Maniac Vlad Florin (:vladmaniac) 2012-07-06 01:30:48 PDT
Enabled for mozmill group and edited the testcase to point that 
https://litmus.mozilla.org/show_test.cgi?id=40884
Comment 37 Dave Hunt (:davehunt) 2012-07-06 02:17:01 PDT
(In reply to Henrik Skupin (:whimboo) from comment #26)
> (In reply to Dave Hunt (:davehunt) from comment #25)
> > It appears that this is still disabled on mozilla-aurora, mozilla-beta,
> > mozilla-release, and mozilla-esr10
> 
> How can this be? The fix landed in March this year and should have been
> settled down to probably release meanwhile. Anthony, haven't you merged
> those skip changes whenever we had a version bump? If yes, there should be
> more than this single one probably we should fix. Dave, have you seen
> something else?

This is the only one I've seen.

Note You need to log in before you can comment on or make changes to this bug.