As a security precaution, we have turned on the setting "Require API key authentication for API requests" for everyone. If this has broken something, please contact bugzilla-admin@mozilla.org
Last Comment Bug 731158 - Mozmill test failure /testPreferences/testPreferredLanguage.js | could not find element Link: Gruppi
: Mozmill test failure /testPreferences/testPreferredLanguage.js | could not fi...
Status: RESOLVED FIXED
[mozmill-test-failure][qa-]
:
Product: Mozilla QA
Classification: Other
Component: Mozmill Tests (show other bugs)
: unspecified
: All All
: -- normal (vote)
: ---
Assigned To: Maniac Vlad Florin (:vladmaniac)
:
:
Mentors:
: 731690 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-02-28 02:43 PST by Maniac Vlad Florin (:vladmaniac)
Modified: 2012-08-14 14:55 PDT (History)
4 users (show)
vlad.mozbugs: in‑litmus+
See Also:
QA Whiteboard:
Iteration: ---
Points: ---
fixed
fixed
fixed
fixed
fixed


Attachments
disable v1.0 [landed] (2.31 KB, patch)
2012-02-28 02:53 PST, Maniac Vlad Florin (:vladmaniac)
anthony.s.hughes: review+
Details | Diff | Splinter Review
disable v1.0 (esr) [landed] (2.26 KB, patch)
2012-03-05 02:01 PST, Maniac Vlad Florin (:vladmaniac)
anthony.s.hughes: review+
Details | Diff | Splinter Review
fix v1.0 [landed:default] (2.64 KB, patch)
2012-03-12 07:58 PDT, Maniac Vlad Florin (:vladmaniac)
anthony.s.hughes: review+
hskupin: feedback+
Details | Diff | Splinter Review
unskip patch (1.29 KB, patch)
2012-07-05 21:44 PDT, Henrik Skupin (:whimboo)
anthony.s.hughes: review+
Details | Diff | Splinter Review

Description User image Maniac Vlad Florin (:vladmaniac) 2012-02-28 02:43:28 PST
Build ID:
Build identifier: Mozilla/5.0 (X11; Linux i686; rv:13.0) Gecko/20120227 Firefox/13.0a1
---------------------------------------------------------------------------------------
Platform: All Firefox branches
---------------------------------------------------------------------------------------
Mozmill Version: Mozmill 1.5.9
---------------------------------------------------------------------------------------
Error:
([object Object])@resource://mozmill/modules/controller.js:877 ()@resource://mozmill/modules/frame.js -> file:///c:/users/mozilla/appdata/local/temp/tmp6vp5zh.mozmill-tests/tests/functional/testPreferences/testPreferredLanguage.js:79 ((function () {controller.open("about:blank");prefs.openPreferencesDialog(controller, prefDialogCallback);controller.open("http://www.google.com/");controller.waitForPageLoad();if (browserLocale == "it") {controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Zaloguj"));controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Grupy"));controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Szukanie zaawansowane"));} else {controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Accedi"));controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Gruppi"));controller.assertNode(new elementslib.Link(controller.tabs.activeTab, "Ricerca avanzata"));}}))@resource://mozmill/modules/frame.js:557 ([object Object])@resource://mozmill/modules/frame.js:626 ([object Object])@resource://mozmill/modules/frame.js:669 ("c:\\users\\mozilla\\appdata\\local\\temp\\tmp6vp5zh.mozmill-tests\\tests\\functional\\testPreferences\\testPreferredLanguage.js")@resource://mozmill/modules/frame.js:506 ("c:\\users\\mozilla\\appdata\\local\\temp\\tmp6vp5zh.mozmill-tests\\tests\\functional\\testPreferences\\testPreferredLanguage.js")@resource://mozmill/modules/frame.js:681 ((function (filename, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestFile(filename);runner.end();return true;}),[object Proxy])@resource://jsbridge/modules/server.js:179 ("5e20e624-615e-11e1-b7d8-005056f2858a",(function (filename, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestFile(filename);runner.end();return true;}),[object Proxy])@resource://jsbridge/modules/server.js:183 
---------------------------------------------------------------------------------------
First failure date: 2012-02-23
---------------------------------------------------------------------------------------
Results dashboard:
http://mozmill-release.blargon7.com/#/functional/failure?branch=13.0&platform=All&from=2012-02-23&to=2012-02-28&test=%2FtestPreferences%2FtestPreferredLanguage.js&func=testPreferredLanguage.js%3A%3AtestSetLanguages
Comment 1 User image Maniac Vlad Florin (:vladmaniac) 2012-02-28 02:53:19 PST
Created attachment 601227 [details] [diff] [review]
disable v1.0 [landed]

This test should use a local test page - its failing sometimes because of the google page. 

The failure can be manually reproduced but I cannot find a fix for it, test related. 

The test fails on all Firefox default and aurora branches since 2012/02/23, but it also can be reproduced on beta and release branch. 
I recommend disabling this test and decide whether we should have our own test page and this test refactored. 

Please debate your ideas on the bug
Comment 2 User image Henrik Skupin (:whimboo) 2012-02-28 03:29:45 PST
Are you sure the link element still exists? Has its name been changed, or replaced by another element?
Comment 3 User image Maniac Vlad Florin (:vladmaniac) 2012-02-28 03:55:55 PST
(In reply to Henrik Skupin (:whimboo) [away 02/17 - 02/26] from comment #2)
> Are you sure the link element still exists? Has its name been changed, or
> replaced by another element?

As I said, the failure is intermittent so the link exists sometimes, and sometimes it cannot be found.
Comment 4 User image Henrik Skupin (:whimboo) 2012-02-28 03:59:34 PST
Then we should use waitForElement before asserting that the element exists.
Comment 5 User image Maniac Vlad Florin (:vladmaniac) 2012-02-28 04:21:12 PST
(In reply to Henrik Skupin (:whimboo) [away 02/17 - 02/26] from comment #4)
> Then we should use waitForElement before asserting that the element exists.

It does not help at all. There will be a timeout failure in waiting for that specific element. 

http://pastebin.mozilla.org/1494559
Comment 6 User image Henrik Skupin (:whimboo) 2012-02-29 12:09:43 PST
*** Bug 731690 has been marked as a duplicate of this bug. ***
Comment 7 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-02-29 12:49:44 PST
Comment on attachment 601227 [details] [diff] [review]
disable v1.0 [landed]

Patch is fine however I am going to hold off check-in so we can use it as a training check-in for Vlad's commit access tomorrow.
Comment 8 User image Henrik Skupin (:whimboo) 2012-03-02 01:02:31 PST
Can we please get this checked-in? We don't really want to wait until someone has commit privs. We have to disable broken tests immediately. Thanks.
Comment 9 User image Maniac Vlad Florin (:vladmaniac) 2012-03-02 01:11:32 PST
(In reply to Henrik Skupin (:whimboo) from comment #8)
> Can we please get this checked-in? We don't really want to wait until
> someone has commit privs. We have to disable broken tests immediately.
> Thanks.

Anthony is not around at this time, so I guess no one besides you can check it in at this moment
Comment 10 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-02 07:59:49 PST
Comment on attachment 601227 [details] [diff] [review]
disable v1.0 [landed]

Agreed that we can't wait any longer for check-ins. Vlad, once you get your commit access squared away we will practice using whatever is in the current queue.

Patch has now been landed where it fails:
http://hg.mozilla.org/qa/mozmill-tests/rev/36e75bd8fbe9 (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/0210192ecf02 (mozilla-aurora)
Comment 11 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-03 07:30:00 PST
Test also fails for 10.0.3 ESR today:
http://mozmill-release.blargon7.com/#/functional/report/e438d6e3916b2b636037d77445184f64

Patch does not apply cleanly, please provide an ESR skip patch.
Comment 12 User image Maniac Vlad Florin (:vladmaniac) 2012-03-05 02:01:49 PST
Created attachment 602831 [details] [diff] [review]
disable v1.0 (esr) [landed]

Adding skip test patch on esr branch - I don't know why it did not apply cleanly, locally it did for me

This new patch is for mozilla-esr10 branch exclusively, as requested
Comment 13 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-05 14:59:25 PST
Comment on attachment 602831 [details] [diff] [review]
disable v1.0 (esr) [landed]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/236966250704 (mozilla-esr10)
Comment 14 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-06 11:02:03 PST
Comment on attachment 601227 [details] [diff] [review]
disable v1.0 [landed]

Disabled the test on remaining branches as this failed today with Beta. It's not disabled everywhere.

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/aca4c2481e87 (mozilla-beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/4262ee1a05c3 (mozilla-release)
Comment 15 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-06 12:15:04 PST
Comment on attachment 601227 [details] [diff] [review]
disable v1.0 [landed]

Also on mozilla-1.9.2...

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/272a63fb3dd4 (mozilla-1.9.2)
Comment 16 User image Maniac Vlad Florin (:vladmaniac) 2012-03-12 03:21:08 PDT
What can we do to fix this test judging by the fact that this is a Google issue? 
We'd want to fix it as soon as possible as part of our target to provide fixes for skipped tests.
Comment 17 User image Henrik Skupin (:whimboo) 2012-03-12 03:53:35 PDT
Some questions:
1. In such a case does Google load the Italian version of the page?
2. Are the other links we are referencing present?
Comment 18 User image Maniac Vlad Florin (:vladmaniac) 2012-03-12 05:02:03 PDT
(In reply to Henrik Skupin (:whimboo) from comment #17)
> Some questions:
> 1. In such a case does Google load the Italian version of the page?
Yes, the Italian version is loaded, but sometimes the page looks different, probably a bad redirect  
> 2. Are the other links we are referencing present?
The other links are available afaik 

Are you suggesting to just test for another link ?
Comment 19 User image Henrik Skupin (:whimboo) 2012-03-12 07:28:44 PDT
Yes, just use another link which will indeed be there all the time.
Comment 20 User image Maniac Vlad Florin (:vladmaniac) 2012-03-12 07:58:37 PDT
Created attachment 604937 [details] [diff] [review]
fix v1.0 [landed:default]

Proposed initial fix, which passes for all branches. 
Since google changed, we changed the test to search for a link which is always present
Comment 21 User image Henrik Skupin (:whimboo) 2012-03-12 08:11:02 PDT
Comment on attachment 604937 [details] [diff] [review]
fix v1.0 [landed:default]

Looks better.
Comment 22 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-12 11:16:05 PDT
Comment on attachment 604937 [details] [diff] [review]
fix v1.0 [landed:default]

Looks fine to me.
Comment 23 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-12 11:18:09 PDT
Comment on attachment 604937 [details] [diff] [review]
fix v1.0 [landed:default]

Landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/af518eba0d7b (default)
Comment 24 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-03-12 11:18:42 PDT
If this passes on default, I'll go ahead and land on other branches.
Comment 25 User image Dave Hunt (:davehunt) 2012-07-05 15:24:34 PDT
It appears that this is still disabled on mozilla-aurora, mozilla-beta, mozilla-release, and mozilla-esr10
Comment 26 User image Henrik Skupin (:whimboo) 2012-07-05 21:38:18 PDT
(In reply to Dave Hunt (:davehunt) from comment #25)
> It appears that this is still disabled on mozilla-aurora, mozilla-beta,
> mozilla-release, and mozilla-esr10

How can this be? The fix landed in March this year and should have been settled down to probably release meanwhile. Anthony, haven't you merged those skip changes whenever we had a version bump? If yes, there should be more than this single one probably we should fix. Dave, have you seen something else?
Comment 27 User image Henrik Skupin (:whimboo) 2012-07-05 21:42:39 PDT
Not sure what actually happened here, but the code changes have been merged except that the skipped lines have been removed. We just have to unskip those tests on older branches and have to find out why that happened.
Comment 28 User image Henrik Skupin (:whimboo) 2012-07-05 21:44:41 PDT
Created attachment 639581 [details] [diff] [review]
unskip patch

Dave or Anthony, whoever comes first please review this patch.
Comment 29 User image Henrik Skupin (:whimboo) 2012-07-05 21:50:58 PDT
So the merge went wrong from default to mozilla-aurora and the skip lines have been re-added:

http://hg.mozilla.org/qa/mozmill-tests/rev/e070324ab322

Anthony, when you do merges and you are unclear please always consult the bug and appropriate dashboard results. Usually those unskip patches have to be forwarded to the next branch. The only exception are remote tests for the discovery pane so far.
Comment 30 User image Henrik Skupin (:whimboo) 2012-07-05 21:52:09 PDT
Transplanted original patch to ESR10:
http://hg.mozilla.org/qa/mozmill-tests/rev/299061be6cdf
Comment 31 User image Anthony Hughes (:ashughes) [GFX][QA][Mentor] 2012-07-05 22:01:05 PDT
Comment on attachment 639581 [details] [diff] [review]
unskip patch

Patch looks fine to me.

Note that with skipped tests, I err on the side of caution. If a test is skipped in Firefox 15 Nightly, I assume it needs to continue to be disabled when it merges, unless the comment specifically indicates the test is fine for Aurora. It's far better, in my opinion, to have to re-enable unintentionally skipped tests then to have to disable unintentionally enabled tests which are failing.

Perhaps we need a more descriptive commenting scheme for skipped blocks so that it is more evident.
Comment 32 User image Henrik Skupin (:whimboo) 2012-07-05 22:07:10 PDT
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #31)
> Note that with skipped tests, I err on the side of caution. If a test is
> skipped in Firefox 15 Nightly, I assume it needs to continue to be disabled
> when it merges, unless the comment specifically indicates the test is fine
> for Aurora. It's far better, in my opinion, to have to re-enable

That's true but a different scenario. In this case the test wasn't marked as skip anymore. The final patch removed the skip lines. See your comment 23 with the changeset landed. 

Pushed to remaining branches:
http://hg.mozilla.org/qa/mozmill-tests/rev/0381a4a8ff08 (aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/9448e8b6ca1f (beta)
http://hg.mozilla.org/qa/mozmill-tests/rev/e4d1ddec0276 (release)

Vlad, please check all the Litmus tests and re-enable those for mozmill if necessary.
Comment 33 User image Maniac Vlad Florin (:vladmaniac) 2012-07-06 01:05:00 PDT
There is no Litmus work to be done here as the test is no longer available. 

I did: 
1. Browsed 'manage testcases' section in Litmus and found no test under Options/Preferences to match this bug
2. Browsed our mozmill repo, took the litmus id from the map and searched - no results 

https://litmus.mozilla.org/single_result.cgi?id=8322 

In conclusion, dismissing the litmus flag
Comment 34 User image Maniac Vlad Florin (:vladmaniac) 2012-07-06 01:21:56 PDT
It seems that we have the test for 3.6 branch 

https://litmus.mozilla.org/show_test.cgi?id=8322
Comment 35 User image Maniac Vlad Florin (:vladmaniac) 2012-07-06 01:27:13 PDT
Following Litmus tests are not disabled 

aurora: https://litmus.mozilla.org/show_test.cgi?id=15786
release: https://litmus.mozilla.org/show_test.cgi?id=55779
beta: https://litmus.mozilla.org/show_test.cgi?id=64133

We need to re-enable for esr
Comment 36 User image Maniac Vlad Florin (:vladmaniac) 2012-07-06 01:30:48 PDT
Enabled for mozmill group and edited the testcase to point that 
https://litmus.mozilla.org/show_test.cgi?id=40884
Comment 37 User image Dave Hunt (:davehunt) 2012-07-06 02:17:01 PDT
(In reply to Henrik Skupin (:whimboo) from comment #26)
> (In reply to Dave Hunt (:davehunt) from comment #25)
> > It appears that this is still disabled on mozilla-aurora, mozilla-beta,
> > mozilla-release, and mozilla-esr10
> 
> How can this be? The fix landed in March this year and should have been
> settled down to probably release meanwhile. Anthony, haven't you merged
> those skip changes whenever we had a version bump? If yes, there should be
> more than this single one probably we should fix. Dave, have you seen
> something else?

This is the only one I've seen.

Note You need to log in before you can comment on or make changes to this bug.