679330 - Test failure "aElement is undefined" for Discovery Pane tests

Status: RESOLVED WONTFIX

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Maniac Vlad Florin (:vladmaniac)]

MODULE: tests/remote/restartTests/testDiscoveryPane_installPickOfMonthAddon/test1.js    
TEST:   test1.js   
FAIL:   ERROR | Test Failure: {"exception": {"stack": "DiscoveryPane_getInstallSource((void 0))@resource://mozmill/stdlib/securable-module.js -> file:///home/vladmaniac/Desktop/discpanemonthtest/mozmill-tests/lib/addons.js:1259\n@:0\n", "message": "aElement is undefined", "fileName": "resource://mozmill/stdlib/securable-module.js -> file:///home/vladmaniac/Desktop/discpanemonthtest/mozmill-tests/lib/addons.js", "name": "TypeError", "lineNumber": 1259}}

BRANCH: mozilla-beta, mozilla-release

Comment 1

[u415141]

A Pivotal Tracker story has been created for this Bug: https://www.pivotaltracker.com/story/show/17058875

Comment 2

[Maniac Vlad Florin (:vladmaniac)]

Removing the getInstallSource() callback gets us a jsbridge error 

ERROR | Test Failure: {"exception": {"stack": "([object Proxy])@resource://mozmill/modules/controller.js:489\ntestInstallPickOfTheMonthAddon()@resource://mozmill/modules/frame.js -> file:///home/vladmaniac/Desktop/discpanemonthtest/mozmill-tests/tests/remote/restartTests/testDiscoveryPane_installPickOfMonthAddon/test1.js:98\n(testInstallPickOfTheMonthAddon)@resource://mozmill/modules/frame.js:554\n([object Object])@resource://mozmill/modules/frame.js:623\n([object Object])@resource://mozmill/modules/frame.js:666\n(\"/home/vladmaniac/Desktop/discpanemonthtest/mozmill-tests/tests/remote/restartTests/testDiscoveryPane_installPickOfMonthAddon/test1.js\")@resource://mozmill/modules/frame.js:503\n(\"/home/vladmaniac/Desktop/discpanemonthtest/mozmill-tests/tests/remote/restartTests/testDiscoveryPane_installPickOfMonthAddon/test1.js\")@resource://mozmill/modules/frame.js:678\n((function (filename, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestFile(filename);runner.end();return true;}),[object Proxy])@resource://jsbridge/modules/server.js:179\n(\"2f2fa7d8-c80d-11e0-bada-000049905903\",(function (filename, invokedFromIDE) {var runner = new Runner(new Collector, invokedFromIDE);runner.runTestFile(filename);runner.end();return true;}),[object Proxy])@resource://jsbridge/modules/server.js:183\n@:0\n", "message": "elem.getNode is not a function", "fileName": "resource://mozmill/modules/controller.js", "name": "TypeError", "lineNumber": 489}}

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Please post a link into the URL field of all the failures or at least an example. Also some code snippets around the fialing line would be helpful.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Where does it fail? I also don't see such reports on our release dashboard.

Comment 5

[u279076]

This failure is not on mozmill-release. Based on the error logs posted above it looks to be local to Vlad's environment.

Comment 6

[Maniac Vlad Florin (:vladmaniac)]

Yep, it's a result from running testrun_remote.py locally, without --report=URL option

Comment 7

[u279076]

This test is already landed on release and beta -- is this blocking landing of the test on other branches? If not, I would not waste much time on it.

Comment 8

[Maniac Vlad Florin (:vladmaniac)]

this test is not applicable for other branches, we skip it for default and aurora. I suspect an amo change causes this, because it happens on all firefox versions from 4 to 8 and it didn't fail with this before. 

I am currently irc - ing with krupa and marlena on this one maybe we can figure it out.

Comment 9

[u279076]

Ok -- I would just caution you against spending too much time on this issue. If it is a blocker to new test development then we can escalate it. However, since it's not happening anywhere else other than your local environment, it's not a big priority.

If after a couple hours you don't have a clearer idea of why this is happening, please mark this bug UNCONFIRMED and move onto something higher priority.

Thanks

Comment 10

[krupa raj[:krupa]]

We changed the way we choose add-ons for 'Mozilla pick of the month' panel. That is what is causing the error. The test needs to be updated.

Comment 11

[Maniac Vlad Florin (:vladmaniac)]

Thanks for clarifications krupa!

Comment 12

[u279076]

(In reply to krupa raj 82[:krupa] from comment #10)
> We changed the way we choose add-ons for 'Mozilla pick of the month' panel.
> That is what is causing the error. The test needs to be updated.

When will these changes be pushed to production?

Comment 13

[krupa raj[:krupa]]

There are a few outstanding bugs. Once they are resolved, it will be available in prod (probably in 2 weeks).

Also:

1. The selection will now be locale-specific. If there is no selection for a locale, we show the add-on selected for the default locale.

2. More details are available at bug 676307.

Comment 14

[u279076]

So timing will be crucial here. Vlad, please implement the changes necessary for the test. We will refrain from checking in until the change has landed on production.

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+1]]

We do not use the staging site for our tests so anything which has been changed and not landed on production yet, cannot be the reason for this failure.

Vlad, please see comment 3 and post some more information. Thanks.

Comment 16

[u279076]

(In reply to Henrik Skupin (:whimboo) from comment #15)
> We do not use the staging site for our tests so anything which has been
> changed and not landed on production yet, cannot be the reason for this
> failure.
> 
> Vlad, please see comment 3 and post some more information. Thanks.

My point in comment 14 was more about being proactive about refactoring this test for the upcoming change and not directed at thia failure. Henrik's point about changes in staging not being the cause of this failure is probably accurate.

Comment 17

[Maniac Vlad Florin (:vladmaniac)]

Report of the failure: http://mozmill-crowd.brasstacks.mozilla.com/#/remote/report/76f06d5489983c787231f337efd1cc57

Comment 18

[Maniac Vlad Florin (:vladmaniac)]

The error occurs at this line of code 

var currentInstallSource = discovery.getInstallSource(addToFirefox) 

Checked getInstallSource() in addons.js and it's fine, passes on other tests, and the expected install source src  = "discovery-promo" can be found if inspecting addToFirefox link. 

When removing the above code from the test, the echoed error is the one from comment 2

Comment 19

[Henrik Skupin [:whimboo][⌚️UTC+1]]

You cannot remove this line because currentInstallSource will be undefined when it is used. So the addToFirefox button is a valid element and getInstallSource returns the correct value? I suspect the former assumption is not valid, at least when seeing the error from comment 1.

Comment 20

[Maniac Vlad Florin (:vladmaniac)]

addToFirefox is the problem, from my point of view. I did not remove only that line, i removed everything install source related so that the test does not check for src anymore. 

After removing that, the test freezes after getting to the pick of the month pane, and doesn't click on addToFirefox. Instead, it echoes the error from comment 2

Comment 22

[Maniac Vlad Florin (:vladmaniac)]

We'll not fix this since the PickOfMonth Panel from Collections is removed from the DiscoveryPane tests

Comment 23

[Henrik Skupin [:whimboo][⌚️UTC+1]]

No, there are also some other tests which are affected by this. It's an underlying issue not directly tied to this specific remote test.

Also can you be a bit more clear on the fact that this panel will be removed? When will that happen? It would mean we also have to remove the test from our repository.

Comment 24

[Maniac Vlad Florin (:vladmaniac)]

Filed bug 688146 to remove those tests from hg

Comment 25

[u279076]

This is WONTFIX because we are currently disabling all Discovery Pane tests due to their volatility. In the future, we should not develop tests for features which are tied to ever changing web content.