1154945 - Test failures in addon tests due to changed behavior in installing addons

Status: RESOLVED INVALID

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Syd Polk :sydpolk]

Module:    functional
Test:      testAddons    
Failure:   "Addon child element has been found."
Branches:  mozilla-beta
Platforms: all

This is failing for all builds of Firefox 37, Firefox 38 and Firefox 39 for Desktop.

On Firefox 40, the error message is:

"AddonsManager_isAddonInstalled: Add-on has been specified. - got 'undefined'"

Comment 1

[Robert Kaiser]

Mossop, do you know someone who knows the add-ons code well and has time to find out if those failures are test or code issues?

Comment 2

[Dave Townsend [:mossop]]

Honestly no-one meets both those criteria right now.

Based on the code I'm guessing that 40 behaves differently because of the changes to the add-on install UI.

Was this passing in 36?

Comment 3

[Syd Polk :sydpolk]

This broke in mozmill-tests with rev a14822a6b10d. I will disable the tests for all releases. There are also failures starting 3/25 even with this backed out because FF now presents a confirmation alert for installing extensions.

Comment 4

[Dave Townsend [:mossop]]

If a mozmill change broke this then it seems likely this is a test issue. The change looks to be disabling a test just on windows though so the fact it broke all platforms is surprising. If it was really that change then sounds like testPluginInDifferentStates.js isn't cleaning up properly or something.

Comment 5

[Syd Polk :sydpolk]

There are two issues.

Mozmill changes caused some failures in this files. Others later were caused by additional UI in the extension addition interface.

Comment 6

[Syd Polk :sydpolk]

So, most test files in firefox/tests/functional/testAddons have failures in all versions of Firefox. I will be disabling them all.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Syd Polk :sydpolk from comment #6)
> So, most test files in firefox/tests/functional/testAddons have failures in
> all versions of Firefox. I will be disabling them all.

No work happened here in the last two weeks. So I'm going to transform this bug to cover the topic Dave mentioned above with the changes for Firefox 40. Any failure in testChangeTheme.js is already covered by another bug. This should really be add-on general.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: skip.patch

I landed this skip patch which disables all tests which are installing add-ons.

https://hg.mozilla.org/qa/mozmill-tests/rev/a12f79e5074d (default)

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As Dave said this regression is caused by a change in the doorhanger for add-on installation. As the dashboard states the failures started on March 25th. A quick look at the pushlog [1] shows me that it is related to bug 1139656. Maybe it's just another id for the install button, which needs an update. I will know soon.

[1] https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=cc0950b7a369&tochange=db0409de517a

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The installation process has been totally changed and it would take too much time for me now to get this fixed in the Mozmill tests. A better approach here would be to wait for the implementation of those tests as Firefox UI tests. With the debugger capabilities of Marionette it will be much easier to solve.

I will leave those tests as disabled but won't close the bug right now.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Mozmill tests have been superseded by Marionette tests.