1113709 - Test failure "Plain-theme is marked to be enabled. - 'upgrade' should equal 'enable'" in testChangeTheme.js

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Mihaela Velimiroviciu (:mihaelav)]

Module:    testInstallTheme
Test:      testChangeTheme.js
Failure:   Plain-theme is marked to be enabled. - 'upgrade' should equal 'enable'
Branches:  37a1.0
Platforms: All

Reports:
http://mozmill-daily.blargon7.com/#/functional/failure?app=Firefox&branch=All&platform=All&from=2014-12-12&to=2014-12-19&test=%2FtestAddons%2FtestChangeTheme.js&func=testInstallTheme

Comment 1

[Cosmin Malutan, [:cosmin-malutan]]

This fails all the time, it should have been disabled! 
http://mozmill-daily.blargon7.com/#/functional/failure?app=Firefox&branch=All&platform=All&from=2014-12-12&to=&test=%2FtestAddons%2FtestChangeTheme.js&func=testInstallTheme

Comment 2

[Cosmin Malutan, [:cosmin-malutan]]

Attachment: skip-patch

Henrik can you please review this skip-patch?

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 8539994 [details] [diff] [review]
skip-patch

Review of attachment 8539994 [details] [diff] [review]:
-----------------------------------------------------------------

This is a regression, so please get this one investigated ASAP. Something seems to be wrong with the theme versioning?

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 8539994 [details] [diff] [review]
skip-patch

Review of attachment 8539994 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry, but this patch cannot be applied.

Comment 5

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 8539994 [details] [diff] [review]
skip-patch

Review of attachment 8539994 [details] [diff] [review]:
-----------------------------------------------------------------

It applied well to me, tested, landed:
http://hg.mozilla.org/qa/mozmill-tests/rev/0432a77a4c31 (default)

Something is strange as we have multiple tests with similar issues:
bug 1112517 - Test failure 'Service is no longer marked for uninstall - 'uninstall' should equal ''
bug 1108948 - Test failure 'Addon is no longer marked for uninstall' 
We have to investigate this.

Comment 6

[Cosmin Malutan, [:cosmin-malutan]]

This is strange, all Jenkins jobs failed since Friday on production/staging I couldn't reproduce it locally or on production. The extension didn't changed so I don't think it might be a change in addon version, I expect to be a change either in library or in Firefox itself.

Comment 7

[Andreea Matei [:AndreeaMatei]]

I see it fails at an assert.equal, maybe it's a race condition and we need to waitFor there:
http://hg.mozilla.org/qa/mozmill-tests/file/default/firefox/tests/functional/testAddons/testChangeTheme.js#l99

Comment 8

[Cosmin Malutan, [:cosmin-malutan]]

I checked with a build from weekend, it still passes for me so the remaining thing I can think of it might be the machine loading so a waitFor would be a good try.

Comment 9

[Cosmin Malutan, [:cosmin-malutan]]

Attachment: patch v1.0

Andreea can you check this?

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Cosmin Malutan from comment #8)
> I checked with a build from weekend, it still passes for me so the remaining
> thing I can think of it might be the machine loading so a waitFor would be a
> good try.

Where did you check that? On one of the machines in SCL3? It was failing all the time as you stated, so I don't see why you cannot reproduce it anymore now.

Comment 11

[Cosmin Malutan, [:cosmin-malutan]]

I failed on production on all nodes, you might check mozilla-central_functional jobs on Jenkins, as I stated all jobs failed. I couldn't reproduce the failure on neither locally or on production. I ran the test itself at least 20 times on production and locally with latest Nightly, also complete testruns.
I keep testing it.

Comment 12

[Cosmin Malutan, [:cosmin-malutan]]

Oh my, I think I did something wrong firs time I ran a complete testrun. It fails all the times when I ran complete testruns. The theme is already installed when we ran the test.
The reason is that with the re-factoring from bug 1108993, we don't uninstall the theme installed in the first restart test, which bu the way is duplicated and should be fully removed. 
The re-factored test was previously removing the Theme installed in 
> https://github.com/mozilla/qa-mozmill-tests/blob/master/firefox/tests/functional/restartTests/testAddons_installTheme/manifest.ini
Now the re-factored test runs after the change theme test.
All that we have to do is to fully remove the restartTests/testAddons_installTheme as is duplicated in testInstallUninstallTheme

Comment 13

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 8540123 [details] [diff] [review]
patch v1.0

Review of attachment 8540123 [details] [diff] [review]:
-----------------------------------------------------------------

Meanwhile Cosmin reproduced this after multiple testruns, he's working on the real cause and fix.

Comment 14

[Cosmin Malutan, [:cosmin-malutan]]

It will be fixed with bug 1110639

Comment 15

[Andreea Matei [:AndreeaMatei]]

We fixed that on nightly, will continue with the other branches there but here it was the only one affected. Cosmin, please recheck now to be sure I can backout the skip. Thanks!

Comment 16

[Cosmin Malutan, [:cosmin-malutan]]

Yep, that fixed it, it can be re-enabled on default.
http://mozmill-crowd.blargon7.com/#/functional/report/f4189e259b668417231c4d9ed055fc87

Comment 17

[Andreea Matei [:AndreeaMatei]]

Enabled:
http://hg.mozilla.org/qa/mozmill-tests/rev/266a4b541b43 (default)
Thanks!