905170 - Test failure "The theme '[e-mail]' is enabled - got 'false'" in testAddons_changeTheme/test2.js

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect, P1)

Description

[Cosmin Malutan, [:cosmin-malutan]]

Happened today on Windows Vista (x86) with Nightly it:
http://mozmill-daily.blargon7.com/#/functional/report/b7ef1fb3d9703aeaf2c46e07d2bd253b

I could't reproduce it locally, probably it's an machine issue.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This is suspicious and I would like to see some investigation today. When I see the list of add-ons in the report the default theme is marked as enabled and compatible. Could this be a misbehavior of the Add-ons manager API?

Failed on Vista (mm-win-vista-3) again:
http://mozmill-daily.blargon7.com/#/functional/report/b7ef1fb3d9703aeaf2c46e07d2cb817b

> Tema predefinito 26.0a1 (enabled, compatible)

Comment 2

[Andreea Matei [:AndreeaMatei]]

This failed again today on fr locale, Windows Vista.
I believe bug 904610 is dependent on this one as it failed once on the following test (test3) and since then we only caught this error in test2.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Which Vista machine? I believe it's always the same node which causes us the problems here.

Comment 4

[Andreea Matei [:AndreeaMatei]]

mm-win-vista-3 indeed, it's the same node as you stated in comment 1.

Comment 5

[Andrei Eftimie]

Does not seem to be related to 1 machine.
Failed on mm-win-vista-32-2 (I do remeber we recently renamed them, is this the same machine mentioned previously?) and on mm-win-7-32-3

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This fails intermittently, so not a P1 but P2.

Comment 7

[Andrei Eftimie]

This fails 2-3 times a day on CI, yet I haven't been able to reproduce the failure locally in 100 runs (slimmed down testruns)

Comment 8

[Andrei Eftimie]

I wasn't able to reproduce this on CI either.
(Same machine, same FF build)

This is puzzling since it fails a couple of times per day on the regular testruns.

Comment 9

[Cosmin Malutan, [:cosmin-malutan]]

It fails on all machines with Nightly and it reproduces locally so we may want to skip this test.
http://mozmill-daily.blargon7.com/#/functional/report/26d5854914a68aa13b333f281597c249
http://mozmill-daily.blargon7.com/#/functional/failure?branch=All&platform=All&from=2013-09-04&to=&test=%2FrestartTests%2FtestAddons_changeTheme%2Ftest2.js&func=test2.js%3A%3AtestThemeIsInstalled

Comment 10

[Andrei Eftimie]

Attachment: skip.patch

Yes, this reproduces constantly now with the latest nightly.

Here is a skip patch.
I'll do a regression range now.

Comment 11

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 803005 [details] [diff] [review]
skip.patch

Review of attachment 803005 [details] [diff] [review]:
-----------------------------------------------------------------

The nit applies to all test files.

::: tests/functional/restartTests/testAddons_changeTheme/test1.js
@@ +101,5 @@
> +
> +setupModule.__force_skip__ = "Bug 905170 - The theme '[e-mail]'" +
> +                             "is enabled - got 'false'";
> +teardownModule.__force_skip__ = "Bug 905170 - The theme '[e-mail]'" +
> +                                "is enabled - got 'false'";

Small update needed here cause the message will get linked together without a whitespace between the concatenated strings.

Comment 12

[Andrei Eftimie]

Attachment: skip.patch

Added a space to have a nice message.

Comment 13

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 803015 [details] [diff] [review]
skip.patch

Review of attachment 803015 [details] [diff] [review]:
-----------------------------------------------------------------

Disabled:
http://hg.mozilla.org/qa/mozmill-tests/rev/60eccf81d91b (default)

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+1]]

What is the problem here, Andrei? We have to get this fixed asap.

Comment 15

[Andrei Eftimie]

Clear regression.

Passes with Nightly from 2013-09-10
Fails with Nightly from 2013-09-11

Working with tinderbox builds to get the regressor changeset.

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Not sure why this has been set to critical, we don't have a crash, abort, or freeze here.

Comment 17

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Andrei Eftimie from comment #15)
> Passes with Nightly from 2013-09-10
> Fails with Nightly from 2013-09-11

Sounds like a different issue given that this bug is already open a couple of weeks. But lets see.

Comment 18

[Andrei Eftimie]

(In reply to Henrik Skupin (:whimboo) from comment #17)
> (In reply to Andrei Eftimie from comment #15)
> > Passes with Nightly from 2013-09-10
> > Fails with Nightly from 2013-09-11
> 
> Sounds like a different issue given that this bug is already open a couple
> of weeks. But lets see.

Indeed, lets see what changed for this to fail constantly.
It _might_ be a completely different issue.

Comment 19

[Andrei Eftimie]

Regressor is bug 888347
Changeset: http://hg.mozilla.org/integration/mozilla-inbound/rev/efc4235dd32d

Does this happen because we're in a restart test?

Comment 20

[Andrei Eftimie]

First thought was that startUserShutdown() might need an update, but we also fail in mozmill 2.0 with restartApplication()

Comment 21

[Andrei Eftimie]

The new failure has the same cause as bug 915193

I propose to move all discussions there, since the original failure in this bug is probably unrelated.

Comment 22

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As what is mentioned on bug 911820 it explains why it was failing sporadically before. Lets hope for a quick fix.

Comment 23

[Andreea Matei [:AndreeaMatei]]

Fixed by bug 911621 and reenabled in bug 915193.