924797 - Test failure "Shockwave Flash state is Disabled in about:plugins - got 'false'" in testAddons/testPluginDisableAffectsAboutPlugins.js

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Cosmin Malutan, [:cosmin-malutan]]

This failure happened today on as and ko locales with Beta on Ubuntu 12.04(x86) and 13.04(x64).
It is reproducible and it seems that after we disable the plugins, when we check the state on addonsManager they are disabled and on about:plugins they are enabled.
It did't reproduces on en-US.
http://mozmill-release.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8bea6dacdc
http://mozmill-release.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8bea6c4233

Comment 1

[Cosmin Malutan, [:cosmin-malutan]]

It fails on some locales because when testing plugin state in "about:plugins" we check if dtd string "state_enabled" is contained in the plugin state. 
On some languages the "disabled" word is formed from "enabled" plus a negation prefix (something like "not-enabled"), so when we check if state contains "enabled" it will return true on both enabled and disabled state.
I have a fix patch for this, but I want to test it properly first on more locales, so I will return with the fix patch soon.

Comment 2

[Cosmin Malutan, [:cosmin-malutan]]

Attachment: patch_v1.0

Here is the patch for beta.
Reports:
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaa59e59
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaaa2a38
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaaa3032
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaaa4042
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaaabf81
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaaab06a
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaaaaf6f
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaab031d
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaab10bd
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaab3520

Comment 3

[Andrei Eftimie]

Comment on attachment 814896 [details] [diff] [review]
patch_v1.0

Review of attachment 814896 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good, should be more reliable.

Landed down to beta:
http://hg.mozilla.org/qa/mozmill-tests/rev/f03f4d907d3e (default)
http://hg.mozilla.org/qa/mozmill-tests/rev/30a1e2535683 (mozilla-aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/1604770a5d84 (mozilla-beta)

I am assuming lower branches are affected as well, but please check if they are and if this applies cleanly on them as well.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Andrei Eftimie from comment #3)
> Landed down to beta:
> http://hg.mozilla.org/qa/mozmill-tests/rev/f03f4d907d3e (default)
> http://hg.mozilla.org/qa/mozmill-tests/rev/30a1e2535683 (mozilla-aurora)
> http://hg.mozilla.org/qa/mozmill-tests/rev/1604770a5d84 (mozilla-beta)

Why are we landing fixes on aurora and beta as well? This can hurt us a lot as mentioned a couple of times when especially beta starts to fail now. We really have to be cautious about landings and get green results on default first.

Comment 5

[Andrei Eftimie]

As discussed on IRC, I'll no longer land on lower branches until we have green runs in CI on default first.

If the failure is on localised builds against which we don't run daily testruns, we should trigger them manually on those specific locales.

Comment 6

[Cosmin Malutan, [:cosmin-malutan]]

The patch applies cleanly on esr24 and release too.

http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beae70aa9
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beae7597e
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beae6f75c

Here are reports from default:
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beae14695
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beae110f8
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beae12dff

Comment 7

[Cosmin Malutan, [:cosmin-malutan]]

Esr17 is unaffected as it handles this check differently (it checks if addon is on about:plugins and not if it has an enabled/disabled state).

Here are the reports from esr with an affected locale:
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaed59e0
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaecfb78
http://mozmill-crowd.blargon7.com/#/functional/report/6d6f6a58b02eeffc06eafa8beaed0d83

Comment 8

[Andrei Eftimie]

No failures on CI testruns, lets get this on the remaining affected branches:

http://hg.mozilla.org/qa/mozmill-tests/rev/aa510e02bfd9 (mozilla-release)
http://hg.mozilla.org/qa/mozmill-tests/rev/f7bec5889609 (mozilla-esr24)

Comment 9

[Andrei Eftimie]

I've noticed this failure on ESR17 on a local run:
http://mozmill-crowd.blargon7.com/#/functional/report/ec449c026814fd64783d738c02983391

Cosmin, could you please check and confirm if it is the same issue, or if it is something else we might want to fix it in another bug.

The failure message is the same, but in comment 7 you state that ESR17 is not affected.

Comment 10

[Cosmin Malutan, [:cosmin-malutan]]

Hi Andrei, it's not the same issue, as I said on comment 7, in Esr17 we check if the plugin is listed in the about:plugins page and not if there is an property "disabled" in plugins content. So being a different issue I raised bug 927361 for this.