Test failure "'false' should equal 'true'" in testAddons_changeTheme/test3.js

RESOLVED INVALID

Status

Mozilla QA
Mozmill Tests
RESOLVED INVALID
5 years ago
a year ago

People

(Reporter: cosmin, Unassigned)

Tracking

unspecified
Dependency tree / graph

Firefox Tracking Flags

(firefox35 affected, firefox36 fixed, firefox37 fixed, firefox38 fixed, firefox-esr31 affected)

Details

(URL)

Attachments

(1 attachment)

(Reporter)

Description

5 years ago
Happened today on Windows 7 x64 with Nightly de
I couldn't reproduce it locally.

http://mozmill-daily.blargon7.com/#/functional/report/b7ef1fb3d9703aeaf2c46e07d281e719

Comment 1

5 years ago
The URL field should point to a list of all failures if it is a functional test.
Also remember to set the OS/tracking flags.
status-firefox22: --- → unaffected
status-firefox23: --- → unaffected
status-firefox24: --- → unaffected
status-firefox25: --- → unaffected
status-firefox26: --- → affected
status-firefox-esr17: --- → unaffected
OS: Linux → Windows 7
Depends on: 905170

Comment 2

5 years ago
This fails in test3 without the mentioned failure in test2 from bug 905170
No longer depends on: 905170

Comment 3

5 years ago
Happened again today on Mac OS X 10.8.4 (x86_64) with Firefox 26.0a2 'it' using Mozmill 2.0:
http://mozmill-daily.blargon7.com/#/functional/report/6ec6776efe900da3fd2b64a750c0152d

Comment 4

4 years ago
Happened today on MAC OS X 10.7 with Aurora zh-CN.
http://mozmill-daily.blargon7.com/#/functional/report/c90eeeb0a5511695efc7462553aa0fe1
(Reporter)

Comment 5

4 years ago
Failed again with Beta  ja-JP-mac on mac 10.8 (mm-osx-108-4)
http://mozmill-release.blargon7.com/#/functional/report/44e0169acb64b68a44c1ad880b35e38d

Comment 6

4 years ago
This failure is just fallout from bug 972912.
Depends on: 972912
Test failure for Firefox 31.0 (31.0, he, 20140616143923), Mac OS X 10.7.5 (x86_64).

/restartTests/testAddons_changeTheme/test3.js 	testChangedThemeToDefault 	

    'false' should equal 'true' 

http://mozmill-release.blargon7.com/#/functional/report/6959f9a8236961096763389083a7a3d7

Comment 9

4 years ago
(In reply to Andreea Matei [:AndreeaMatei] from comment #8)
> Failed on Beta ru, on OS X 10.7.5:
> http://mozmill-release.blargon7.com/#/functional/report/
> 883d8406dab55a771a00934158ae8cac

Also fallout from bug 972912
This has failed 5 times in the last week on Beta:
http://mozmill-release.blargon7.com/#/functional/failure?app=Firefox&branch=31.0&platform=Mac&from=2014-07-01&to=2014-07-08&test=%2FrestartTests%2FtestAddons_changeTheme%2Ftest3.js&func=testChangedThemeToDefault

Can we get this test disabled for Firefox 31?
status-firefox26: affected → wontfix
status-firefox31: --- → affected
OS: Windows 7 → All
Hardware: x86_64 → All

Comment 11

4 years ago
(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #10)
> This has failed 5 times in the last week on Beta:
> http://mozmill-release.blargon7.com/#/functional/
> failure?app=Firefox&branch=31.0&platform=Mac&from=2014-07-01&to=2014-07-
> 08&test=%2FrestartTests%2FtestAddons_changeTheme%2Ftest3.
> js&func=testChangedThemeToDefault
> 
> Can we get this test disabled for Firefox 31?

Unfortunately this is not this tests fault, but the harness itself.
It's the same issue as mentioned before. We have 3 test files and test2 was not run!
This is again mozmill bug 972912.

One thing we might do is to refactor all restart tests to be just 1 file, we should be able to avoid these kind of failures where 1 testfile is not running.

Updated

4 years ago
Depends on: 1036294

Comment 13

4 years ago
(In reply to Teodor Druta from comment #12)
> Created attachment 8531198 [details] [diff] [review]
> skipchangetheme.patch
> 
> Multiple failures on OSX beta/aurora/default branches on multiple locales
> 
> http://mozmill-daily.blargon7.com/#/functional/
> failure?app=Firefox&branch=37&platform=Mac&from=2014-11-
> 25&to=&test=%2FrestartTests%2FtestAddons_changeTheme%2Ftest3.
> js&func=testChangedThemeToDefault
> http://mozmill-release.blargon7.com/#/functional/
> failure?app=Firefox&branch=35&platform=Mac&from=2014-11-
> 25&to=&test=%2FrestartTests%2FtestAddons_changeTheme%2Ftest3.
> js&func=testChangedThemeToDefault
> 
> I say we should skip this test.

Actually it seems that this is the failure reported in Bug 972912.

Updated

4 years ago
Attachment #8531198 - Flags: review?(mihaela.velimiroviciu)
Attachment #8531198 - Flags: review?(andreea.matei)
So this can only happen with release and esr31 branches, as the other have the test in a single file.
status-firefox22: unaffected → ---
status-firefox23: unaffected → ---
status-firefox24: unaffected → ---
status-firefox25: unaffected → ---
status-firefox26: wontfix → ---
status-firefox31: affected → ---
status-firefox35: --- → affected
status-firefox36: --- → fixed
status-firefox37: --- → fixed
status-firefox38: --- → fixed
status-firefox-esr17: unaffected → ---
status-firefox-esr31: --- → affected
Mozmill tests have been superseded by Marionette tests.
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.