1032073 - Test failure 'aModule.tabBrowser is undefined' in /testAddons/testSearchAddons.js and /testGeolocation/testShareLocation.js

Status: RESOLVED INVALID

(Mozilla QA Graveyard :: Mozmill Tests, defect, P1)

Description

[Andrei Eftimie]

Module:    teardownModule
Test:      /testAddons/testSearchAddons.js    
Failure:   aModule.tabBrowser is undefined
Branches:  mozilla-aurora
Platforms: OSX
Report:    http://mozmill-daily.blargon7.com/#/remote/report/59ac3f70f127c02da3a59eb1fc195f2e

Failed twice. Both on OSX.

Weird failure since `aModule.tabBrowser` is created in setupModule and used in the actual test. The failure happened in `teardownModule`...

There's a weird message in the Console _before_ the testrun started. I can't see how that could affect this failure, but here it is:
> 09:50:06 Started by user anonymous
> 09:50:06 ln builds/lastSuccessfulBuild /data/mozmill-ci/jenkins-master/jobs/mozilla-aurora_remote/lastSuccessful failed
> 09:50:06 java.nio.file.FileAlreadyExistsException: /data/mozmill-ci/jenkins-master/jobs/mozilla-aurora_remote/lastSuccessful
> 09:50:06  at sun.nio.fs.UnixException.translateToIOException(UnixException.java:88)
> 09:50:06  at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> 09:50:06  at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> 09:50:06  at sun.nio.fs.UnixFileSystemProvider.createSymbolicLink(UnixFileSystemProvider.java:456)
> 09:50:06  at java.nio.file.Files.createSymbolicLink(Files.java:996)
> 09:50:06  at sun.reflect.GeneratedMethodAccessor221.invoke(Unknown Source)
> 09:50:06  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 09:50:06  at java.lang.reflect.Method.invoke(Method.java:606)
> 09:50:06  at hudson.Util.createSymlinkJava7(Util.java:1148)
> 09:50:06  at hudson.Util.createSymlink(Util.java:1064)
> 09:50:06  at hudson.model.Run.createSymlink(Run.java:1790)
> 09:50:06  at hudson.model.Run.updateSymlinks(Run.java:1771)
> 09:50:06  at hudson.model.Run.execute(Run.java:1682)
> 09:50:06  at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> 09:50:06  at hudson.model.ResourceController.execute(ResourceController.java:88)
> 09:50:06  at hudson.model.Executor.run(Executor.java:231)
> 09:50:06 [EnvInject] - Loading node environment variables.
> 09:50:06 Building remotely on mm-osx-108-3 (mac 64bit 10.8) in workspace jenkins/workspace/mozilla-aurora_remote
> 09:50:06 
> 09:50:08 Deleting project workspace... done
> 09:50:08 
> 09:50:08 Restoring workspace from build #89622 of project get_mozmill-tests

Comment 1

[Andrei Eftimie]

This hasn't failed again since the initial report (3 times total on the 29th and 30th June).
http://mozmill-daily.blargon7.com/#/remote/failure?app=Firefox&branch=All&platform=All&from=2014-06-29&test=%2FtestAddons%2FtestSearchAddons.js&func=teardownModule

But we have another instance, failked twice in /testGeolocation/testShareLocation.js:
http://mozmill-daily.blargon7.com/#/functional/failure?app=Firefox&branch=All&platform=All&from=2014-08-15&test=%2FtestGeolocation%2FtestShareLocation.js&func=teardownModule


Report: http://mozmill-daily.blargon7.com/#/functional/report/2561af7d8c83a450772e8c43801d39ee

Comment 2

[Andreea Matei [:AndreeaMatei]]

Happened again with Aurora on mm-ub-1310-64-1:
http://mozmill-daily.blargon7.com/#/functional/report/2561af7d8c83a450772e8c438087abd7

From the log I think it started from the previous test, testGeolocation started with teardown directly:

03:41:27 TEST-START | testGeolocation/testNotNowShareLocation.js | setupTest
03:41:27 TEST-START | testGeolocation/testNotNowShareLocation.js | testNotNowShareLocation
03:41:28 TEST-PASS | testGeolocation/testNotNowShareLocation.js | testNotNowShareLocation
03:41:28 TEST-START | testGeolocation/testNotNowShareLocation.js | teardownTest
03:41:28 TEST-END | testGeolocation/testNotNowShareLocation.js | finished in 960ms
03:41:28 *** WIFI GEO: shutdown called
03:41:29 
03:41:29 (process:29283): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
03:41:33 TEST-START | testGeolocation/testShareLocation.js | teardownModule
03:41:33 ERROR | Test Failure | {
03:41:33   "exception": {
03:41:33     "message": "aModule.tabBrowser is undefined", 
03:41:33     "lineNumber": 34, 
03:41:33     "name": "TypeError", 
03:41:33     "fileName": "resource://mozmill/modules/frame.js -> file:///home/mozauto/jenkins/workspace/mozilla-aurora_functional/data/mozmill-tests/firefox/tests/functional/testGeolocation/testShareLocation.js"
03:41:33   }
03:41:33 }
03:41:33 TEST-UNEXPECTED-FAIL | testGeolocation/testShareLocation.js | teardownModule

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Well, we do not list setupModule and teardownModule as executed as long as they pass. So setupModule could still have been run. It's very suspicious, and hard to tell anything as long as someone cannot reproduce it locally. :/

Comment 4

[Andrei Eftimie]

Same failure in another test /testGeolocation/testShareLocation.js, might be the same underlying issue:
http://mozmill-release.blargon7.com/#/functional/report/2f56cb3a3728c2c47cd1b44f77504a8d

Comment 5

[Andrei Eftimie]

And failed again, 
http://mozmill-release.blargon7.com/#/functional/report/2f982f72826307fed840a3b11cd6f5f6

Same as before:
> 22:55:53 TEST-START | testGeolocation/testNotNowShareLocation.js | setupTest
> 22:55:53 TEST-START | testGeolocation/testNotNowShareLocation.js | testNotNowShareLocation
> 22:55:54 TEST-PASS | testGeolocation/testNotNowShareLocation.js | testNotNowShareLocation
> 22:55:54 TEST-START | testGeolocation/testNotNowShareLocation.js | teardownTest
> 22:55:54 TEST-END | testGeolocation/testNotNowShareLocation.js | finished in 1009ms
> 22:55:55 
> 22:55:55 (process:15209): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
> 22:55:59 TEST-START | testGeolocation/testShareLocation.js | teardownModule
> 22:55:59 ERROR | Test Failure | {
> 22:55:59   "exception": {
> 22:55:59     "message": "aModule.tabBrowser is undefined", 
> 22:55:59     "lineNumber": 34, 
> 22:55:59     "name": "TypeError", 
> 22:55:59     "fileName": "resource://mozmill/modules/frame.js -> file:///home/mozauto/jenkins/workspace/release-mozilla-beta_functional/data/mozmill-tests/firefox/tests/functional/testGeolocation/testShareLocation.js"
> 22:55:59   }
> 22:55:59 }
> 22:55:59 TEST-UNEXPECTED-FAIL | testGeolocation/testShareLocation.js | teardownModule
> 22:55:59 TEST-END | testGeolocation/testShareLocation.js | finished in 6ms

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Please follow the order of started methods here. This is a restart test and teardownModule should not have been executed before `testNotNowShareLocationPersisted` has been run. What I think happened here is that after the restart in teardownTest, Firefox came up again, the test method was not found, and no setupTest has been executed. Surprisingly I can see this in my refactored software update tests.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

See bug 1072266.

Comment 8

[Andreea Matei [:AndreeaMatei]]

This did not happen again, for testSearchAddons there are 3 reports in june when the bug was filed. For testShareLocation I don't see anything else than the one Andrei mentioned. I think this was more related to bug 972912, as I don't see setupModule to have run.

Comment 9

[Mihaela Velimiroviciu (:mihaelav)]

Failing again on Beta (35.0) and Aurora(36.0):
* http://mozmill-release.blargon7.com/#/functional/failure?app=Firefox&branch=All&platform=All&from=2015-01-05&test=%2FtestGeolocation%2FtestShareLocation.js&func=teardownModule
* http://mozmill-daily.blargon7.com/#/functional/failure?app=Firefox&branch=All&platform=All&from=2015-01-0&test=%2FtestGeolocation%2FtestShareLocation.js&func=teardownModule

Comment 10

[Mihaela Velimiroviciu (:mihaelav)]

Many failures in the latest 36 beta:
http://mozmill-release.blargon7.com/#/functional/failure?app=Firefox&branch=36&platform=All&from=2015-01-14&test=%2FtestGeolocation%2FtestShareLocation.js&func=teardownModule

Comment 11

[Andreea Matei [:AndreeaMatei]]

Lets skip this on Linux, seems the only one affected for testShareLocation

Comment 12

[Teodor Druta]

Attachment: skiptstshrlctn.patch

So, because there were many failures lately on beta & ubuntu, we decided we should skip testShareLocation.js on Linux for mozilla-beta branch. Here is the skip patch.

Comment 13

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 8549507 [details] [diff] [review]
skiptstshrlctn.patch

Review of attachment 8549507 [details] [diff] [review]:
-----------------------------------------------------------------

http://hg.mozilla.org/qa/mozmill-tests/rev/00f0b67f6e3d (beta)

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+1]]

In this case please really really work on a reproduction case. This would be so important to get this fixed if it fails a lot.

Comment 15

[Andreea Matei [:AndreeaMatei]]

Transplanted to release too, we had failures with today's 35.0.1:
https://hg.mozilla.org/qa/mozmill-tests/rev/d94d5580c40e (release)

Comment 16

[Daniela Domnici]

Failed 22 times over the weekend with Nightly, Aurora and ESR31 on Ubuntu 14.04 and Ubuntu 14.10.

http://mozmill-daily.blargon7.com/#/functional/failure?app=Firefox&branch=All&platform=All&from=2015-01-23&to=2015-01-26&test=%2FtestGeolocation%2FtestShareLocation.js&func=teardownModule

Comment 17

[Daniela Domnici]

Failed again(11 times) with Nightly(38.0a1) and Aurora(37.0a2).

Reports: http://mozmill-daily.blargon7.com/#/functional/failure?app=Firefox&branch=All&platform=All&from=2015-01-26&to=&test=%2FtestGeolocation%2FtestShareLocation.js&func=teardownModule

Comment 18

[Daniela Domnici]

Failed again over the weekend with Aurora.

Reports: http://mozmill-daily.blargon7.com/#/remote/failure?app=Firefox&branch=All&platform=All&from=2015-02-06&to=2015-02-09&test=%2FtestAddons%2FtestSearchAddons.js&func=teardownModule

Comment 19

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The skip for testShareLocation.js should have been across all OS. It's also failing a lot on OS X now. I will have a skip patch shortly.

Comment 20

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Skip patch landed for all the branches:

https://hg.mozilla.org/qa/mozmill-tests/rev/057ab2bf9d06 (default)
https://hg.mozilla.org/qa/mozmill-tests/rev/fb307758c7f9 (aurora)
https://hg.mozilla.org/qa/mozmill-tests/rev/ebe1c9af4c89 (beta)

Comment 21

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Mozmill tests have been superseded by Marionette tests.