1291985 - DON'T USE WinXP debug only TEST-UNEXPECTED-TIMEOUT | toolkit/mozapps/update/tests/unit_service_updater/bootstrapSvc.js | Test timed out - followed by numerous other update test failures on WinXP debug only

Status: RESOLVED FIXED

(Toolkit :: Application Update, defect, P3)

Description

[Treeherder Bug Filer]

Filed by: philringnalda [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=4559942&repo=mozilla-central

http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-central-win32-debug/1470258289/mozilla-central_xp_ix-debug_test-xpcshell-bm119-tests1-windows-build54.txt.gz

Comment 1

[Intermittent Failures Robot]

7 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 4
* mozilla-central: 2
* autoland: 1

Platform breakdown:
* windowsxp: 6
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1291985&startday=2016-08-01&endday=2016-08-07&tree=all

Comment 2

[Robert Strong (they/them - no direct email)]

Bug 1287176 will make it so the parallel tests use a separate directory when they are retried which should get rid of the NS_ERROR_FILE_ACCESS_DENIED error which doesn't provide any insight as to why the test actually fails. A different error should be reported if it still fails along with a new bug filed.

Comment 3

[Intermittent Failures Robot]

14 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 7
* fx-team: 4
* mozilla-inbound: 3

Platform breakdown:
* windowsxp: 14

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1291985&startday=2016-08-08&endday=2016-08-14&tree=all

Comment 4

[Robert Strong (they/them - no direct email)]

The isProcessRunning check always returns true for this failure and it appears that all of the tests that call isProcessRunning also fail when bootstrapSvc.js fails. This could be an intermittent bug in XP or a difference in the build systems when using the code in isProcessRunning and I'll try to find a more reliable way to accomplish what isProcessRunning accomplishes to fix this.

Comment 5

[Robert Strong (they/them - no direct email)]

Attachment: patch - fix tests that run in parallel

Matt, I mistakenly added the isProcessRunning call to the parallel tests. It shouldn't be called for these tests since it runs in parallel and tests that run at the same time will give a false positive. This won't fix this bug entirely but it should make it so the parallel tests that fail in this instance succeed. It will also give me confirmation as to whether the call to isProcessRunning is returning a false positive for the sequential service tests.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4d707b973192
Remove failing check from tests that run in parallel. r=mhowell

Comment 7

[Robert Strong (they/them - no direct email)]

I went ahead and pushed this test only change to mozilla-aurora
https://hg.mozilla.org/releases/mozilla-aurora/rev/07189ee916253fe9692528551ad1b96bc6c1bccc

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/4d707b973192

Comment 9

[Robert Strong (they/them - no direct email)]

Attachment: patch rev1 - disable on XP debug

Matt, this appears to be specific to XP debug only so instead of spending time on trying to figure out why I'm opting to disable these tests on XP debug.

Comment 10

[Robert Strong (they/them - no direct email)]

This should just work but those are famous last words so here is a try push
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5a321fce10e5

Comment 11

[Robert Strong (they/them - no direct email)]

Comment on attachment 8784177 [details] [diff] [review]
patch rev1 - disable on XP debug

Looks like the build system is having issues. I'll push to try again soonish.

Comment 12

[Robert Strong (they/them - no direct email)]

Yep, bug 1297547 is making the builds fail

Comment 13

[Robert Strong (they/them - no direct email)]

try push
https://treeherder.mozilla.org/#/jobs?repo=try&revision=9324516b5a6c

Comment 14

[Robert Strong (they/them - no direct email)]

Since all of the tests specified didn't run I went ahead and pushed this to oak
https://treeherder.mozilla.org/#/jobs?repo=oak&revision=189f55e63199

Comment 15

[Robert Strong (they/them - no direct email)]

Attachment: patch rev2 - disable on XP debug

Comment 16

[Robert Strong (they/them - no direct email)]

Comment on attachment 8784542 [details] [diff] [review]
patch rev2 - disable on XP debug

I verified that the affected tests ran on all platforms except for Win XP debug so this should be good to go.
https://treeherder.mozilla.org/#/jobs?repo=oak&revision=189f55e63199&selectedJob=216459

Comment 17

[Molly Howell (she/her) (no longer active)]

Comment on attachment 8784542 [details] [diff] [review]
patch rev2 - disable on XP debug

Review of attachment 8784542 [details] [diff] [review]:
-----------------------------------------------------------------

Most of the XP jobs are still pending (!), but xpcshell on opt and debug have passed, so I think we're good.

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/9338a982d9ae
disable updater xpcshell tests on XP debug to prevent intermittent failures due to Process32FirstW failures on debug. r=mhowell

Comment 19

[Robert Strong (they/them - no direct email)]

Pushed test only fix to mozilla-aurora
https://hg.mozilla.org/releases/mozilla-aurora/rev/458d5f0a55bb

Comment 20

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/9338a982d9ae

Comment 23

[Intermittent Failures Robot]

6 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 2
* mozilla-aurora: 2
* mozilla-inbound: 1
* autoland: 1

Platform breakdown:
* windowsxp: 6

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1291985&startday=2016-08-22&endday=2016-08-28&tree=all

Comment 24

[Robert Strong (they/them - no direct email)]

Both patches (test only changes) pushed to mozilla-esr45
https://hg.mozilla.org/releases/mozilla-esr45/rev/8e2c74a48ae2
https://hg.mozilla.org/releases/mozilla-esr45/rev/5f10ad071721