1354850 - Use updater to run general app update service tests and remove gonk test code

Status: RESOLVED FIXED

(Toolkit :: Application Update, enhancement)

Description

[Robert Strong (they/them - no direct email)]

The general service tests should use the same method to run the general tests as the non service tests. This will simplify the test code and make it so these tests won't fail if there are changes in the application that fail. There will still be end to end testing using the non general app update tests that will still fail if there are changes in the application that fail so we shouldn't miss those failures and this should make it clear when that is the case.

Note: this should help with the recent huge increase of app update test timeouts that showed up.

Comment 1

[Robert Strong (they/them - no direct email)]

Attachment: Patch in progress

Pushed to try
https://treeherder.mozilla.org/#/jobs?repo=try&revision=9d05833735b26c2acf0697e7f6e0924ce6a915ec

Comment 2

[Robert Strong (they/them - no direct email)]

Most of the recent test failures have been for the service which launches firefox.exe and this patch might very well fix the majority of those.

Comment 3

[Robert Strong (they/them - no direct email)]

Attachment: Patch

Try push
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f252264fee193ac7170924a35aa3bb6a8a3d1ccf

Win8 x64 Debug try push with --tag appupdate. This way the tests runs are fast so the cost of a bunch of retriggers is reduced. Hope the try syntax works as it is supposed to.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0cf9605909256e7f3380c6dfffdc0fdf110a5c67

Comment 4

[Robert Strong (they/them - no direct email)]

Comment on attachment 8856208 [details] [diff] [review]
Patch

Tests looking fine. I'm also retriggering this try run a bunch of times to see if I can get it to fail
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0cf9605909256e7f3380c6dfffdc0fdf110a5c67

Comment 5

[Robert Strong (they/them - no direct email)]

Note to self: all but one of the recent test failures have been on Win8 x64 debug. Perhaps just disable the tests on Win8 x64 debug since these popped up out of nowhere and it may be due to slow build systems.

Comment 6

[Robert Strong (they/them - no direct email)]

The one case where it failed on Win 7 debug it also failed in the same place as the Win8 x64 debug failures.

launching C:\\windows\\System32\\cmd.exe /D /Q /C c:\\slave\\test\\build\\tests\\xpcshell\\tests\\toolkit\\mozapps\\update\\tests\\unit_service_updater\\marFileInUseSuccessCompleteSvc_win\\dir.app\\firefox.exe -no-remote -test-process-updates -dump-args c:\\slave\\test\\build\\tests\\xpcshell\\tests\\toolkit\\mozapps\\update\\tests\\unit_service_updater\\marFileInUseSuccessCompleteSvc_win_app_args_log >nul

This patch makes it so the majority of tests don't call firefox.exe so I'm going to hold off on disabling these on Win8 x64 debug.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/9e567506bd73
Test only - Bug 1354850 - Use updater to run general app update service tests and remove gonk test code. r=mhowell

Comment 8

[Robert Strong (they/them - no direct email)]

After going through the logs for the failing tests I think this patch will fix all of the bugs I just added to the blocking list.

It won't fix bug 1354290 or bug 1354746 since those launch firefox.exe where the tests appear to be having an intermittent problem.

Comment 9

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/9e567506bd73

Comment 10

[Robert Strong (they/them - no direct email)]

Attachment: patch for beta - test only

Pushed to mozilla-beta
https://hg.mozilla.org/releases/mozilla-beta/rev/40a36afd6f60931900d12226b4c4bd7a60e5a343

Comment 11

[Robert Strong (they/them - no direct email)]

Pushed to mozilla-esr52
https://hg.mozilla.org/releases/mozilla-esr52/rev/3a49fe1696720a9586e37ab5d37d886987820b46

Comment 12

[Robert Strong (they/them - no direct email)]

Attachment: patch for esr52 - test only