1894138 - Perma Windows shippable xpcshell toolkit/mozapps/update/tests/unit_service_updater/checkUpdaterSigSvc.js | run_test - [run_test : 48] the maintenance service exit value should be 0 - 1 == 0

Status: RESOLVED FIXED

(Toolkit :: Application Update, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ctuns [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=456348207&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LvQUBI3mRuORrr5mC-L6hQ/runs/0/artifacts/public/logs/live_backing.log

---

[task 2024-04-29T23:50:43.910Z] 23:50:43     INFO -  TEST-START | toolkit/mozapps/update/tests/unit_service_updater/checkUpdaterSigSvc.js
[task 2024-04-29T23:50:44.161Z] 23:50:44  WARNING -  TEST-UNEXPECTED-FAIL | toolkit/mozapps/update/tests/unit_service_updater/checkUpdaterSigSvc.js | xpcshell return code: 0
[task 2024-04-29T23:50:44.162Z] 23:50:44     INFO -  TEST-INFO took 251ms
[task 2024-04-29T23:50:44.162Z] 23:50:44     INFO -  >>>>>>>
[task 2024-04-29T23:50:44.163Z] 23:50:44     INFO -  (xpcshell/head.js) | test MAIN run_test pending (1)
[task 2024-04-29T23:50:44.163Z] 23:50:44     INFO -  "23:50:44:058 | TEST-INFO | D:/task_171443222893968/build/tests/xpcshell/tests/toolkit/mozapps/update/tests/unit_service_updater/checkUpdaterSigSvc.js | [run_test : 22] Launching maintenance service bin: D:\\task_171443222893968\\build\\application\\firefox\\maintenanceservice.exe to check updater: D:\\task_171443222893968\\build\\application\\firefox\\updater.exe signature."
[task 2024-04-29T23:50:44.164Z] 23:50:44  WARNING -  TEST-UNEXPECTED-FAIL | toolkit/mozapps/update/tests/unit_service_updater/checkUpdaterSigSvc.js | run_test - [run_test : 48] the maintenance service exit value should be 0 - 1 == 0
[task 2024-04-29T23:50:44.164Z] 23:50:44     INFO -  D:/task_171443222893968/build/tests/xpcshell/tests/toolkit/mozapps/update/tests/unit_service_updater/checkUpdaterSigSvc.js:run_test:48
[task 2024-04-29T23:50:44.165Z] 23:50:44     INFO -  D:\task_171443222893968\build\tests\xpcshell\head.js:_execute_test:590
[task 2024-04-29T23:50:44.165Z] 23:50:44     INFO -  -e:null:1
[task 2024-04-29T23:50:44.165Z] 23:50:44     INFO -  exiting test
[task 2024-04-29T23:50:44.165Z] 23:50:44     INFO -  <<<<<<<
[task 2024-04-29T23:50:44.166Z] 23:50:44     INFO -  TEST-START | netwerk/test/unit/test_alt-data_simple.js

Comment 1

[Cristian Tuns]

Hi, can you please take a look at this?
I didn't want to back this out https://hg.mozilla.org/mozilla-central/rev/470ddb7d30be3b3d549123ee8bfffe1e9b87b1d5 because I am not sure if this is causing the failures. This is very frequent on central and it would require a backout or a fix.
https://treeherder.mozilla.org/jobs?repo=mozilla-central&group_state=expanded&resultStatus=testfailed%2Cbusted%2Cexception%2Cusercancel&searchStr=windows%2Cxpcshell&revision=650dda9187436a12c66c2e81441e68e1d481e164&selectedTaskRun=dAHBaEaJQGaOvNR2Oi9iUQ.0
Thanks!

Comment 2

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This looks unrelated to Will's changes.

Ben, could this be a regression from bug 1889299? Needinfo for you because jcristau has a public holiday tomorrow.

Comment 3

[William Durand [:willdurand]]

I was submitting a comment when :aryx left the comment above.

FTR, I pushed to Try two times:

• once with the latest tip at the time (that had the changes for Bug 1893116): https://treeherder.mozilla.org/jobs?repo=try&revision=462897dd87faf01feb90b2ca312ee0ea602fdf05
• once with the changes reverted: https://treeherder.mozilla.org/jobs?repo=try&revision=ef5c2e00e1602dbb609302deb5a3692581b89dd7

In both cases, test-windows11-64-2009-shippable-qr/opt-xpcshell-spi-nw-4 passed.

Additionally, I ran this test locally (and it isn't exactly trivial). The test ran successfully with and without the changes for Bug 1893116.

Comment 4

[[e-mail] (:bhearsum)]

It certainly seems likely that the authenticode cert change is the issue. I think this is ultimately failing due to https://searchfox.org/mozilla-central/source/toolkit/mozapps/update/common/certificatecheck.cpp#25 returning an error, which seems to inspect the cert that the updater(?) is signed with.

We know for a fact that one of the fields in the cert changed from "Mountain View" to "San Francisco", so if this is indeed the case, this is a real problem (and not just in unit tests).

bytesized: can you confirm that the maintenance service is impacted by a change to the authenticode cert metadata? If it is, does this hork updates entirely, or can updates still happen (just not via the maintenance service)?

We should consider backing out the authenticode change and https://phabricator.services.mozilla.com/D208972 (to avoid re-busting MSIX builds) in the meantime.

Comment 5

[Robin Steuber (they/them) [:bytesized]]

(In reply to [e-mail] (:bhearsum) from comment #4)

bytesized: can you confirm that the maintenance service is impacted by a change to the authenticode cert metadata?

I'm sorry, I don't know quite this deeply how this stuff works. I can see that we fail if the issuer is different than expected, but I don't really understand what all is encompassed in that check.

If it is, does this hork updates entirely, or can updates still happen (just not via the maintenance service)?

So, I believe that there are generally three certificate checks in the update process: (1) The updater checking that the MAR is properly signed, (2) the maintenance service checking that the updater is properly signed, and (3) the updater checking that the post update binary is signed properly. Since we are talking about authenticode, I imagine that the first is not a concern but I'm not an expert in this domain so I'm not 100% positive. If the second check fails, this will prevent us from using the maintenance service but we should be able to fall back to showing a UAC elevation prompt. Potentially the third could cause issues, but shouldn't prevent update from happening.

Comment 6

[[e-mail] (:bhearsum)]

Thanks Robin - it's good to know that this is unlikely to be a serious update issue.

I'm going to back out the authenticode cert change out of an abundance of caution for now while we investigate this further. We have until mid June to make the switch over - so plenty of time still.

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1889299

:jcristau, since you are the author of the regressor, bug 1889299, could you take a look?

For more information, please visit BugBot documentation.

Comment 8

[Cosmin Sabou [:CosminS]]

Should be fixed by this backout https://hg.mozilla.org/mozilla-central/rev/b7a1a8a3af7f8cb576e533dc6537f14dab105c55

Comment 9

[Intermittent Failures Robot]

16 failures in 772 pushes (0.021 failures/push) were associated with this bug yesterday.

Repository breakdown:

• comm-central: 6
• mozilla-central: 10

Platform and build breakdown:

• windows11-32-2009-shippable-qr: 7
  • opt: 7
• windows11-64-2009-shippable-qr: 7
  • opt: 7
• windows10-64-2009-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1894138&startday=2024-04-30&endday=2024-04-30&tree=all

Comment 10

[Intermittent Failures Robot]

29 failures in 4059 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 23
• comm-central: 6

Platform and build breakdown:

• windows10-64-2009-shippable-qr: 3
  • opt: 3
• windows11-32-2009-shippable-qr: 11
  • opt: 11
• windows11-64-2009-shippable-qr: 15
  • opt: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1894138&startday=2024-04-29&endday=2024-05-05&tree=all