1825141 - Perma [tier 2] Diffoscope firefox/browser/defaults/preferences/firefox.js differs. See the diff.html or diff.txt artifact

Status: RESOLVED FIXED

(Toolkit :: Telemetry, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: nfay [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=410567835&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/W23JTl57SSSCA5JcgW_sCQ/runs/0/artifacts/public/logs/live_backing.log

---

[task 2023-03-28T21:03:56.558Z] ++ dirname /builds/worker/bin/get_and_diffoscope
[task 2023-03-28T21:03:56.559Z] + /builds/worker/bin/report_error diff
[task 2023-03-28T21:03:56.568Z] TEST-UNEXPECTED-FAIL | firefox/browser/defaults/preferences/firefox.js differs. See the diff.html or diff.txt artifact
[task 2023-03-28T21:03:56.569Z] + fail
[task 2023-03-28T21:03:56.569Z] + exit 1
[taskcluster 2023-03-28 21:03:56.843Z] === Task Finished ===
[taskcluster 2023-03-28 21:03:58.998Z] Unsuccessful task run with exit code: 1 completed in 90.346 seconds

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

:chutten, since you are the author of the regressor, bug 1820241, could you take a look?

For more information, please visit auto_nag documentation.

Comment 2

[Chris H-C :chutten]

It's the addition of this line:

 	1061 	pref("telemetry.fog.artifact_build",·true);

Which is added when there's MOZ_ARTIFACT_BUILDS, which I guess the diffoscope attempt build runs, but the reference build didn't?

:glandium, as the person I see at the intersection of both diffoscope and artifact builds, any advice?

Comment 3

[Mike Hommey [:glandium]]

So what's going on here is that the windows arm64 builds are an aggregate of a pure arm64 build and a i686 build, and that's done via artifact builds. I don't remember why we went with artifact builds for that, rather than the same-ish thing as universal builds for mac. Well, for one thing, the universal builds for mac on taskcluster are much newer...
Anyways, what that means is that MOZ_ARTIFACT_BUILDS is set for those builds.
And what we do is that for good measure, we check that the aggregated (artifact) build and the pure arm64 build are not diverging (which has caught problems in the past, so it's not like we're doing this for no reason).
Unfortunately, bug 1820241 did introduce a divergence...

I'd say the easy cop out here would be to not set the pref on arm64 windows builds, artifact builds or not. We don't support building on arm64 windows at the moment, and it's not like running arm64 windows binaries will work on x86_64 windows.

Another cop out would be to add another exception like https://searchfox.org/mozilla-central/rev/42747dfd314e4c939dc7c33a13e1a2fddf4926fc/taskcluster/ci/diffoscope/kind.yml#49-52

Comment 4

[Nick Alexander :nalexander [he/him]]

Mild preference for baking this into kind.yml -- that's the first place to look for surprises when comparing/diffoscoping binaries.

Comment 5

[Chris H-C :chutten]

Attachment: Bug 1825141 - Ignore firefox.js' telemetry.fog.artifact_build line for diffoscope builds r?glandium!

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/891cd6589fa0
Ignore firefox.js' telemetry.fog.artifact_build line for diffoscope builds r=glandium

Comment 7

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/891cd6589fa0

Comment 8

[Intermittent Failures Robot]

41 failures in 3863 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #19 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 35
• mozilla-central: 6

Platform and build breakdown:

• diff: 41
  • opt: 41

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1825141&startday=2023-03-27&endday=2023-04-02&tree=all