1635755 - Perma TEST-UNEXPECTED-FAIL | target.generated-files.tar.gz/target.generated-files.tar/ipc/ipdl/BlobTypes.cpp differs. See the diff.html or diff.txt artifact

Status: RESOLVED FIXED

(Core :: IPC, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: rgurzau [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=301016931&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AXdYVYV7R_WqZWaiM3F5Ow/runs/0/artifacts/public/logs/live_backing.log

---

[task 2020-05-06T11:31:24.256Z] |########################## | 97% xpcom/ds/nsGkAtomConsts.h ETA: 0:00:00
[task 2020-05-06T11:31:24.259Z] |########################## | 97% xpcom/ds/nsGkAtomList.h ETA: 0:00:00
[task 2020-05-06T11:31:24.277Z] |########################## | 98% …parser/xpidl/xpidllex.py ETA: 0:00:00
[task 2020-05-06T11:31:24.308Z] |########################## | 98% …arser/xpidl/xpidlyacc.py ETA: 0:00:00
[task 2020-05-06T11:31:25.854Z] |########################## | 98% …lect/xptinfo/xptdata.cpp ETA: 0:00:00
[task 2020-05-06T11:31:25.854Z] |########################## | 99% …s/gtest/dafsa_test_1.inc ETA: 0:00:00
[task 2020-05-06T11:31:25.855Z] |########################## | 99% xpcom/xpcom-config.h ETA: 0:00:00
[task 2020-05-06T11:31:25.856Z] |########################## | 99% xpcom/xpcom-private.h ETA: 0:00:00
[task 2020-05-06T11:31:25.858Z] |###########################| 100% ETA: 0:00:00
[task 2020-05-06T11:31:25.858Z] |###########################| 100% ETA: 0:00:00
[task 2020-05-06T11:31:25.858Z] |###########################| 100% ETA: 0:00:00
[task 2020-05-06T11:31:25.858Z] |###########################| 100% ETA: 0:00:00
[task 2020-05-06T11:32:52.179Z] |###########################| 100% Time: 0:00:26
[task 2020-05-06T11:32:52.179Z] + report_error diff
[task 2020-05-06T11:32:52.179Z] + python3
[task 2020-05-06T11:32:52.234Z] TEST-UNEXPECTED-FAIL | target.generated-files.tar.gz/target.generated-files.tar/ipc/ipdl/BlobTypes.cpp differs. See the diff.html or diff.txt artifact
[task 2020-05-06T11:32:52.234Z] TEST-UNEXPECTED-FAIL | target.generated-files.tar.gz/target.generated-files.tar/ipc/ipdl/CacheTypes.cpp differs. See the diff.html or diff.txt artifact
[task 2020-05-06T11:32:52.234Z] TEST-UNEXPECTED-FAIL | target.generated-files.tar.gz/target.generated-files.tar/ipc/ipdl/ChannelInfo.cpp differs. See the diff.html or diff.txt artifact
[task 2020-05-06T11:32:52.234Z] TEST-UNEXPECTED-FAIL | target.generated-files.tar.gz/target.generated-files.tar/ipc/ipdl/ClientIPCTypes.cpp differs. See the diff.html or diff.txt artifact
[task 2020-05-06T11:32:52.234Z] TEST-UNEXPECTED-FAIL | target.generated-files.tar.gz/target.generated-files.tar/ipc/ipdl/DOMTypes.cpp differs. See the diff.html or diff.txt artifact

Comment 1

[Natalia Csoregi [:nataliaCs]]

Ricky, is there any chance this was caused by changes in Bug 1632916 ?
Could you take a few mins to check?
Thank you.

Comment 2

[BugBot [:suhaib / :marco]]

Set release status flags based on info from the regressing bug 1632916

Comment 3

[Ricky Stewart]

Python 3's built-in hashing of strings is non-deterministic for some godforsaken reason, so we'll have to replace it with a deterministic hash. I'll get a patch in shortly.

Comment 4

[Ricky Stewart]

Attachment: Bug 1635755 - Part 1: Emit JSON data in deterministic order in `jsonxpt`

xptcodegen.py does a lot of stuff with indexing into the contents of a dict generated by the json library. This is going to yield unpredictable results on Python 3.6 and below where dictionary iteration order is nondeterministic between subsequent invocations of a python3 process. Fix this nondeterminism by loading the JSON data into an OrderedDict, as the documentation suggests.

Comment 5

[Ricky Stewart]

Attachment: Bug 1635755 - Part 2: Use deterministic hashing over types in `ipdl` file generation

The hashes of certain strings is used indirectly in generating IDL C++ files. Before, we were using the hash() function to generate these hashes, which in Python 3 is non-deterministic over subsequent python3 processes, causing bugs like bug 1635755. Instead, use a specific, deterministic hash to avoid spurious diff failures.

The md5 hash function isn't completely cryptographically secure but the security of the hashes isn't important for us in this case since we're just using them as a per-string identifier. We could use a more robust hash function but there may be performance implications from doing so.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/220a0d9666fe
Part 2: Use deterministic hashing over types in `ipdl` file generation r=nika

Comment 8

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/220a0d9666fe

Comment 9

[Cosmin Sabou [:CosminS]]

From the initial list of failures only this was left after today's merge: https://treeherder.mozilla.org/logviewer.html#?job_id=301536129&repo=mozilla-central

TEST-UNEXPECTED-FAIL | target.generated-files.tar.gz/target.generated-files.tar/xpcom/reflect/xptinfo/xptdata.cpp differs. See the diff.html or diff.txt artifact

Comment 10

[Intermittent Failures Robot]

21 failures in 5276 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 21

Platform breakdown:

• diff: 21

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1635755&startday=2020-05-04&endday=2020-05-10&tree=all

Comment 11

[Ricky Stewart]

That's expected, that file is handled by revision D74443 which isn't landed yet.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8932d3f0ed98
Part 1: Emit JSON data in deterministic order in `jsonxpt` r=nika

Comment 13

[Ricky Stewart]

We should be good after the this patch makes it to central.

Comment 14

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/8932d3f0ed98

Comment 15

[Intermittent Failures Robot]

4 failures in 5488 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 4

Platform breakdown:

• diff: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1635755&startday=2020-05-11&endday=2020-05-17&tree=all