Perma [tier2] TEST-UNEXPECTED-FAIL | target.generated-files.tar.gz/target.generated-files.tar/xpcom/idl-parser/xpidl/xpidllex.py differs. See the diff.html or diff.txt artifact
Categories
(Firefox Build System :: Task Configuration, defect, P5)
Tracking
(firefox78 fixed)
| Tracking | Status | |
|---|---|---|
| firefox78 | --- | fixed |
People
(Reporter: intermittent-bug-filer, Assigned: rstewart)
References
Details
(Keywords: intermittent-failure)
Attachments
(1 file)
Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=299396003&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/dlqzxdWxRxGsVvkXveieQg/runs/0/artifacts/public/logs/live_backing.log
task 2020-04-25T11:45:53.403Z] |########################## | 98% …arser/xpidl/xpidlyacc.py ETA: 0:00:00
[task 2020-04-25T11:45:53.409Z] |########################## | 98% …lect/xptinfo/xptdata.cpp ETA: 0:00:00
[task 2020-04-25T11:45:53.410Z] |########################## | 99% …s/gtest/dafsa_test_1.inc ETA: 0:00:00
[task 2020-04-25T11:45:53.411Z] |########################## | 99% xpcom/xpcom-config.h ETA: 0:00:00
[task 2020-04-25T11:45:53.412Z] |########################## | 99% xpcom/xpcom-private.h ETA: 0:00:00
[task 2020-04-25T11:45:53.414Z] |###########################| 100% ETA: 0:00:00
[task 2020-04-25T11:45:53.414Z] |###########################| 100% ETA: 0:00:00
[task 2020-04-25T11:45:53.414Z] |###########################| 100% ETA: 0:00:00
[task 2020-04-25T11:45:53.414Z] |###########################| 100% ETA: 0:00:00
[task 2020-04-25T11:45:56.991Z] |###########################| 100% Time: 0:00:07
[task 2020-04-25T11:45:56.991Z] + report_error diff
[task 2020-04-25T11:45:56.991Z] + python3
[task 2020-04-25T11:45:57.001Z] TEST-UNEXPECTED-FAIL | target.generated-files.tar.gz/target.generated-files.tar/xpcom/idl-parser/xpidl/xpidllex.py differs. See the diff.html or diff.txt artifact
[task 2020-04-25T11:45:57.001Z] TEST-UNEXPECTED-FAIL | target.generated-files.tar.gz/target.generated-files.tar/xpcom/idl-parser/xpidl/xpidlyacc.py differs. See the diff.html or diff.txt artifact
[task 2020-04-25T11:45:57.002Z] + fail
[task 2020-04-25T11:45:57.002Z] + exit 1
[taskcluster 2020-04-25 11:45:57.368Z] === Task Finished ===
[taskcluster 2020-04-25 11:45:58.129Z] Unsuccessful task run with exit code: 1 completed in 32.362 seconds
|
||
Comment 1•4 years ago
|
||
Mike this showed up again on mozilla central starting with this merge: https://hg.mozilla.org/mozilla-central/pushloghtml?changeset=d8a8178627c4fab05f1eb46ee40f55db207c7c18
Can you please take a look?
| Comment hidden (Intermittent Failures Robot) |
|
||
Comment 3•4 years ago
|
||
Probably bug 1621451 or bug 1621361
|
Assignee | |
Comment 4•4 years ago
|
||
Ah yeah, so the GeneratedFile template doesn't handle the py3 argument any more, but it doesn't yell at you if you do use it, so the py2=True in xpcom/idl-parser/xpidl/moz.build was just getting ignored because I forgot about it. ¯_(ツ)_/¯
I haven't been looking very closely at this work so far. Let me take a swing at it.
|
|
||
Updated•4 years ago
|
|
|
Assignee | |
Comment 5•4 years ago
|
||
ply, by design, does not produce reproducible table files; hence bug 1633156. (Note that this was always true, but only became a problem once we switched to Python 3, which has more unpredictable dict iteration order than Python 2.7, at least prior to 3.7.)
In any other circumstance I would consider submitting a patch to ply to fix this, but as of the in-progress version 4.0 of the library, it doesn't even emit this cached data any more, and indeed the latest version of the code doesn't even call open() at all except to do logging or to read the text data to be parsed from stdin. So if we were going to pin our future on ply and upgrade to later versions of the library in the future, we would have to live in a world where ply doesn't generate cached table files for us anyway.
Emitting the cached table files so later build steps can consume them is an "optimization", but it's not clear exactly how much actual value that optimization provides overall. Quoth the CHANGES file from that repository:
PLY no longer writes cached table files. Honestly, the use of
the cached files made more sense when I was developing PLY on
my 200Mhz PC in 2001. It's not as much as an issue now. For small
to medium sized grammars, PLY should be almost instantaneous.
In practice, I have found this to be true; namely, ./mach build pre-export export takes just about as long on my machine after this patch as it did before, and in a try push I performed, there's no noticeable performance regression from applying this patch. In local testing I also found that generating the LALR tables in calls to yacc() takes about 0.01s on my machine generally, and we generate these tables a couple dozen times total over the course of the export tier now. This isn't nothing, but in my opinion it's also not nearly long enough where it would be a concern given how long export already takes.
That CHANGES file also stresses that if caching this data is important, we have the option of doing so via pickle. If and when we decide that re-enabling this optimization is valuable for us, we should take control of this process and perform the generation in such a way that we can guarantee reproducibility.
| Comment hidden (Intermittent Failures Robot) |
|
|
||
Comment 8•4 years ago
|
||
| bugherder | ||
|
||
Updated•4 years ago
|
| Comment hidden (Intermittent Failures Robot) |
Description
•