Closed Bug 1267351 Opened 4 years ago Closed 4 years ago

.hgignore and .gitignore should contain files generated when running reftests

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(firefox48 affected, firefox49 fixed)

RESOLVED FIXED
mozilla49
Tracking Status
firefox48 --- affected
firefox49 --- fixed

People

(Reporter: botond, Assigned: botond)

References

Details

Attachments

(2 files, 1 obsolete file)

When I run reftests, two files are generated in my srcdir which are neither tracked nor ignored by hg:

lextab.py
reftest.log

We should ignore these.
Assignee: nobody → botond
I haven't seen lextab.py, but please add reftest.log to .gitignore as well :)
Comment on attachment 8745004 [details]
MozReview Request: Bug 1267351 - Add lextab.py (generated during a reftest run) to .hgignore and .gitignore. r=glandium

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48769/diff/1-2/
Attachment #8745004 - Attachment description: MozReview Request: 1267351 - Add files generated during a reftest run to .hgignore. r=glandium → MozReview Request: Bug 1267351 - Add files generated during a reftest run to .hgignore and .gitignore. r=glandium
Now with .gitignore as well
Summary: .hgignore should contain files generated when running reftests → .hgignore and .gitignore should contain files generated when running reftests
Comment on attachment 8745004 [details]
MozReview Request: Bug 1267351 - Add lextab.py (generated during a reftest run) to .hgignore and .gitignore. r=glandium

https://reviewboard.mozilla.org/r/48769/#review45693

I don't think blindly ignoring the files is the right thing to do. Firstly, nothing should be creating that lextab.py file in the top source directly. Secondly, the reftest harness shouldn't be saving a log file in the current directory.
Attachment #8745004 - Flags: review?(mh+mozilla)
(In reply to Mike Hommey [:glandium] from comment #5)
> Firstly, nothing should be creating that lextab.py file in the top source directly.

Do you know what's creating it at all?

> Secondly, the reftest harness shouldn't be saving a log file in the current directory.

Where should it put the log file?
(In reply to Botond Ballo [:botond] from comment #6)
> (In reply to Mike Hommey [:glandium] from comment #5)
> > Firstly, nothing should be creating that lextab.py file in the top source directly.
> 
> Do you know what's creating it at all?

Nothing should be afaik. I don't have a tree in a state where it could run reftests, so I can't tell either way. You'll have to dig what's creating it (with e.g. strace, dtruss or whatever is relevant on your platform)

> > Secondly, the reftest harness shouldn't be saving a log file in the current directory.
> 
> Where should it put the log file?

It should probably not output any log file at all.
(In reply to Mike Hommey [:glandium] from comment #7)
> It should probably not output any log file at all.

If it really needs to, somewhere in the objdir, preferably in objdir/.mozbuild
I've traced the creation of lextab.py this far; I don't have time to trace further right now, but perhaps this is enough for someone to say what's going on.

As for reftest.log, inquiries are being made in bug 1034290 comment 59. I'll follow up once we have a resolution there.
(In reply to Botond Ballo [:botond] from comment #9)
> As for reftest.log, inquiries are being made in bug 1034290 comment 59. I'll
> follow up once we have a resolution there.

Generation of the restest.log file during local runs was disabled in bug 1271448.
Here's a complete backtrace for the creation of lextab.py; it's being created by a python package named PLY.

I filed an issue against PLY [1] to investigate further.

[1] https://github.com/dabeaz/ply/issues/96
Attachment #8746237 - Attachment is obsolete: true
(In reply to Botond Ballo [:botond] from comment #11)
> Here's a complete backtrace for the creation of lextab.py; it's being
> created by a python package named PLY.
> 
> I filed an issue against PLY [1] to investigate further.
> 
> [1] https://github.com/dabeaz/ply/issues/96

Ok, so it looks like this is file created for performance purposes: it contains a lexer table, which is generated the first time it's needed, and stored so it doesn't need to be generated for subsequent invocations.

It would be nice to have this created in the object directory rather than the source directory, and PLY supports being passed a directory where to write the file, but it's being invoked pretty deep in the stack (i.e. across several third-party packages), so I don't think we can pass in the object directory.

Gregory, do you have any idea what we should do about this file? Is its creation a bug in some component of the stack (see the attachment "Complete backtrace for creation of lextab.py" on this bug), or should we just live with it being created in the source directory and add it to .hgignore / .gitignore?
Flags: needinfo?(gps)
That is an ugly stack. Easiest to just ignore the file and keep a bug open to fix it later.
Flags: needinfo?(gps)
Comment on attachment 8745004 [details]
MozReview Request: Bug 1267351 - Add lextab.py (generated during a reftest run) to .hgignore and .gitignore. r=glandium

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/48769/diff/2-3/
Attachment #8745004 - Attachment description: MozReview Request: Bug 1267351 - Add files generated during a reftest run to .hgignore and .gitignore. r=glandium → MozReview Request: Bug 1267351 - Add lextab.py (generated during a reftest run) to .hgignore and .gitignore. r=glandium
Attachment #8745004 - Flags: review?(mh+mozilla)
^ As per comment 13.
Comment on attachment 8745004 [details]
MozReview Request: Bug 1267351 - Add lextab.py (generated during a reftest run) to .hgignore and .gitignore. r=glandium

https://reviewboard.mozilla.org/r/48769/#review53868

Please file a followup to avoid the file being created in the first place (or making it created somewhere else)
Attachment #8745004 - Flags: review?(mh+mozilla) → review+
Pushed by bballo@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7f15db5b8c9e
Add lextab.py (generated during a reftest run) to .hgignore and .gitignore. r=glandium DONTBUILD
See Also: → 1277621
(In reply to Mike Hommey [:glandium] from comment #16)
> Please file a followup to avoid the file being created in the first place
> (or making it created somewhere else)

Filed bug 1277621.
https://hg.mozilla.org/mozilla-central/rev/7f15db5b8c9e
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla49
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.