1712817 - Improve the header file logic in ./mach vendor

Status: RESOLVED FIXED

(Developer Infrastructure :: Mach Vendor & Updatebot, defect, P3)

Description

[Tom Ritter [:tjr]]

If you take a May 25 2021 checkout of m-c (e.g. 0c8d79edfc9a27b8eab7a71c7addeb223845275f ) and try to update libdav1d to a revision as of that date (e.g. c54add020492e3cca0da5ab90fa69c92ba496384) it will add ../../../third_party/dav1d/src/arm/asm-offsets.h to SOURCES. It should go into EXPORTS.

We should detect if a file added is a header instead of a source file, and if so only consider adding it to EXPORTS rather than SOURCES.

Comment 1

[Tom Ritter [:tjr]]

Attachment: Bug 1712817: Support adding header files to EXPORTS in mozbuild rewriting r?jewilde

Depends on D120480

Comment 2

[Tom Ritter [:tjr]]

Attachment: Bug 1712817: Add tests for mozbuild file editing r?jewilde

Depends on D120481

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d04e9ef5a778
Support adding header files to EXPORTS in mozbuild rewriting r=jewilde
https://hg.mozilla.org/integration/autoland/rev/292e32e520c1
Add tests for mozbuild file editing r=jewilde

Comment 4

[Narcis Beleuzu [:NarcisB]]

Backed out for Linting failure

Backout link: https://hg.mozilla.org/integration/autoland/rev/9eb10837f685f6a3762bb5db311b6b2006a0de37
Log link: https://treeherder.mozilla.org/logviewer?job_id=347171348&repo=autoland&lineNumber=118

Also:
f8 - https://treeherder.mozilla.org/logviewer?job_id=347171371&repo=autoland&lineNumber=546
mbu - https://treeherder.mozilla.org/logviewer?job_id=347171380&repo=autoland&lineNumber=1591

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/752e9fe54b3f
Support adding header files to EXPORTS in mozbuild rewriting r=jewilde
https://hg.mozilla.org/integration/autoland/rev/d4b681a741bd
Add tests for mozbuild file editing r=jewilde

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/182ae38519ef
fix black lint failures. a=lint-fix CLOSED TREE

Comment 7

[Atila Butkovits]

Backed out for causing failures at test_rewrite_mozbuild.py.

Backout link: https://hg.mozilla.org/integration/autoland/rev/5f3b07899482fb2b15dfd026e225f9e0c24c2d02

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&selectedTaskRun=fLhLtw-xTtqBUZPcHDfSEw.0&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=d4b681a741bd44b710aa6f19fa3cdb90fb00c50d

Failure log: https://treeherder.mozilla.org/logviewer?job_id=347266623&repo=autoland&lineNumber=1156

Comment 8

[Tom Ritter [:tjr]]

The tests are failing because I am now more relying on ast_get_source_segment in the test and it is failing because while I copy/pasted the function from python 3.8 into 3.7 it has probably been silently failing because end_lindno is not present in Python 3.7

I might be back to trying to get Python 3.8 into the Updatebot Docker image. That would make sure Updatebot works better but not solve the problem for the tests which will run on the normal build machines which are 3.7

Comment 9

[Mike Hommey [:glandium]]

We should detect if a file added is a header instead of a source file, and if so only consider adding it to EXPORTS rather than SOURCES.

Actually... we shouldn't blindly add things to EXPORTS. EXPORTS is an unnecessary step in most cases. In most cases, the few places that need the headers would add a LOCAL_INCLUDES.

Comment 10

[Tom Ritter [:tjr]]

Attachment: Bug 1712817: Do not add newly added header files to EXPORTS automatically r?jewilde

Depends on D121379

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4dbaf9a28c11
Support adding header files to EXPORTS in mozbuild rewriting r=jewilde
https://hg.mozilla.org/integration/autoland/rev/a1b4ab355164
Add tests for mozbuild file editing r=jewilde
https://hg.mozilla.org/integration/autoland/rev/3e1f61fc1927
Do not add newly added header files to EXPORTS automatically r=jewilde

Comment 12

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/4dbaf9a28c11
https://hg.mozilla.org/mozilla-central/rev/a1b4ab355164
https://hg.mozilla.org/mozilla-central/rev/3e1f61fc1927

Comment 13

[Tom Ritter [:tjr]]

Re-assigning bugs to the new Mach Vendor and Updatebot Component. Guid for this change: ade7b229-ec85-41f8-bafe-9d4451bfeed1

Comment 14

[Tom Ritter [:tjr]]

A mass updatebot change inadvertently added a security group to bugs. Removing it. Guid for this change: 09d63db4-11a1-4820-a398-d718f112f885