Parallelize make export

RESOLVED FIXED in mozilla27

Status

()

Core
Build Config
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: glandium, Assigned: glandium)

Tracking

Trunk
mozilla27
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 2 obsolete attachments)

Comment hidden (empty)
(Assignee)

Comment 1

4 years ago
Created attachment 803668 [details] [diff] [review]
Parallelize make export
Attachment #803668 - Flags: review?(gps)
Comment on attachment 803668 [details] [diff] [review]
Parallelize make export

Review of attachment 803668 [details] [diff] [review]:
-----------------------------------------------------------------

I think you should cancel this review and move this patch into bug 907365, as I'm pretty sure the broken webidl dependencies will land before bug 907365. I'll bug khuey IRL right now...
I don't want to be bugged IRL :-(
(Assignee)

Comment 4

4 years ago
(In reply to Gregory Szorc [:gps] from comment #2)
> Comment on attachment 803668 [details] [diff] [review]
> Parallelize make export
> 
> Review of attachment 803668 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> I think you should cancel this review and move this patch into bug 907365,
> as I'm pretty sure the broken webidl dependencies will land before bug
> 907365.

Unfortunately, bug 907365 is not the only problem. I already got three different issues with make export running in parallel, one of which i addressed with explicit dependencies defined in Makefile.in, and the other two that i just got now and haven't investigated.

For the record:
 0:52.13 Traceback (most recent call last):
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 180, in <module>
 0:52.13     sys.exit(_nsinstall_internal(argv[1:]))
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 145, in _nsinstall_internal
 0:52.13     copy_all_entries(args, target)
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 106, in copy_all_entries
 0:52.13     handleTarget(e, dest)
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 135, in handleTarget
 0:52.13     shutil.copy(srcpath, targetpath)
 0:52.13   File "/usr/lib/python2.7/shutil.py", line 119, in copy
 0:52.13 Creating /home/mh/build/obj-x86_64-unknown-linux-gnu/security/build/../../dist/private/dbm
 0:52.15 cd src; /usr/bin/make private_export
 0:52.16 There are no private exports.
 0:52.16 There are no private exports.
 0:52.16 cd softoken; /usr/bin/make private_export
 0:52.17 Traceback (most recent call last):
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 180, in <module>
 0:52.17     sys.exit(_nsinstall_internal(argv[1:]))
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 145, in _nsinstall_internal
 0:52.17     copy_all_entries(args, target)
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 106, in copy_all_entries
 0:52.17     handleTarget(e, dest)
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 135, in handleTarget
 0:52.17     shutil.copy(srcpath, targetpath)
 0:52.17   File "/usr/lib/python2.7/shutil.py", line 119, in copy
 0:52.17 There are no private exports.
 0:52.17 cd legacydb; /usr/bin/make private_export
 0:52.17     copyfile(src, dst)
 0:52.17   File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
 0:52.17     with open(src, 'rb') as fsrc:
 0:52.17 IOError: [Errno 13] Permission denied: u'/home/mh/mozilla-central/mfbt/MathAlgorithms.h'
 0:52.18     copyfile(src, dst)
 0:52.18   File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
 0:52.18     with open(src, 'rb') as fsrc:
 0:52.18 IOError: [Errno 13] Permission denied: u'/home/mh/mozilla-central/mfbt/MemoryReporting.h'

and:

 4:35.80 Generating WebIDL bindings
 4:35.93 Traceback (most recent call last):
 4:35.93   File "/home/mh/mozilla-central/config/pythonpath.py", line 56, in <module>
 4:35.93     main(sys.argv[1:])
 4:35.93   File "/home/mh/mozilla-central/config/pythonpath.py", line 48, in main
 4:35.93     execfile(script, frozenglobals)
 4:35.93   File "/home/mh/mozilla-central/dom/bindings/BindingGen.py", line 98, in <module>
 4:35.93     main()
 4:35.93   File "/home/mh/mozilla-central/dom/bindings/BindingGen.py", line 58, in main
 4:35.93     config = cPickle.load(f)
 4:35.93 ValueError: unregistered extension code 264
 4:35.97 make[5]: *** [.BindingGen] Error 1
(Assignee)

Comment 5

4 years ago
Created attachment 806963 [details] [diff] [review]
Parallelize make export

I prefer to keep this separated.
Attachment #806963 - Flags: review?(gps)
(Assignee)

Updated

4 years ago
Attachment #803668 - Attachment is obsolete: true
Attachment #803668 - Flags: review?(gps)
Comment on attachment 806963 [details] [diff] [review]
Parallelize make export

Review of attachment 806963 [details] [diff] [review]:
-----------------------------------------------------------------

::: Makefile.in
@@ +206,5 @@
>  endif
> +
> +js/xpconnect/src/export: config/makefiles/precompile/export
> +accessible/src/xpcom/export: config/makefiles/precompile/export
> +js/src/export: mfbt/export

Shouldn't this be behind an ifdef MOZ_PSEUDO_DERECURSE?
Attachment #806963 - Flags: review?(gps) → review+
(Assignee)

Comment 7

4 years ago
Created attachment 807564 [details] [diff] [review]
Parallelize make export

Maybe I'm being over-cautious, but i prefer to provide an easy escape route.
Attachment #807564 - Flags: review?(gps)
(Assignee)

Updated

4 years ago
Attachment #806963 - Attachment is obsolete: true
Comment on attachment 807564 [details] [diff] [review]
Parallelize make export

Review of attachment 807564 [details] [diff] [review]:
-----------------------------------------------------------------

Clever.
Attachment #807564 - Flags: review?(gps) → review+
(Assignee)

Comment 9

4 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/9cc7634555c3
(Assignee)

Comment 10

4 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/9eba1b4d798f
https://hg.mozilla.org/mozilla-central/rev/9cc7634555c3
https://hg.mozilla.org/mozilla-central/rev/9eba1b4d798f
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla27

Comment 12

4 years ago
glandium, I see you added

DEFINES['FOO'] = '1'

to the toplevel moz.build file in 9cc7634555c3. That's not needed, right?
(Assignee)

Comment 13

4 years ago
(In reply to :Benjamin Peterson from comment #12)
> glandium, I see you added
> 
> DEFINES['FOO'] = '1'
> 
> to the toplevel moz.build file in 9cc7634555c3. That's not needed, right?

Erf. No.

Comment 14

4 years ago
Okay. I killed it.
https://hg.mozilla.org/integration/mozilla-inbound/rev/c87681163457
https://hg.mozilla.org/mozilla-central/rev/c87681163457
You need to log in before you can comment on or make changes to this bug.