Closed Bug 915648 Opened 7 years ago Closed 7 years ago

Parallelize make export

Categories

(Firefox Build System :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla27

People

(Reporter: glandium, Assigned: glandium)

References

Details

Attachments

(1 file, 2 obsolete files)

No description provided.
Attached patch Parallelize make export (obsolete) — Splinter Review
Attachment #803668 - Flags: review?(gps)
Comment on attachment 803668 [details] [diff] [review]
Parallelize make export

Review of attachment 803668 [details] [diff] [review]:
-----------------------------------------------------------------

I think you should cancel this review and move this patch into bug 907365, as I'm pretty sure the broken webidl dependencies will land before bug 907365. I'll bug khuey IRL right now...
I don't want to be bugged IRL :-(
(In reply to Gregory Szorc [:gps] from comment #2)
> Comment on attachment 803668 [details] [diff] [review]
> Parallelize make export
> 
> Review of attachment 803668 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> I think you should cancel this review and move this patch into bug 907365,
> as I'm pretty sure the broken webidl dependencies will land before bug
> 907365.

Unfortunately, bug 907365 is not the only problem. I already got three different issues with make export running in parallel, one of which i addressed with explicit dependencies defined in Makefile.in, and the other two that i just got now and haven't investigated.

For the record:
 0:52.13 Traceback (most recent call last):
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 180, in <module>
 0:52.13     sys.exit(_nsinstall_internal(argv[1:]))
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 145, in _nsinstall_internal
 0:52.13     copy_all_entries(args, target)
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 106, in copy_all_entries
 0:52.13     handleTarget(e, dest)
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 135, in handleTarget
 0:52.13     shutil.copy(srcpath, targetpath)
 0:52.13   File "/usr/lib/python2.7/shutil.py", line 119, in copy
 0:52.13 Creating /home/mh/build/obj-x86_64-unknown-linux-gnu/security/build/../../dist/private/dbm
 0:52.15 cd src; /usr/bin/make private_export
 0:52.16 There are no private exports.
 0:52.16 There are no private exports.
 0:52.16 cd softoken; /usr/bin/make private_export
 0:52.17 Traceback (most recent call last):
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 180, in <module>
 0:52.17     sys.exit(_nsinstall_internal(argv[1:]))
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 145, in _nsinstall_internal
 0:52.17     copy_all_entries(args, target)
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 106, in copy_all_entries
 0:52.17     handleTarget(e, dest)
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 135, in handleTarget
 0:52.17     shutil.copy(srcpath, targetpath)
 0:52.17   File "/usr/lib/python2.7/shutil.py", line 119, in copy
 0:52.17 There are no private exports.
 0:52.17 cd legacydb; /usr/bin/make private_export
 0:52.17     copyfile(src, dst)
 0:52.17   File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
 0:52.17     with open(src, 'rb') as fsrc:
 0:52.17 IOError: [Errno 13] Permission denied: u'/home/mh/mozilla-central/mfbt/MathAlgorithms.h'
 0:52.18     copyfile(src, dst)
 0:52.18   File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
 0:52.18     with open(src, 'rb') as fsrc:
 0:52.18 IOError: [Errno 13] Permission denied: u'/home/mh/mozilla-central/mfbt/MemoryReporting.h'

and:

 4:35.80 Generating WebIDL bindings
 4:35.93 Traceback (most recent call last):
 4:35.93   File "/home/mh/mozilla-central/config/pythonpath.py", line 56, in <module>
 4:35.93     main(sys.argv[1:])
 4:35.93   File "/home/mh/mozilla-central/config/pythonpath.py", line 48, in main
 4:35.93     execfile(script, frozenglobals)
 4:35.93   File "/home/mh/mozilla-central/dom/bindings/BindingGen.py", line 98, in <module>
 4:35.93     main()
 4:35.93   File "/home/mh/mozilla-central/dom/bindings/BindingGen.py", line 58, in main
 4:35.93     config = cPickle.load(f)
 4:35.93 ValueError: unregistered extension code 264
 4:35.97 make[5]: *** [.BindingGen] Error 1
Attached patch Parallelize make export (obsolete) — Splinter Review
I prefer to keep this separated.
Attachment #806963 - Flags: review?(gps)
Attachment #803668 - Attachment is obsolete: true
Attachment #803668 - Flags: review?(gps)
Comment on attachment 806963 [details] [diff] [review]
Parallelize make export

Review of attachment 806963 [details] [diff] [review]:
-----------------------------------------------------------------

::: Makefile.in
@@ +206,5 @@
>  endif
> +
> +js/xpconnect/src/export: config/makefiles/precompile/export
> +accessible/src/xpcom/export: config/makefiles/precompile/export
> +js/src/export: mfbt/export

Shouldn't this be behind an ifdef MOZ_PSEUDO_DERECURSE?
Attachment #806963 - Flags: review?(gps) → review+
Maybe I'm being over-cautious, but i prefer to provide an easy escape route.
Attachment #807564 - Flags: review?(gps)
Attachment #806963 - Attachment is obsolete: true
Comment on attachment 807564 [details] [diff] [review]
Parallelize make export

Review of attachment 807564 [details] [diff] [review]:
-----------------------------------------------------------------

Clever.
Attachment #807564 - Flags: review?(gps) → review+
glandium, I see you added

DEFINES['FOO'] = '1'

to the toplevel moz.build file in 9cc7634555c3. That's not needed, right?
(In reply to :Benjamin Peterson from comment #12)
> glandium, I see you added
> 
> DEFINES['FOO'] = '1'
> 
> to the toplevel moz.build file in 9cc7634555c3. That's not needed, right?

Erf. No.
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.