Closed Bug 915648 Opened 10 years ago Closed 10 years ago

Parallelize make export

Categories

(Firefox Build System :: General, defect)

Product:
Firefox Build System
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
mozilla27

People

(Reporter: glandium, Assigned: glandium)

References

Details

Attachments

(1 file, 2 obsolete files)

Parallelize make export
6.66 KB, patch
gps
: review+
Details | Diff | Splinter Review 
      No description provided.
Attached patch Parallelize make export (obsolete) — — Splinter Review 

        Attachment #803668 -
        Flags: review?(gps)

    
    

    
  
Comment on attachment 803668 [details] [diff] [review]
Parallelize make export

Review of attachment 803668 [details] [diff] [review]:
-----------------------------------------------------------------

I think you should cancel this review and move this patch into bug 907365, as I'm pretty sure the broken webidl dependencies will land before bug 907365. I'll bug khuey IRL right now...

    
    

    
  
I don't want to be bugged IRL :-(

    
    

    
  
(In reply to Gregory Szorc [:gps] from comment #2)
> Comment on attachment 803668 [details] [diff] [review]
> Parallelize make export
> 
> Review of attachment 803668 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> I think you should cancel this review and move this patch into bug 907365,
> as I'm pretty sure the broken webidl dependencies will land before bug
> 907365.

Unfortunately, bug 907365 is not the only problem. I already got three different issues with make export running in parallel, one of which i addressed with explicit dependencies defined in Makefile.in, and the other two that i just got now and haven't investigated.

For the record:
 0:52.13 Traceback (most recent call last):
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 180, in <module>
 0:52.13     sys.exit(_nsinstall_internal(argv[1:]))
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 145, in _nsinstall_internal
 0:52.13     copy_all_entries(args, target)
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 106, in copy_all_entries
 0:52.13     handleTarget(e, dest)
 0:52.13   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 135, in handleTarget
 0:52.13     shutil.copy(srcpath, targetpath)
 0:52.13   File "/usr/lib/python2.7/shutil.py", line 119, in copy
 0:52.13 Creating /home/mh/build/obj-x86_64-unknown-linux-gnu/security/build/../../dist/private/dbm
 0:52.15 cd src; /usr/bin/make private_export
 0:52.16 There are no private exports.
 0:52.16 There are no private exports.
 0:52.16 cd softoken; /usr/bin/make private_export
 0:52.17 Traceback (most recent call last):
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 180, in <module>
 0:52.17     sys.exit(_nsinstall_internal(argv[1:]))
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 145, in _nsinstall_internal
 0:52.17     copy_all_entries(args, target)
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 106, in copy_all_entries
 0:52.17     handleTarget(e, dest)
 0:52.17   File "/home/mh/mozilla-central/js/src/config/nsinstall.py", line 135, in handleTarget
 0:52.17     shutil.copy(srcpath, targetpath)
 0:52.17   File "/usr/lib/python2.7/shutil.py", line 119, in copy
 0:52.17 There are no private exports.
 0:52.17 cd legacydb; /usr/bin/make private_export
 0:52.17     copyfile(src, dst)
 0:52.17   File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
 0:52.17     with open(src, 'rb') as fsrc:
 0:52.17 IOError: [Errno 13] Permission denied: u'/home/mh/mozilla-central/mfbt/MathAlgorithms.h'
 0:52.18     copyfile(src, dst)
 0:52.18   File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
 0:52.18     with open(src, 'rb') as fsrc:
 0:52.18 IOError: [Errno 13] Permission denied: u'/home/mh/mozilla-central/mfbt/MemoryReporting.h'

and:

 4:35.80 Generating WebIDL bindings
 4:35.93 Traceback (most recent call last):
 4:35.93   File "/home/mh/mozilla-central/config/pythonpath.py", line 56, in <module>
 4:35.93     main(sys.argv[1:])
 4:35.93   File "/home/mh/mozilla-central/config/pythonpath.py", line 48, in main
 4:35.93     execfile(script, frozenglobals)
 4:35.93   File "/home/mh/mozilla-central/dom/bindings/BindingGen.py", line 98, in <module>
 4:35.93     main()
 4:35.93   File "/home/mh/mozilla-central/dom/bindings/BindingGen.py", line 58, in main
 4:35.93     config = cPickle.load(f)
 4:35.93 ValueError: unregistered extension code 264
 4:35.97 make[5]: *** [.BindingGen] Error 1

    
    

    
  

      
      
      
      

        Attached patch
          
          
        Parallelize make export (obsolete)
        — 
          — Splinter Review
      

    


  
I prefer to keep this separated.

        Attachment #806963 -
        Flags: review?(gps)

    
  

        Attachment #803668 -
        Attachment is obsolete: true

        Attachment #803668 -
        Flags: review?(gps)

    
    

    
  
Comment on attachment 806963 [details] [diff] [review]
Parallelize make export

Review of attachment 806963 [details] [diff] [review]:
-----------------------------------------------------------------

::: Makefile.in
@@ +206,5 @@
>  endif
> +
> +js/xpconnect/src/export: config/makefiles/precompile/export
> +accessible/src/xpcom/export: config/makefiles/precompile/export
> +js/src/export: mfbt/export

Shouldn't this be behind an ifdef MOZ_PSEUDO_DERECURSE?

        Attachment #806963 -
        Flags: review?(gps) → review+

    
    

    
  


  
Maybe I'm being over-cautious, but i prefer to provide an easy escape route.

        Attachment #807564 -
        Flags: review?(gps)

    
  

        Attachment #806963 -
        Attachment is obsolete: true

    
    

    
  
Comment on attachment 807564 [details] [diff] [review]
Parallelize make export

Review of attachment 807564 [details] [diff] [review]:
-----------------------------------------------------------------

Clever.

        Attachment #807564 -
        Flags: review?(gps) → review+

    
    

    
  
glandium, I see you added

DEFINES['FOO'] = '1'

to the toplevel moz.build file in 9cc7634555c3. That's not needed, right?

    
    

    
  
(In reply to :Benjamin Peterson from comment #12)
> glandium, I see you added
> 
> DEFINES['FOO'] = '1'
> 
> to the toplevel moz.build file in 9cc7634555c3. That's not needed, right?

Erf. No.

    
  
Product: Core → Firefox Build System

          You need to log in
          before you can comment on or make changes to this bug.