Closed Bug 485390 Opened 14 years ago Closed 14 years ago

Race creating JAR files in JarMaker.py with highly-parallel builds

Categories

(Firefox Build System :: General, defect, P2)

Product:
Firefox Build System
Component:
General
Platform:
x86
macOS
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: benjamin, Assigned: benjamin)

References

Details

(Keywords: fixed1.9.1)

Attachments

(2 files, 1 obsolete file)

Fix race in JarMaker.py creating a .jar file that doesn't exist yet, rev. 1
2.53 KB, patch
ted
: review+
Details | Diff | Splinter Review
Script I used for testing
436 bytes, text/plain
Details 
utils.lockFile doesn't always lock the file: O_EXCL seems to only take effect as long as you're holding the file descriptor open.

This puts a lie to the expected meaning of O_EXCL, but I don't care. It works.

I think this should block 1.9.1 because with highly parallel builds it's not hard to end up with JAR files which are incomplete because multiple JarMakers are trying to write at the same time.
Flags: blocking1.9.1+
Attached patch Keep the fd open, rev. 1 (obsolete) — — Splinter Review 

        Attachment #369540 -
        Flags: review?(ted.mielczarek)

    
    

    
  
Comment on attachment 369540 [details] [diff] [review]
Keep the fd open, rev. 1

Good catch!

        Attachment #369540 -
        Flags: review?(ted.mielczarek) → review+

    
  
Blocks: 485412

    
    

    
  
I pushed this yesterday but it didn't actually fix the problem, so I'm backing it out. I found the race elsewhere (in the JarMaker logic which tests whether the .jar file exists)

    
    

    
  


  

        Attachment #369540 -
        Attachment is obsolete: true

        Attachment #370193 -
        Flags: review?(ted.mielczarek)

        Attachment #370193 -
        Flags: review?(ted.mielczarek) → review+

    
    

    
  
Comment on attachment 370193 [details] [diff] [review]
Fix race in JarMaker.py creating a .jar file that doesn't exist yet, rev. 1

I shudder in fear at all the other race conditions waiting in our build system.

    
    

    
  
http://hg.mozilla.org/mozilla-central/rev/e1465ef7add6
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED

    
  
Priority: -- → P2
Summary: utils.lockFile doesn't → Race creating JAR files in JarMaker.py with highly-parallel builds

    
    

    
  


  
This is the shell script I used for race-testing, saved for posterity

    
  
Product: Core → Firefox Build System

          You need to log in
          before you can comment on or make changes to this bug.