race condition with lock files in build-list.pl (components.list.lck: Permission denied, all-test-dirs.list.lck: Permission denied)

RESOLVED FIXED in mozilla1.9.3a1

Status

Firefox Build System
General
RESOLVED FIXED
9 years ago
4 months ago

People

(Reporter: ted, Assigned: ted)

Tracking

Trunk
mozilla1.9.3a1
x86
Windows XP

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Assignee)

Description

9 years ago
I just hit two of these in a row, maybe I'm unlucky. I'm guessing this is something similar to bug 522804, but in mozLock.pm, which is what build-list.pl uses for locking. Maybe I should just rewrite build-list.pl in Python and use lockFile from utils.py (since I fixed the race conditions there already).

Comment 1

9 years ago
Don't know why I'm cc'ed on this bug - did you mean Neil Deakin?
(Assignee)

Comment 2

9 years ago
Wrong Neil. :)

Comment 3

9 years ago
My original complaint to Ted referred to components.list.lck but for comparison, I just got this on SFU with a -j2 build:
>/usr/local/bin/perl -I/dev/fs/C/hg/comm/mozilla/config /dev/fs/C/hg/comm/mozilla/config/build-list.pl \
>  ../../../mozilla/_tests/xpcshell/all-test-dirs.list \
>  test_imap/unit
>/usr/local/bin/perl -I/dev/fs/C/hg/comm/mozilla/config /dev/fs/C/hg/comm/mozilla/config/build-list.pl \
>  ../../../mozilla/_tests/xpcshell/all-test-dirs.list \
>  test_addbook/unit
>/dev/fs/C/hg/mozilla/_tests/xpcshell/all-test-dirs.list.lck: Device busy
>gmake[6]: *** [libs] Error 16
>gmake[6]: Leaving directory `/dev/fs/C/hg/mailnews/addrbook/test'
>gmake[5]: *** [test_libs] Error 2
>gmake[5]: *** Waiting for unfinished jobs....
(Device busy is the SFU error when trying to delete a file that is open.
Text file busy is the SFU error when trying to delete a running executable.)
(Assignee)

Comment 4

9 years ago
I rewrote build-list.pl in Python (it's only about 30 lines) since I already fixed the race condition bugs in the lockFile implementation in config/utils.py.

I'll have a patch up shortly.
Assignee: nobody → ted.mielczarek
(Assignee)

Comment 5

9 years ago
Created attachment 415144 [details] [diff] [review]
rewrite build-list.pl in Python

The implementation is pretty simple, I've included some basic unit tests as well. Most of this patch is just a search and replace from calls to build-list.pl to calls to buildlist.py. That bit in toolkit/xre/Makefile.in referencing embedding/browser/gtk/src/Makefile.in no longer seems to be relevant, so I've removed it.

Seemed easier to write this than to fumble through fixing mozLock.pm.
Attachment #415144 - Flags: review?(benjamin)
(Assignee)

Updated

9 years ago
Status: NEW → ASSIGNED

Comment 6

9 years ago
Comment on attachment 415144 [details] [diff] [review]
rewrite build-list.pl in Python

Awesome. If you're going to remove build-list.pl right away, make sure comm-central guys know (or you can leave it around for a bit while they update).
Attachment #415144 - Flags: review?(benjamin) → review+
(Assignee)

Comment 7

9 years ago
Good call, CCing some more c-c folks. The removal is really just a search-and-replace, so the patch should be trivial.
Created attachment 415337 [details] [diff] [review]
c-c

That world with branches instead of ifdefs for other people's branches, where it would be a simple search/replace? I want to live there.
Attachment #415337 - Flags: review?(bugzilla)

Comment 9

9 years ago
ifdef MOZILLA_1_9_2_BRANCH
BUILDLIST = $(PERL) -I$(MOZILLA_DIR)/config $(MOZILLA_DIR)/config/build-list.pl
else
BUILDLIST = $(PYTHON) $(MOZILLA_DIR)/config/buildlist.py
endif
(Assignee)

Comment 10

9 years ago
I can just leave build-list.pl there until you stop the madness, if that would make your life easier.
(Assignee)

Comment 11

9 years ago
(but you have to promise to remove it for me later.)
O, my future daughter! O, my future ducats!

Using BUILDLIST would be prettier now, but only if some unknown person in the unknowable future will realize that they should completely undo it while dropping 1_9_2 ifdefs, rather than just dropping the build-list.pl version and leaving our copy of rules.mk oddly different. My version looks like crap now, but ensures that no matter how blindly someone removes the ifdefs, we'll end up where we want to be, with as little difference from mozilla-central as possible. (Or, I guess we could define BUILDLIST in mozilla-central's copy, rather than having eight repeats of "$(PYTHON) $(MOZILLA_DIR)/config/buildlist.py".)
(Assignee)

Comment 13

9 years ago
Pushed to m-c:
http://hg.mozilla.org/mozilla-central/rev/83b704c74e5f
Status: ASSIGNED → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → FIXED
Attachment #415337 - Flags: review?(bugzilla) → review+
Target Milestone: --- → mozilla1.9.3a1
Flags: in-testsuite-
(Assignee)

Comment 15

9 years ago
I totally checked in tests with the Python rewrite.
Flags: in-testsuite- → in-testsuite+

Updated

4 months ago
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.