Closed Bug 675084 Opened 13 years ago Closed 13 years ago

PGO builds with pymake are broken because pymake chokes on a builtin with shell metacharacters

Categories

(Firefox Build System :: General, defect)

x86
Windows 7
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: RyanVM, Assigned: khuey)

References

Details

Attachments

(1 file)

libs_tier_nspr
c:\mozbuild\mozilla-central\nsprpub\config\rules.mk:384:0:native command 'pymake.builtins rm -f *.pgd *.gcda': shell metacharacter '*' in command line
c:\mozbuild\mozilla-central\nsprpub\config\rules.mk:189:0: command 'c:/mozbuild/python/python.exe c:/mozbuild/mozilla-central/build/pymake/pymake/../make.py -C config export' failed, return code 2
<all>: Found error
<all>: Found error
c:\mozbuild\mozilla-central\objdir-fx\config\nspr\Makefile:64:0: command 'c:/mozbuild/python/python.exe c:/mozbuild/mozilla-central/build/pymake/pymake/../make.py -C ../../nsprpub PROFILE_USE_CFLAGS="-GL -wd4624 -wd4952"' failed, return code 2
c:\mozbuild\mozilla-central\config\rules.mk:710:0: command 'c:/mozbuild/python/python.exe c:/mozbuild/mozilla-central/build/pymake/pymake/../make.py -C config/nspr libs' failed, return code 2
c:\mozbuild\mozilla-central\config\rules.mk:721:0: command 'c:/mozbuild/python/python.exe c:/mozbuild/mozilla-central/build/pymake/pymake/../make.py libs_tier_nspr' failed, return code 2
c:\mozbuild\mozilla-central\config\rules.mk:671:0: command 'c:/mozbuild/python/python.exe c:/mozbuild/mozilla-central/build/pymake/pymake/../make.py  tier_nspr' failed, return code 2
c:\mozbuild\mozilla-central\client.mk:349:0: command 'c:/mozbuild/python/python.exe c:/mozbuild/mozilla-central/build/pymake/pymake/../make.py -s -j 4 -C c:/mozbuild/mozilla-central/objdir-fx' failed, return code 2
c:\mozbuild\mozilla-central\client.mk:212:0: command 'c:/mozbuild/python/python.exe c:/mozbuild/mozilla-central/build/pymake/pymake/../make.py -f c:/mozbuild/mozilla-central/client.mk realbuild MOZ_PROFILE_GENERATE=1' failed, return code 2
c:\mozbuild\mozilla-central\client.mk:174:0: command 'c:/mozbuild/python/python.exe c:/mozbuild/mozilla-central/build/pymake/pymake/../make.py -f c:/mozbuild/mozilla-central/client.mk profiledbuild' failed, return code 2
This is a pymake bug.  We should be falling back to the shell rm here.
Assignee: wtc → nobody
Component: NSPR → Build Config
Product: NSPR → Core
QA Contact: nspr → build-config
Version: 4.8.9 → unspecified
Hrmph. We should just support shell globbing in pymake native commands.
Assignee: nobody → khuey
Summary: Bug 665978 broke PGO pymake builds → PGO builds with pymake are broken because pymake chokes on a builtin with shell metacharacters
Comment on attachment 554925 [details] [diff] [review]
Implement shell globbing for pymake

Review of attachment 554925 [details] [diff] [review]:
-----------------------------------------------------------------

::: pymake/process.py
@@ +41,5 @@
> +    """
> +    globbedargs = []
> +    for arg in args:
> +        if _needsglob.search(arg):
> +            globbedargs.extend(glob.glob(os.path.join(cwd, arg)))

It seems unfortunate to run this regex for every single argument. Can you either a) run it once over all args or b) find out if glob is smart enough to not iterate over the entire directory unless there are wildcard characters?

::: tests/native-command-shell-glob.mk
@@ +8,4 @@
>  	$(RM) shell-glob-test/*.txt
> +	$(RM) shell-glob-test/?.foo
> +	rmdir shell-glob-test
> +	@echo TEST-PASS

I was confused as to why this was patching an existing test. You landed the test in the pymake repo first!
Attachment #554925 - Flags: review?(ted.mielczarek) → review+
So the problem with globbing everything is that things that don't exist will vanish from the command line.  Do you really think the regex matching will be slow enough for this to matter?
Ah, that's a good point, you're right. Too bad that both your code and glob will regex check each argument. :-/
The pymake change in this bug broke builds on OS X and potentially other platforms. See bug 683686.
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: