Linux SpiderMonkey builds failing to find an hg to clone with

RESOLVED FIXED

Status

Release Engineering
General Automation
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: philor, Assigned: sfink)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [spidermonkey])

Attachments

(4 attachments, 1 obsolete attachment)

(Reporter)

Description

5 years ago
e.g. https://tbpl.mozilla.org/php/getParsedLog.php?id=15537136&tree=Mozilla-Inbound
bld-centos5-32-vmw-017

Upon execvpe hg ['hg', 'clone', 'http://hg.mozilla.org/build/tools', 'scripts'] in environment id 171984116
:Traceback (most recent call last):
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.6/site-packages/twisted/internet/process.py", line 414, in _fork
    executable, args, environment)
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.6/site-packages/twisted/internet/process.py", line 460, in _execChild
    os.execvpe(executable, args, environment)
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.6/os.py", line 353, in execvpe
    _execvpe(file, args, env)
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.6/os.py", line 380, in _execvpe
    func(fullname, *argrest)
OSError: [Errno 2] No such file or directory
hg is at /tools/python/bin/hg on them, and they don't have the /tools/python27/bin or /tools/python27-mercurial which are on the path. Probably affects that whole class of machines - bld-centos5-64-vmw-NNN - but just the spidermonkey builds.
(Reporter)

Comment 2

5 years ago
Apparently a change last Thursday - https://secure.pub.build.mozilla.org/buildapi/recent/bld-centos5-64-vmw-001?numbuilds=125 says it did a green spidermonkey job at 7am, and a red one at 8:46pm.
(Reporter)

Comment 3

5 years ago
They get their env from http://hg.mozilla.org/build/buildbot-configs/annotate/f9bd9eadca7a/mozilla/config.py#l862, and http://hg.mozilla.org/build/buildbot-configs/diff/18d1ebb7084a/mozilla/config.py gave that env a path that works for the mock slaves.
Blocks: 772446
We should switch these over to use the mock slaves as well.
(Assignee)

Comment 5

5 years ago
(In reply to Chris AtLee [:catlee] from comment #4)
> We should switch these over to use the mock slaves as well.

What would this take? On the face of it, it seems like I'd just need to do something very similar to what I'm doing over in bug 775355, then copy over the mock-related arguments. That runs and produces reasonable-looking stuff, but I don't know how to properly test without some test slaves to kick around. I haven't attempted that yet; not sure how doable it is.
(Assignee)

Comment 6

5 years ago
Created attachment 670944 [details] [diff] [review]
Patch 3/3: Copy the mock-related arguments into the spidermonkey project builders' ScriptFactories

I'll put that question in the form of a set of patches
Attachment #670944 - Flags: review?(catlee)
(Assignee)

Updated

5 years ago
Assignee: nobody → sphink
(Assignee)

Comment 7

5 years ago
Created attachment 670945 [details] [diff] [review]
Patch 2/3: Give the spidermonkey project configs the entire platform config settings

Sorry, these patches are all out of order. I'll update the descriptions with a reasonable order.
Attachment #670945 - Flags: review?(catlee)
(Assignee)

Comment 8

5 years ago
Created attachment 670946 [details] [diff] [review]
Patch 1/3: Share the whole platform config chunks instead of just the env for spidermonkey project configs

This change would need to be made simultaneously to the buildbotcustom stuff
Attachment #670946 - Flags: review?(catlee)
(Assignee)

Updated

5 years ago
Attachment #670945 - Attachment description: Give the spidermonkey project configs the entire platform config settings → Patch 2/3: Give the spidermonkey project configs the entire platform config settings
(Assignee)

Updated

5 years ago
Attachment #670946 - Attachment description: Share the whole platform config chunks instead of just the env for spidermonkey project configs → Patch 1/3: Share the whole platform config chunks instead of just the env for spidermonkey project configs
(Assignee)

Updated

5 years ago
Attachment #670944 - Attachment description: Copy the mock-related arguments into the spidermonkey project builders' ScriptFactories → Patch 3/3: Copy the mock-related arguments into the spidermonkey project builders' ScriptFactories
(Assignee)

Updated

5 years ago
Blocks: 775355

Updated

5 years ago
Attachment #670946 - Flags: review?(catlee) → review+

Updated

5 years ago
Attachment #670945 - Flags: review?(catlee) → review+
Comment on attachment 670944 [details] [diff] [review]
Patch 3/3: Copy the mock-related arguments into the spidermonkey project builders' ScriptFactories

Review of attachment 670944 [details] [diff] [review]:
-----------------------------------------------------------------

::: misc.py
@@ +2345,5 @@
> +            factory_platform_args = [ 'use_mock',
> +                                      'mock_target',
> +                                      'mock_packages',
> +                                      'mock_copyin_files' ]
> +            factory_kwargs = dict([ (a, pf.get(a, None)) for a in factory_platform_args ])

Can you change this to only set factory_kwargs that appear in pf? e.g.

for a in factory_platform_args:
   if a in pf:
      factory_kwargs[a] = pf[a]

This makes the diff vs. the current configuration easier to read. As this patch stands you're changing the default value use_mock to None instead of False.
Attachment #670944 - Flags: review?(catlee) → review-
(Assignee)

Comment 10

5 years ago
Created attachment 674282 [details] [diff] [review]
Copy the mock-related arguments into the spidermonkey project builders' ScriptFactories

Ok, thanks.
Attachment #674282 - Flags: review?(catlee)
(Assignee)

Updated

5 years ago
Attachment #670944 - Attachment is obsolete: true
Blocks: 805577
Comment on attachment 674282 [details] [diff] [review]
Copy the mock-related arguments into the spidermonkey project builders' ScriptFactories

Review of attachment 674282 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good. Please add a line to initialize factory_kwargs if required.

::: misc.py
@@ +2350,5 @@
>          for variant in variants:
> +            factory_platform_args = [ 'use_mock',
> +                                      'mock_target',
> +                                      'mock_packages',
> +                                      'mock_copyin_files' ]

I think you need a "factory_kwargs = {}" here
Attachment #674282 - Flags: review?(catlee) → review+
(Assignee)

Comment 12

5 years ago
(In reply to Chris AtLee [:catlee] from comment #11)
> ::: misc.py
> @@ +2350,5 @@
> >          for variant in variants:
> > +            factory_platform_args = [ 'use_mock',
> > +                                      'mock_target',
> > +                                      'mock_packages',
> > +                                      'mock_copyin_files' ]
> 
> I think you need a "factory_kwargs = {}" here

And I think you're right. Sorry about that; I should have at least done rudimentary testing. I always mess up python scoping.

I have now done my basic testing (loaded the config and dumped things out, and ran the master to make sure it started up.)
(Assignee)

Updated

5 years ago
Attachment #670946 - Flags: checked-in+
(Assignee)

Updated

5 years ago
Attachment #674282 - Flags: checked-in+
(Assignee)

Comment 13

5 years ago
http://hg.mozilla.org/build/buildbot-configs/rev/d999fc274b17
(Assignee)

Comment 14

5 years ago
http://hg.mozilla.org/build/buildbotcustom/rev/2104f40b645c
http://hg.mozilla.org/build/buildbotcustom/rev/ef41cdf3ef3b
(Assignee)

Comment 15

5 years ago
Created attachment 675719 [details] [diff] [review]
spidermonkey builder names need to be more unique.
(Assignee)

Updated

5 years ago
Attachment #670945 - Flags: checked-in+
(Assignee)

Updated

5 years ago
Attachment #675719 - Flags: checked-in+
In production.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
This may have caused bug 807313.
Huh, I'm seeing this again on both:
https://tbpl.mozilla.org/?tree=Mozilla-Inbound&noignore=1&jobname=spidermonkey
https://tbpl.mozilla.org/?tree=Try&noignore=1&jobname=spidermonkey
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.