821097 - Get spidermonkey rootanalysis builds running on try, under mock

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Steve Fink [:sfink] [:s:]]

All of the sad and crazy stuff for getting these going. These now work in my test environment. (The test itself fails, but for a valid reason.)

Comment 1

[Steve Fink [:sfink] [:s:]]

Attachment: Pass through some try checkout options and switch to the current gcc version.

The spidermonkey.sh script is used by a ScriptFactory, and is responsible for checking out its source repo. The proper fix would be to refactor buildbotcustom to add a type of scripted job that has the checkout already waiting for it, but with the possibility of mock and/or try being involved, it looked like it would be a pretty big project (or involve lots of code duplication). I'm going for the cowardly option of passing the needed information through to the script and continuing to do the checkout (and other tasks) within the script.

Comment 2

[Steve Fink [:sfink] [:s:]]

Attachment: Use the spidermonkey slaves passed in as a parameter rather than pulling them from the platform_config

This patch is stolen from bug 810374, which I'm going to close out. This is for ensuring that spidermonkey builds use the right slaves (try vs production)

Comment 3

[Steve Fink [:sfink] [:s:]]

Attachment: Pass through configuration data to spidermonkey jobs that need it for mock and/or try builds

This patch passes through environment variables (which allows running inside mock) and base_mirror_urls/base_bundle_urls (which allows pulling from try).

Comment 4

[Steve Fink [:sfink] [:s:]]

Attachment: Re-enable spidermonkey builds

This just turns spidermonkey try builds back on. They won't actually work without the other patches in this bug.

Comment 5

[Steve Fink [:sfink] [:s:]]

Attachment: Pass in the appropriate slaves and configuration data to the spidermonkey builds

The spidermonkey builds need various bits of config data passed in. This is one way of doing that.

test-masters.sh passes with these changes.

Comment 6

[Steve Fink [:sfink] [:s:]]

Attachment: Pass in the appropriate slaves and configuration data to the spidermonkey builds

This was is a little simpler and more general.

Comment 7

[[e-mail] (:bhearsum)]

Comment on attachment 691570 [details] [diff] [review]
Pass through some try checkout options and switch to the current gcc version.

Review of attachment 691570 [details] [diff] [review]:
-----------------------------------------------------------------

::: scripts/spidermonkey_builds/spidermonkey.sh
@@ +92,5 @@
>      NSPR64="--enable-64bit"
>  fi
>  
>  if [ "$OSTYPE" = "linux-gnu" ]; then
> +    GCCDIR=/tools/gcc-4.5-0moz3

This seems like it should be set in or passed down from buildbot-configs, since it can be different on different branches. r=me with that changed.

Comment 8

[[e-mail] (:bhearsum)]

Comment on attachment 691572 [details] [diff] [review]
Use the spidermonkey slaves passed in as a parameter rather than pulling them from the platform_config

Review of attachment 691572 [details] [diff] [review]:
-----------------------------------------------------------------

::: misc.py
@@ +2461,5 @@
>  
>              builder = {'name': prettyName,
>                      'builddir': '%s_%s_spidermonkey-%s' % (branch, platform, variant),
>                      'slavebuilddir': reallyShort('%s_%s_spidermonkey-%s' % (branch, platform, variant)),
> +                    'slavenames': SLAVES[platform.replace('-debug', '', 1)],

Why is this necessary? pf['slaves'] should be correct, even for debug builds....

Comment 9

[[e-mail] (:bhearsum)]

Comment on attachment 691575 [details] [diff] [review]
Pass through configuration data to spidermonkey jobs that need it for mock and/or try builds

Review of attachment 691575 [details] [diff] [review]:
-----------------------------------------------------------------

::: misc.py
@@ +2443,5 @@
>                      factory_kwargs[a] = pf[a]
> +            factory_kwargs['env'] = env
> +
> +            extra_args = []
> +            if config.get('enable_try'):

I think we'd win by setting these regardless of whether we're building on try or not. Unless you've got a reason not to, let's just remove this condition.

Comment 10

[[e-mail] (:bhearsum)]

Comment on attachment 691960 [details] [diff] [review]
Pass in the appropriate slaves and configuration data to the spidermonkey builds

Review of attachment 691960 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla/scheduler_master.cfg
@@ +78,5 @@
>      branchObjects['change_source'] = usefulChangeSources
>      buildObjects = mergeBuildObjects(buildObjects, branchObjects)
>  
>  for project in ACTIVE_PROJECTS:
> +    slaves = TRY_SLAVES if PROJECTS[project].get('enable_try') else SLAVES

Please put this on multiple lines to make it easier to read. r=me with that changed.

Comment 11

[[e-mail] (:bhearsum)]

Comment on attachment 691570 [details] [diff] [review]
Pass through some try checkout options and switch to the current gcc version.

Review of attachment 691570 [details] [diff] [review]:
-----------------------------------------------------------------

::: scripts/spidermonkey_builds/spidermonkey.sh
@@ +92,5 @@
>      NSPR64="--enable-64bit"
>  fi
>  
>  if [ "$OSTYPE" = "linux-gnu" ]; then
> +    GCCDIR=/tools/gcc-4.5-0moz3

We were talking on IRC and I realized that we're already hardcoding this. Given that, this is fine.

Comment 12

[Steve Fink [:sfink] [:s:]]

(In reply to Ben Hearsum [:bhearsum] from comment #9)
> Comment on attachment 691575 [details] [diff] [review]
> Pass through configuration data to spidermonkey jobs that need it for mock
> and/or try builds
> 
> Review of attachment 691575 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: misc.py
> @@ +2443,5 @@
> >                      factory_kwargs[a] = pf[a]
> > +            factory_kwargs['env'] = env
> > +
> > +            extra_args = []
> > +            if config.get('enable_try'):
> > +                extra_args += [ '-r', WithProperties("%(revision)s") ]
> 
> I think we'd win by setting these regardless of whether we're building on
> try or not. Unless you've got a reason not to, let's just remove this
> condition.

I'll have to figure out if the revision property is set for non-try pushes.

Comment 13

[Steve Fink [:sfink] [:s:]]

It ate this comment.

(In reply to Ben Hearsum [:bhearsum] from comment #8)
> Comment on attachment 691572 [details] [diff] [review]
> Use the spidermonkey slaves passed in as a parameter rather than pulling
> them from the platform_config
> 
> Review of attachment 691572 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: misc.py
> @@ +2461,5 @@
> >  
> >              builder = {'name': prettyName,
> >                      'builddir': '%s_%s_spidermonkey-%s' % (branch, platform, variant),
> >                      'slavebuilddir': reallyShort('%s_%s_spidermonkey-%s' % (branch, platform, variant)),
> > +                    'slavenames': SLAVES[platform.replace('-debug', '', 1)],
> 
> Why is this necessary? pf['slaves'] should be correct, even for debug
> builds....

Yes, I think you're right. I think this patch is from a while back, before I learned that there are separate masters for try. pf['slaves'] was getting its value from SLAVES[platform] in a script that has access to both SLAVES and TRY_SLAVES, and I thought that couldn't possibly be right.

Good, then this patch is completely bogus and unnecessary.

Comment 14

[Steve Fink [:sfink] [:s:]]

Attachment: Pass through configuration data to spidermonkey jobs that need it for mock and/or try builds

Comment 15

[[e-mail] (:bhearsum)]

Comment on attachment 693684 [details] [diff] [review]
Pass through configuration data to spidermonkey jobs that need it for mock and/or try builds

Review of attachment 693684 [details] [diff] [review]:
-----------------------------------------------------------------

(In reply to Steve Fink [:sfink] from comment #12)
> (In reply to Ben Hearsum [:bhearsum] from comment #9)
> > Comment on attachment 691575 [details] [diff] [review]
> > Pass through configuration data to spidermonkey jobs that need it for mock
> > and/or try builds
> > 
> > Review of attachment 691575 [details] [diff] [review]:
> > -----------------------------------------------------------------
> > 
> > ::: misc.py
> > @@ +2443,5 @@
> > >                      factory_kwargs[a] = pf[a]
> > > +            factory_kwargs['env'] = env
> > > +
> > > +            extra_args = []
> > > +            if config.get('enable_try'):
> > > +                extra_args += [ '-r', WithProperties("%(revision)s") ]
> > 
> > I think we'd win by setting these regardless of whether we're building on
> > try or not. Unless you've got a reason not to, let's just remove this
> > condition.
> 
> I'll have to figure out if the revision property is set for non-try pushes.

I just confirmed that this is indeed set for non-try pushes. This should be fine. Please wait for me to test these patches before landing -- I plan to do that today still.

Comment 16

[Steve Fink [:sfink] [:s:]]

Attachment: Pass in the appropriate slaves and configuration data to the spidermonkey builds

This needs to be rebased past bug 818191. Since you're going to try it out, I'm uploading a rebased patch.

Comment 17

[[e-mail] (:bhearsum)]

I tried running a  "Linux x86-64 mozilla-central spidermonkey_mozilla-inbound-warnaserr build" and got the following:

mock_mozilla -v -r mozilla-centos6-x86_64 --cwd /builds/slave/m-in-lnx64-spidermonkey-warnaserr/. --unpriv --shell '/usr/bin/env HG_SHARE_BASE_DIR="/builds/hg-shared" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_SYMBOLS_EXTRA_BUILDID="linux64" SYMBOL_SERVER_HOST="dev-stage01.srv.releng.scl3.mozilla.com" CCACHE_DIR="/builds/ccache" POST_SYMBOL_UPLOAD_CMD="/usr/local/bin/post-symbol-upload.py" MOZ_OBJDIR="obj-firefox" SYMBOL_SERVER_SSH_KEY="/home/mock_mozilla/.ssh/ffxbld_dsa" PATH="/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin" TINDERBOX_OUTPUT="1" CCACHE_COMPRESS="1" HG_REPO="http://hg.mozilla.org/integration/mozilla-inbound" SYMBOL_SERVER_PATH="/mnt/netapp/breakpad/symbols_ffx/" PROPERTIES_FILE="/builds/slave/m-in-lnx64-spidermonkey-warnaserr/buildprops.json" LC_ALL="C" SYMBOL_SERVER_USER="ffxbld" DISPLAY=":2" CCACHE_UMASK="002" scripts/scripts/spidermonkey_builds/spidermonkey.sh -r '"'"''"'"' -m http://hg-internal.dmz.scl3.mozilla.com/integration/mozilla-inbound -b http://dev-stage01.build.mozilla.org/pub/mozilla.org/firefox/bundles/integration/mozilla-inbound.hg warnaserr'
 in dir /builds/slave/m-in-lnx64-spidermonkey-warnaserr/. (timeout 3600 secs)
 watching logfiles {}
 argv: ['mock_mozilla', '-v', '-r', 'mozilla-centos6-x86_64', '--cwd', '/builds/slave/m-in-lnx64-spidermonkey-warnaserr/.', '--unpriv', '--shell', '/usr/bin/env HG_SHARE_BASE_DIR="/builds/hg-shared" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_SYMBOLS_EXTRA_BUILDID="linux64" SYMBOL_SERVER_HOST="dev-stage01.srv.releng.scl3.mozilla.com" CCACHE_DIR="/builds/ccache" POST_SYMBOL_UPLOAD_CMD="/usr/local/bin/post-symbol-upload.py" MOZ_OBJDIR="obj-firefox" SYMBOL_SERVER_SSH_KEY="/home/mock_mozilla/.ssh/ffxbld_dsa" PATH="/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin" TINDERBOX_OUTPUT="1" CCACHE_COMPRESS="1" HG_REPO="http://hg.mozilla.org/integration/mozilla-inbound" SYMBOL_SERVER_PATH="/mnt/netapp/breakpad/symbols_ffx/" PROPERTIES_FILE="/builds/slave/m-in-lnx64-spidermonkey-warnaserr/buildprops.json" LC_ALL="C" SYMBOL_SERVER_USER="ffxbld" DISPLAY=":2" CCACHE_UMASK="002" scripts/scripts/spidermonkey_builds/spidermonkey.sh -r \'\' -m http://hg-internal.dmz.scl3.mozilla.com/integration/mozilla-inbound -b http://dev-stage01.build.mozilla.org/pub/mozilla.org/firefox/bundles/integration/mozilla-inbound.hg warnaserr']
 environment:
  CCACHE_COMPRESS=1
  CCACHE_DIR=/builds/ccache
  CCACHE_HASHDIR=
  CCACHE_UMASK=002
  DISPLAY=:2
  G_BROKEN_FILENAMES=1
  HG_REPO=http://hg.mozilla.org/integration/mozilla-inbound
  HG_SHARE_BASE_DIR=/builds/hg-shared
  HISTCONTROL=ignoredups
  HISTSIZE=1000
  HOME=/home/cltbld
  HOSTNAME=dev-linux64-ec2-002.build.aws-us-east-1.mozilla.com
  LC_ALL=C
  LESSOPEN=|/usr/bin/lesspipe.sh %s
  LOGNAME=cltbld
  MAIL=/var/spool/mail/cltbld
  MOZ_CRASHREPORTER_NO_REPORT=1
  MOZ_OBJDIR=obj-firefox
  MOZ_SYMBOLS_EXTRA_BUILDID=linux64
  PATH=/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin
  POST_SYMBOL_UPLOAD_CMD=/usr/local/bin/post-symbol-upload.py
  PROPERTIES_FILE=/builds/slave/m-in-lnx64-spidermonkey-warnaserr/buildprops.json
  PWD=/builds/slave/m-in-lnx64-spidermonkey-warnaserr
  SHELL=/bin/bash
  SHLVL=1
  SYMBOL_SERVER_HOST=dev-stage01.srv.releng.scl3.mozilla.com
  SYMBOL_SERVER_PATH=/mnt/netapp/breakpad/symbols_ffx/
  SYMBOL_SERVER_SSH_KEY=/home/mock_mozilla/.ssh/ffxbld_dsa
  SYMBOL_SERVER_USER=ffxbld
  TERM=linux
  TINDERBOX_OUTPUT=1
  USER=cltbld
  _=/tools/buildbot/bin/python
 using PTY: False
INFO: mock_mozilla.py version 1.0.3 starting...
State Changed: init plugins
DEBUG: ensuring that dir exists: /builds/mock_mozilla/cache/mozilla-centos6-x86_64/yum_cache/
INFO: selinux disabled
State Changed: start
DEBUG: mock final configuration:
DEBUG:     scm:  False
DEBUG:     resultdir:  %(basedir)s/%(root)s/result
DEBUG:     legal_host_arches:  ('i386', 'i586', 'i686', 'x86_64')
DEBUG:     dist:  centos6
DEBUG:     internal_setarch:  True
DEBUG:     scm_opts:  {'git_get': 'git clone SCM_BRN git://localhost/SCM_PKG.git SCM_PKG', 'cvs_get': 'cvs -d /srv/cvs co SCM_BRN SCM_PKG', 'git_timestamps': False, 'ext_src_dir': '/dev/null', 'spec': 'SCM_PKG.spec', 'write_tar': False, 'method': 'git', 'svn_get': 'svn co file:///srv/svn/SCM_PKG/SCM_BRN SCM_PKG'}
DEBUG:     use_host_resolv:  True
DEBUG:     more_buildreqs:  {}
DEBUG:     createrepo_command:  /usr/bin/createrepo -d -q -x *.src.rpm
DEBUG:     yum_builddep_opts:  
DEBUG:     yum.conf:  
[main]
cachedir=/var/cache/yum
debuglevel=1
reposdir=/dev/null
logfile=/var/log/yum.log
retries=20
obsoletes=1
gpgcheck=0
assumeyes=1
syslog_ident=mock
syslog_device=

# repos

[centos6]
name=centos6
baseurl=http://repos/repos/yum/mirrors/centos/6/latest/os/x86_64
failovermethod=priority

[centos6-updates]
name=updates
baseurl=http://repos/repos/yum/mirrors/centos/6/latest/updates/x86_64
failovermethod=priority

[releng-centos6-x86_64]
name=releng-centos6-x86_64
baseurl=http://repos/repos/yum/releng/public/CentOS/6/x86_64

[releng-centos6-noarch]
name=releng-centos6-noarch
baseurl=http://repos/repos/yum/releng/public/CentOS/6/noarch

DEBUG:     cleanup_on_failure:  False
DEBUG:     plugins:  ['tmpfs', 'root_cache', 'yum_cache', 'bind_mount', 'ccache', 'selinux']
DEBUG:     chrootuid:  500
DEBUG:     no_root_shells:  False
DEBUG:     root_log_fmt_str:  %(levelname)s %(filename)s:%(lineno)d:  %(message)s
DEBUG:     state_log_fmt_str:  %(asctime)s - %(message)s
DEBUG:     chroothome:  /home/mock_mozilla
DEBUG:     basedir:  /builds/mock_mozilla
DEBUG:     internal_dev_setup:  True
DEBUG:     macros:  {'%_rpmfilename': '%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm', '%_topdir': '/builddir/build'}
DEBUG:     version:  1.0.3
DEBUG:     online:  True
DEBUG:     build_log_fmt_name:  unadorned
DEBUG:     chrootgid:  494
DEBUG:     files:  {}
DEBUG:     rpmbuild_arch:  x86_64
DEBUG:     log_config_file:  logging.ini
DEBUG:     rpmbuild_timeout:  0
DEBUG:     cleanup_on_success:  False
DEBUG:     chroot_setup_cmd:  install bash bzip2 coreutils cpio diffutils fedora-release findutils gawk gcc gcc-c++ grep gzip info make patch redhat-rpm-config rpm-build sed shadow-utils tar unzip util-linux which xz
DEBUG:     useradd:  /usr/sbin/useradd -o -m -u %(uid)s -g %(gid)s -d %(home)s -n %(user)s
DEBUG:     root_log_fmt_name:  detailed
DEBUG:     chroot_name:  mozilla-centos6-x86_64
DEBUG:     plugin_conf:  {'yum_cache_opts': {'max_metadata_age_days': 30, 'root': 'mozilla-centos6-x86_64', 'basedir': '/builds/mock_mozilla/mozilla-centos6-x86_64', 'cachedir': '/builds/mock_mozilla/cache/mozilla-centos6-x86_64', 'online': True, 'cache_topdir': '/builds/mock_mozilla/cache', 'max_age_days': 30, 'dir': '%(cache_topdir)s/%(root)s/yum_cache/'}, 'bind_mount_opts': {'dirs': [('/builds/slave', '/builds/slave'), ('/builds/ccache', '/builds/ccache'), ('/builds/hg-shared', '/builds/hg-shared'), ('/builds/git-shared', '/builds/git-shared'), ('/tmp/.X11-unix', '/tmp/.X11-unix'), ('/builds/mock_mozilla/mozilla-centos6-x86_64/targetdata/', '/targetdata/')], 'basedir': '/builds/mock_mozilla/mozilla-centos6-x86_64', 'root': 'mozilla-centos6-x86_64', 'cachedir': '/builds/mock_mozilla/cache/mozilla-centos6-x86_64', 'cache_topdir': '/builds/mock_mozilla/cache'}, 'root_cache_enable': True, 'selinux_opts': {'basedir': '/builds/mock_mozilla/mozilla-centos6-x86_64', 'root': 'mozilla-centos6-x86_64', 'cachedir': '/builds/mock_mozilla/cache/mozilla-centos6-x86_64', 'cache_topdir': '/builds/mock_mozilla/cache'}, 'ccache_opts': {'basedir': '/builds/mock_mozilla/mozilla-centos6-x86_64', 'max_cache_size': '4G', 'cachedir': '/builds/mock_mozilla/cache/mozilla-centos6-x86_64', 'cache_topdir': '/builds/mock_mozilla/cache', 'root': 'mozilla-centos6-x86_64', 'dir': '%(cache_topdir)s/%(root)s/ccache/'}, 'bind_mount_enable': True, 'selinux_enable': True, 'yum_cache_enable': True, 'tmpfs_enable': False, 'tmpfs_opts': {'basedir': '/builds/mock_mozilla/mozilla-centos6-x86_64', 'required_ram_mb': 900, 'cache_topdir': '/builds/mock_mozilla/cache', 'cachedir': '/builds/mock_mozilla/cache/mozilla-centos6-x86_64', 'max_fs_size': None, 'root': 'mozilla-centos6-x86_64'}, 'ccache_enable': True, 'root_cache_opts': {'exclude_dirs': ['./proc', './sys', './dev', './tmp/ccache', './var/cache/yum'], 'compress_program': 'pigz', 'extension': '.gz', 'root': 'mozilla-centos6-x86_64', 'basedir': '/builds/mock_mozilla/mozilla-centos6-x86_64', 'cachedir': '/builds/mock_mozilla/cache/mozilla-centos6-x86_64', 'cache_topdir': '/builds/mock_mozilla/cache', 'max_age_days': 15, 'dir': '%(cache_topdir)s/%(root)s/root_cache/'}}
DEBUG:     config_paths:  ['/etc/mock_mozilla/site-defaults.cfg', '/etc/mock_mozilla/mozilla-centos6-x86_64.cfg']
DEBUG:     rhnplugin.conf:  
[main]
enabled=0
DEBUG:     build_log_fmt_str:  %(message)s
DEBUG:     state_log_fmt_name:  state
DEBUG:     plugin_dir:  /usr/lib/python2.6/site-packages/mock_mozilla/plugins
DEBUG:     clean:  True
DEBUG:     createrepo_on_rpms:  False
DEBUG:     cache_topdir:  /builds/mock_mozilla/cache
DEBUG:     root:  mozilla-centos6-x86_64
DEBUG:     target_arch:  x86_64
DEBUG: Unsharing. Flags: 131072
State Changed: lock buildroot
DEBUG: shell: calling preshell hooks
DEBUG: ensuring that dir exists: /builds/slave
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/slave
DEBUG: ensuring that dir exists: /builds/ccache
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/ccache
DEBUG: ensuring that dir exists: /builds/hg-shared
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/hg-shared
DEBUG: ensuring that dir exists: /builds/git-shared
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/git-shared
DEBUG: ensuring that dir exists: /tmp/.X11-unix
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/tmp/.X11-unix
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/targetdata/
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/targetdata/
DEBUG: shell: setting up root files
DEBUG: create skeleton dirs
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/var/lib/rpm
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/var/lib/yum
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/var/lib/dbus
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/var/log
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/var/lock/rpm
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/var/cache/yum
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/etc/rpm
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/tmp
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/tmp/ccache
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/var/tmp
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/etc/yum.repos.d
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/etc/yum
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/proc
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/sys
DEBUG: remove tree: /builds/mock_mozilla/mozilla-centos6-x86_64/root/dev
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/dev/pts
DEBUG: creating dir: /builds/mock_mozilla/mozilla-centos6-x86_64/root/dev/pts
DEBUG: ensuring that dir exists: /builds/mock_mozilla/mozilla-centos6-x86_64/root/dev/shm
DEBUG: creating dir: /builds/mock_mozilla/mozilla-centos6-x86_64/root/dev/shm
DEBUG: kver == 2.6.32-220.23.1.el6.centos.plus.x86_64
DEBUG: touch required files
DEBUG: touching file: /builds/mock_mozilla/mozilla-centos6-x86_64/root/etc/mtab
DEBUG: touching file: /builds/mock_mozilla/mozilla-centos6-x86_64/root/etc/fstab
DEBUG: touching file: /builds/mock_mozilla/mozilla-centos6-x86_64/root/var/log/yum.log
DEBUG: shell: mounting all filesystems
DEBUG: mount -n -t proc   mock_chroot_proc   /builds/mock_mozilla/mozilla-centos6-x86_64/root/proc
DEBUG: Executing command: mount -n -t proc   mock_chroot_proc   /builds/mock_mozilla/mozilla-centos6-x86_64/root/proc
DEBUG: Child return code was: 0
DEBUG: mount -n -t sysfs  mock_chroot_sysfs  /builds/mock_mozilla/mozilla-centos6-x86_64/root/sys
DEBUG: Executing command: mount -n -t sysfs  mock_chroot_sysfs  /builds/mock_mozilla/mozilla-centos6-x86_64/root/sys
DEBUG: Child return code was: 0
DEBUG: mount -n --bind /builds/mock_mozilla/cache/mozilla-centos6-x86_64/yum_cache/  /builds/mock_mozilla/mozilla-centos6-x86_64/root/var/cache/yum
DEBUG: Executing command: mount -n --bind /builds/mock_mozilla/cache/mozilla-centos6-x86_64/yum_cache/  /builds/mock_mozilla/mozilla-centos6-x86_64/root/var/cache/yum
DEBUG: Child return code was: 0
DEBUG: mount -n --bind /builds/slave  /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/slave
DEBUG: Executing command: mount -n --bind /builds/slave  /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/slave
DEBUG: Child return code was: 0
DEBUG: mount -n --bind /builds/ccache  /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/ccache
DEBUG: Executing command: mount -n --bind /builds/ccache  /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/ccache
DEBUG: Child return code was: 0
DEBUG: mount -n --bind /builds/hg-shared  /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/hg-shared
DEBUG: Executing command: mount -n --bind /builds/hg-shared  /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/hg-shared
DEBUG: Child return code was: 0
DEBUG: mount -n --bind /builds/git-shared  /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/git-shared
DEBUG: Executing command: mount -n --bind /builds/git-shared  /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/git-shared
DEBUG: Child return code was: 0
DEBUG: mount -n --bind /tmp/.X11-unix  /builds/mock_mozilla/mozilla-centos6-x86_64/root/tmp/.X11-unix
DEBUG: Executing command: mount -n --bind /tmp/.X11-unix  /builds/mock_mozilla/mozilla-centos6-x86_64/root/tmp/.X11-unix
DEBUG: Child return code was: 0
DEBUG: mount -n --bind /builds/mock_mozilla/mozilla-centos6-x86_64/targetdata/  /builds/mock_mozilla/mozilla-centos6-x86_64/root/targetdata/
DEBUG: Executing command: mount -n --bind /builds/mock_mozilla/mozilla-centos6-x86_64/targetdata/  /builds/mock_mozilla/mozilla-centos6-x86_64/root/targetdata/
DEBUG: Child return code was: 0
DEBUG: mount -n --bind /builds/mock_mozilla/cache/mozilla-centos6-x86_64/ccache/  /builds/mock_mozilla/mozilla-centos6-x86_64/root/tmp/ccache
DEBUG: Executing command: mount -n --bind /builds/mock_mozilla/cache/mozilla-centos6-x86_64/ccache/  /builds/mock_mozilla/mozilla-centos6-x86_64/root/tmp/ccache
DEBUG: Child return code was: 0
DEBUG: mount -n -t devpts -o gid=5,mode=0620,ptmxmode=0666,newinstance mock_chroot_devpts /builds/mock_mozilla/mozilla-centos6-x86_64/root/dev/pts
DEBUG: Executing command: mount -n -t devpts -o gid=5,mode=0620,ptmxmode=0666,newinstance mock_chroot_devpts /builds/mock_mozilla/mozilla-centos6-x86_64/root/dev/pts
DEBUG: Child return code was: 0
DEBUG: mount -n -t tmpfs mock_chroot_shmfs /builds/mock_mozilla/mozilla-centos6-x86_64/root/dev/shm
DEBUG: Executing command: mount -n -t tmpfs mock_chroot_shmfs /builds/mock_mozilla/mozilla-centos6-x86_64/root/dev/shm
DEBUG: Child return code was: 0
State Changed: shell
DEBUG: doshell: chrootPath:/builds/mock_mozilla/mozilla-centos6-x86_64/root/, uid:500, gid:494
DEBUG: doshell environment: {'LANG': 'en_US.UTF-8', 'TERM': 'vt100', 'SHELL': '/bin/bash', 'TZ': 'EST5EDT', 'HOSTNAME': 'mock', 'HOME': '/builds', 'PATH': '/usr/bin:/bin:/usr/sbin:/sbin', 'TMPDIR': '/tmp'}
DEBUG: doshell: command: /usr/bin/env HG_SHARE_BASE_DIR="/builds/hg-shared" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_SYMBOLS_EXTRA_BUILDID="linux64" SYMBOL_SERVER_HOST="dev-stage01.srv.releng.scl3.mozilla.com" CCACHE_DIR="/builds/ccache" POST_SYMBOL_UPLOAD_CMD="/usr/local/bin/post-symbol-upload.py" MOZ_OBJDIR="obj-firefox" SYMBOL_SERVER_SSH_KEY="/home/mock_mozilla/.ssh/ffxbld_dsa" PATH="/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin" TINDERBOX_OUTPUT="1" CCACHE_COMPRESS="1" HG_REPO="http://hg.mozilla.org/integration/mozilla-inbound" SYMBOL_SERVER_PATH="/mnt/netapp/breakpad/symbols_ffx/" PROPERTIES_FILE="/builds/slave/m-in-lnx64-spidermonkey-warnaserr/buildprops.json" LC_ALL="C" SYMBOL_SERVER_USER="ffxbld" DISPLAY=":2" CCACHE_UMASK="002" scripts/scripts/spidermonkey_builds/spidermonkey.sh -r '' -m http://hg-internal.dmz.scl3.mozilla.com/integration/mozilla-inbound -b http://dev-stage01.build.mozilla.org/pub/mozilla.org/firefox/bundles/integration/mozilla-inbound.hg warnaserr
DEBUG: child environment: {'LANG': 'en_US.UTF-8', 'TERM': 'vt100', 'SHELL': '/bin/bash', 'TZ': 'EST5EDT', 'HOSTNAME': 'mock', 'HOME': '/builds', 'PATH': '/usr/bin:/bin:/usr/sbin:/sbin', 'TMPDIR': '/tmp'}
++ dirname scripts/scripts/spidermonkey_builds/spidermonkey.sh
+ pushd scripts/scripts/spidermonkey_builds/../../
+ SCRIPTS_DIR=/builds/slave/m-in-lnx64-spidermonkey-warnaserr/scripts
+ popd
+ SPIDERDIR=/builds/slave/m-in-lnx64-spidermonkey-warnaserr/scripts/scripts/spidermonkey_builds
+ '[' -z http://hg.mozilla.org/integration/mozilla-inbound ']'
+ VARIANT=-r
+ '[' '!' -f /builds/slave/m-in-lnx64-spidermonkey-warnaserr/scripts/scripts/spidermonkey_builds/-r ']'
+ echo 'Could not find -r'
Could not find -r
+ echo 'build.sh <variant>'
build.sh <variant>
+ exit 1
DEBUG: shell: unmounting all filesystems
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/dev/shm
DEBUG: Child return code was: 0
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/dev/pts
DEBUG: Child return code was: 0
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/tmp/ccache
DEBUG: Child return code was: 0
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/targetdata/
DEBUG: Child return code was: 0
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/tmp/.X11-unix
DEBUG: Child return code was: 0
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/git-shared
DEBUG: Child return code was: 0
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/hg-shared
DEBUG: Child return code was: 0
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/ccache
DEBUG: Child return code was: 0
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/builds/slave
DEBUG: Child return code was: 0
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/var/cache/yum
DEBUG: Child return code was: 0
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/sys
DEBUG: Child return code was: 0
DEBUG: Executing command: umount -n /builds/mock_mozilla/mozilla-centos6-x86_64/root/proc
DEBUG: Child return code was: 0
DEBUG: shell: calling postshell hooks
State Changed: unlock buildroot
program finished with exit code 1
elapsedTime=0.501376

Comment 18

[[e-mail] (:bhearsum)]

Also, this doesn't seem to pass checkconfig on try, I get the following:
cd master && /builds/buildbot/bhearsum/builds/bin/buildbot checkconfig
/builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/twisted/mail/smtp.py:10: DeprecationWarning: the MimeWriter module is deprecated; use the email package instead
  import MimeWriter, tempfile, rfc822
Traceback (most recent call last):
  File "/builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/buildbot-0.8.2_hg_3dc678eecd11_production_0.8-py2.6.egg/buildbot/scripts/runner.py", line 1039, in doCheckConfig
    ConfigLoader(configFileName=configFileName)
  File "/builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/buildbot-0.8.2_hg_3dc678eecd11_production_0.8-py2.6.egg/buildbot/scripts/checkconfig.py", line 31, in __init__
    self.loadConfig(configFile, check_synchronously_only=True)
  File "/builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/buildbot-0.8.2_hg_3dc678eecd11_production_0.8-py2.6.egg/buildbot/master.py", line 628, in loadConfig
    exec f in localDict
  File "/builds/buildbot/bhearsum/builds/master/master.cfg", line 103, in <module>
    projectObjects = generateProjectObjects(project, PROJECTS[project], SLAVES)
  File "/builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/buildbotcustom/misc.py", line 2664, in generateProjectObjects
    spiderMonkeyObjects = generateSpiderMonkeyObjects(project, config, SLAVES)
  File "/builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/buildbotcustom/misc.py", line 2473, in generateSpiderMonkeyObjects
    config['scripts_repo'],

Comment 19

[[e-mail] (:bhearsum)]

(In reply to Ben Hearsum [:bhearsum] from comment #18)
> Also, this doesn't seem to pass checkconfig on try, I get the following:
> cd master && /builds/buildbot/bhearsum/builds/bin/buildbot checkconfig
> /builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/twisted/mail/
> smtp.py:10: DeprecationWarning: the MimeWriter module is deprecated; use the
> email package instead
>   import MimeWriter, tempfile, rfc822
> Traceback (most recent call last):
>   File
> "/builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/buildbot-0.8.
> 2_hg_3dc678eecd11_production_0.8-py2.6.egg/buildbot/scripts/runner.py", line
> 1039, in doCheckConfig
>     ConfigLoader(configFileName=configFileName)
>   File
> "/builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/buildbot-0.8.
> 2_hg_3dc678eecd11_production_0.8-py2.6.egg/buildbot/scripts/checkconfig.py",
> line 31, in __init__
>     self.loadConfig(configFile, check_synchronously_only=True)
>   File
> "/builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/buildbot-0.8.
> 2_hg_3dc678eecd11_production_0.8-py2.6.egg/buildbot/master.py", line 628, in
> loadConfig
>     exec f in localDict
>   File "/builds/buildbot/bhearsum/builds/master/master.cfg", line 103, in
> <module>
>     projectObjects = generateProjectObjects(project, PROJECTS[project],
> SLAVES)
>   File
> "/builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/buildbotcustom/
> misc.py", line 2664, in generateProjectObjects
>     spiderMonkeyObjects = generateSpiderMonkeyObjects(project, config,
> SLAVES)
>   File
> "/builds/buildbot/bhearsum/builds/lib/python2.6/site-packages/buildbotcustom/
> misc.py", line 2473, in generateSpiderMonkeyObjects
>     config['scripts_repo'],

Ah, this is because you need to update staging_config and preproduction_config.py too. Should've caught that in review, sorry!

Comment 20

[Steve Fink [:sfink] [:s:]]

Attachment: Re-enable spidermonkey builds

Whoops, missed staging_config.py and preproduction_config.py

Comment 21

[Steve Fink [:sfink] [:s:]]

(In reply to Steve Fink [:sfink] from comment #13)
> (In reply to Ben Hearsum [:bhearsum] from comment #8)
> > Comment on attachment 691572 [details] [diff] [review]
> > Use the spidermonkey slaves passed in as a parameter rather than pulling
> > them from the platform_config
> > 
> > Review of attachment 691572 [details] [diff] [review]:
> > -----------------------------------------------------------------
> > 
> > ::: misc.py
> > @@ +2461,5 @@
> > >  
> > >              builder = {'name': prettyName,
> > >                      'builddir': '%s_%s_spidermonkey-%s' % (branch, platform, variant),
> > >                      'slavebuilddir': reallyShort('%s_%s_spidermonkey-%s' % (branch, platform, variant)),
> > > +                    'slavenames': SLAVES[platform.replace('-debug', '', 1)],
> > 
> > Why is this necessary? pf['slaves'] should be correct, even for debug
> > builds....
> 
> Yes, I think you're right. I think this patch is from a while back, before I
> learned that there are separate masters for try. pf['slaves'] was getting
> its value from SLAVES[platform] in a script that has access to both SLAVES
> and TRY_SLAVES, and I thought that couldn't possibly be right.
> 
> Good, then this patch is completely bogus and unnecessary.

Ok, the reason why I was doing this is that eg build_master.cfg has the following imports, in this order:

  import buildbotcustom.misc
  import config
  import master_localconfig

Say master_localconfig points to try_localconfig. Then it will overwrite SLAVES with TRY_SLAVES. But it's too late; config.py has already been loaded and it set PLATFORM_VARS['linux64-debug']['slaves'] to SLAVES['mock'] using the original SLAVES (namely, production slaves instead of try slaves.)

I'm a bit scared of reordering the imports, because the interdependencies are pretty tangled (there's at least one cycle). But my original "fix" made things messier, too.

Comment 22

[Steve Fink [:sfink] [:s:]]

Attachment: Point PROJECTS' platforms directly to the relevant branch

On IRC, bhearsum suggested just fixing this with

+PROJECTS['spidermonkey_try']['platforms']['linux64-debug']['slaves'] = TRY_SLAVES['mock']

That's straightforward, but it'd be nice to have something more general. The attached patch is what I was thinking of. If you also add in a fix for bug 811723, test-master.sh passes with this.

This patch is a little heavyweight, but it removes the need to update multiple places to add/switch platforms for the rootanalysis project build, and makes it slightly easier to get other project jobs running on mock/try if we want to.

Then again, the style of config.py is to be pretty explicit. Either way will work for me. I hereby submit my works to the will of the Elder RelEng Gods.

Comment 23

[[e-mail] (:bhearsum)]

Comment on attachment 693684 [details] [diff] [review]
Pass through configuration data to spidermonkey jobs that need it for mock and/or try builds

Did you obsolete the wrong patch? This one looks necessary still.

Comment 24

[[e-mail] (:bhearsum)]

Comment on attachment 694136 [details] [diff] [review]
Point PROJECTS' platforms directly to the relevant branch

Review of attachment 694136 [details] [diff] [review]:
-----------------------------------------------------------------

There's a bit of testing I want to do with this, but I'm currently stuck on this error:
  File "/home/bhearsum/Mozilla/checkouts/clean/buildbot-configs/mozilla/master.cfg", line 16, in <module>
    import master_localconfig
  File "/tmp/tmpi8Z4Z9/master_localconfig.py", line 61, in <module>
AttributeError: 'list' object has no attribute 'items'


Which I think is because of the unbitrotting that you did. In particular, this doesn't look right:
if 'limit_projects' in master_config:
    ACTIVE_PROJECTS = [x.encode("utf-8") for x in master_config['limit_projects']]
else:
    ACTIVE_PROJECTS = PROJECTS.keys()
ACTIVE_PROJECTS = [ k for k,v in ACTIVE_PROJECTS.items() if not v.get('enable_try') ]

(ACTIVE_PROJECTS is always a list now, so .items() will never work.)

::: mozilla/config.py
@@ +977,5 @@
>          'variants': {
>              'linux64-debug':  ['rootanalysis'],
>          },
>          'platforms': {
> +            'linux64-debug': True, # Filled in with branch-specific values below

For style consistency, use {} instead of True here, like https://github.com/mozilla/build-buildbot-configs/blob/master/mozilla/config.py#L1060

Comment 25

[Steve Fink [:sfink] [:s:]]

Argh, sorry, I seem to be making a mess of this. I have a version somewhere that fixes the ACTIVE_PROJECTS thing; I must have qreffed it into the wrong patch or something. Lemme try to straighten this out...

Comment 26

[Steve Fink [:sfink] [:s:]]

Attachment: Pass in the appropriate slaves and configuration data to the spidermonkey builds

Oh, I had that fix as a separate patch that I forgot to qfold. And I've fixed bzexport, so it can save me from myself (it figures out which patch to obsolete automatically, which is way less error-prone than relying on me.)

Comment 27

[Steve Fink [:sfink] [:s:]]

Attachment: Point PROJECTS' platforms directly to the relevant branch

Switched 'True's to '{}'s

Comment 28

[Steve Fink [:sfink] [:s:]]

Attachment: Pass through configuration data to spidermonkey jobs that need it for mock and/or try builds

I think you're right; I think I mistakenly obsoleted this one.

I'm getting lost in these 3 different repos.

Comment 29

[[e-mail] (:bhearsum)]

Comment on attachment 694036 [details] [diff] [review]
Re-enable spidermonkey builds

I gave this another run through in staging, looks good with this latest batch of patches!

Comment 30

[Steve Fink [:sfink] [:s:]]

http://hg.mozilla.org/build/buildbot-configs/rev/57fa222efed1
http://hg.mozilla.org/build/buildbot-configs/rev/8ec6aebdb5e3
http://hg.mozilla.org/build/buildbot-configs/rev/18812beb3440

Comment 31

[Steve Fink [:sfink] [:s:]]

http://hg.mozilla.org/build/tools/rev/70418714bfbf

Comment 32

[Steve Fink [:sfink] [:s:]]

http://hg.mozilla.org/build/buildbotcustom/rev/fe472c742fee

Comment 33

[hwine]

in production

Comment 34

[Chris AtLee [:catlee]]

I think these builders need to be added to nomergeBuilders (http://hg.mozilla.org/build/buildbotcustom/file/fe472c742fee/misc.py#l429)

Right now it looks like they're getting run, but are combining results from multiple try pushes, which probably isn't what you want.

Comment 35

[Steve Fink [:sfink] [:s:]]

(In reply to Chris AtLee [:catlee] from comment #34)
> I think these builders need to be added to nomergeBuilders
> (http://hg.mozilla.org/build/buildbotcustom/file/fe472c742fee/misc.py#l429)
> 
> Right now it looks like they're getting run, but are combining results from
> multiple try pushes, which probably isn't what you want.

Filed bug 824802.

This in production, so it can be marked resolved now, no?