1415803 - |make check| orange hglib.error.CommandError: (255, 'Mercurial Distributed SCM\n\nbasic commands:\n\n ...

Status: RESOLVED WONTFIX

(SeaMonkey :: Build Config, defect)

Description

[Edmund Wong (:ewong)]

Getting this during a |make check|

Comparing manifests...
Comparing packages...
make[3]: Leaving directory `/builds/slave/c-cen-t-lnx/build/objdir/config/tests/src-simple'
make[2]: Leaving directory `/builds/slave/c-cen-t-lnx/build/objdir/config'
make[2]: Entering directory `/builds/slave/c-cen-t-lnx/build/objdir/js/src'
(cd /builds/slave/c-cen-t-lnx/build/mozilla/js/src && /builds/slave/c-cen-t-lnx/build/objdir/_virtualenv/bin/python /builds/slave/c-cen-t-lnx/build/mozilla/config/check_spidermonkey_style.py);
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-lnx/build/mozilla/config/check_spidermonkey_style.py", line 600, in <module>
    main()
  File "/builds/slave/c-cen-t-lnx/build/mozilla/config/check_spidermonkey_style.py", line 589, in main
    ok = check_style()
  File "/builds/slave/c-cen-t-lnx/build/mozilla/config/check_spidermonkey_style.py", line 258, in check_style
    for filename in repo.get_files_in_working_directory():
  File "/builds/slave/c-cen-t-lnx/build/mozilla/python/mozversioncontrol/mozversioncontrol/__init__.py", line 299, in get_files_in_working_directory
    self._run_in_client([b'files', b'-0']).split(b'\0')
  File "/builds/slave/c-cen-t-lnx/build/mozilla/python/mozversioncontrol/mozversioncontrol/__init__.py", line 235, in _run_in_client
    return self._client.rawcommand(args)
  File "/builds/slave/c-cen-t-lnx/build/mozilla/third_party/python/hglib/hglib/client.py", line 246, in rawcommand
    raise error.CommandError(args, ret, out, err)
hglib.error.CommandError: (255, 'Mercurial Distributed SCM\n\nbasic commands:\n\n add           add the specified files on the next commit\n annotate      show changeset information by line for each file\n clone         make a copy of an existing repository\n commit        commit the specified files or all outstanding changes\n diff          diff repository (or selected files)\n export        dump the header and diffs for one or more changesets\n forget        forget the specified files on the next commit\n init          create a new repository in the given directory\n log           show revision history of entire repository or files\n merge         merge working directory with another revision\n pull          pull changes from the specified source\n push          push changes to the specified destination\n qdiff         diff of the current patch and subsequent modifications\n qnew          create a new patch\n qpop          pop the current patch off the stack\n qpush         push the next patch onto the stack\n qrefresh      update the current patch\n remove        remove the specified files on the next commit\n serve         start stand-alone webserver\n status        show changed files in the working directory\n summary       summarize working directory state\n update        update working directory (or switch revisions)\n\nuse "hg help" for the full list of commands or "hg -v" for details', "hg: unknown command 'files'")
make[2]: *** [check-style] Error 1
(cd /builds/slave/c-cen-t-lnx/build/mozilla/js/src && /builds/slave/c-cen-t-lnx/build/objdir/_virtualenv/bin/python /builds/slave/c-cen-t-lnx/build/mozilla/config/check_macroassembler_style.py);
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-lnx/build/mozilla/config/check_macroassembler_style.py", line 296, in <module>
    main()
  File "/builds/slave/c-cen-t-lnx/build/mozilla/config/check_macroassembler_style.py", line 285, in main
    ok = check_style()
  File "/builds/slave/c-cen-t-lnx/build/mozilla/config/check_macroassembler_style.py", line 258, in check_style
    for filename in repo.get_files_in_working_directory():
  File "/builds/slave/c-cen-t-lnx/build/mozilla/python/mozversioncontrol/mozversioncontrol/__init__.py", line 299, in get_files_in_working_directory
    self._run_in_client([b'files', b'-0']).split(b'\0')
  File "/builds/slave/c-cen-t-lnx/build/mozilla/python/mozversioncontrol/mozversioncontrol/__init__.py", line 235, in _run_in_client
    return self._client.rawcommand(args)
  File "/builds/slave/c-cen-t-lnx/build/mozilla/third_party/python/hglib/hglib/client.py", line 246, in rawcommand
    raise error.CommandError(args, ret, out, err)
hglib.error.CommandError: (255, 'Mercurial Distributed SCM\n\nbasic commands:\n\n add           add the specified files on the next commit\n annotate      show changeset information by line for each file\n clone         make a copy of an existing repository\n commit        commit the specified files or all outstanding changes\n diff          diff repository (or selected files)\n export        dump the header and diffs for one or more changesets\n forget        forget the specified files on the next commit\n init          create a new repository in the given directory\n log           show revision history of entire repository or files\n merge         merge working directory with another revision\n pull          pull changes from the specified source\n push          push changes to the specified destination\n qdiff         diff of the current patch and subsequent modifications\n qnew          create a new patch\n qpop          pop the current patch off the stack\n qpush         push the next patch onto the stack\n qrefresh      update the current patch\n remove        remove the specified files on the next commit\n serve         start stand-alone webserver\n status        show changed files in the working directory\n summary       summarize working directory state\n update        update working directory (or switch revisions)\n\nuse "hg help" for the full list of commands or "hg -v" for details', "hg: unknown command 'files'")
make[2]: *** [check-masm] Error 1
(cd /builds/slave/c-cen-t-lnx/build/mozilla && /builds/slave/c-cen-t-lnx/build/objdir/_virtualenv/bin/python /builds/slave/c-cen-t-lnx/build/mozilla/config/check_js_msg_encoding.py);
Traceback (most recent call last):
  File "/builds/slave/c-cen-t-lnx/build/mozilla/config/check_js_msg_encoding.py", line 68, in <module>
    main()
  File "/builds/slave/c-cen-t-lnx/build/mozilla/config/check_js_msg_encoding.py", line 62, in main
    if not check_files():
  File "/builds/slave/c-cen-t-lnx/build/mozilla/config/check_js_msg_encoding.py", line 53, in check_files
    for filename in repo.get_files_in_working_directory():
  File "/builds/slave/c-cen-t-lnx/build/mozilla/python/mozversioncontrol/mozversioncontrol/__init__.py", line 299, in get_files_in_working_directory
    self._run_in_client([b'files', b'-0']).split(b'\0')
  File "/builds/slave/c-cen-t-lnx/build/mozilla/python/mozversioncontrol/mozversioncontrol/__init__.py", line 235, in _run_in_client
    return self._client.rawcommand(args)
  File "/builds/slave/c-cen-t-lnx/build/mozilla/third_party/python/hglib/hglib/client.py", line 246, in rawcommand
    raise error.CommandError(args, ret, out, err)
hglib.error.CommandError: (255, 'Mercurial Distributed SCM\n\nbasic commands:\n\n add           add the specified files on the next commit\n annotate      show changeset information by line for each file\n clone         make a copy of an existing repository\n commit        commit the specified files or all outstanding changes\n diff          diff repository (or selected files)\n export        dump the header and diffs for one or more changesets\n forget        forget the specified files on the next commit\n init          create a new repository in the given directory\n log           show revision history of entire repository or files\n merge         merge working directory with another revision\n pull          pull changes from the specified source\n push          push changes to the specified destination\n qdiff         diff of the current patch and subsequent modifications\n qnew          create a new patch\n qpop          pop the current patch off the stack\n qpush         push the next patch onto the stack\n qrefresh      update the current patch\n remove        remove the specified files on the next commit\n serve         start stand-alone webserver\n status        show changed files in the working directory\n summary       summarize working directory state\n update        update working directory (or switch revisions)\n\nuse "hg help" for the full list of commands or "hg -v" for details', "hg: unknown command 'files'")
make[2]: *** [check-js-msg] Error 1
(cd /builds/slave/c-cen-t-lnx/build/mozilla && /builds/slave/c-cen-t-lnx/build/objdir/_virtualenv/bin/python /builds/slave/c-cen-t-lnx/build/mozilla/config/check_js_opcode.py);
TEST-PASS | check_js_opcode.py | ok
make[2]: Target `check' not remade because of errors.
make[2]: Leaving directory `/builds/slave/c-cen-t-lnx/build/objdir/js/src'
make[1]: *** [js/src/check] Error 2
make[2]: Entering directory `/builds/slave/c-cen-t-lnx/build/objdir/memory/replace/logalloc/replay'
grep "^1 " /builds/slave/c-cen-t-lnx/build/mozilla/memory/replace/logalloc/replay/replay.log > expected_output.log
MALLOC_LOG=1 LD_PRELOAD=/builds/slave/c-cen-t-lnx/build/objdir/memory/replace/logalloc/replay/../liblogalloc.so ./logalloc-replay < /builds/slave/c-cen-t-lnx/build/mozilla/memory/replace/logalloc/replay/replay.log | sed -n '/jemalloc_stats/,$p' | /builds/slave/c-cen-t-lnx/build/objdir/_virtualenv/bin/python /builds/slave/c-cen-t-lnx/build/mozilla/memory/replace/logalloc/replay/logalloc_munge.py | diff -w - expected_output.log
#0 mapped: 1048576; allocated: 0; waste: 0; dirty: 0; bookkeep: 4096; binunused: 0
#10 mapped: 2097152; allocated: 10336; waste: 0; dirty: 0; bookkeep: 9280; binunused: 66400
rm -f test_output.log
MALLOC_LOG=test_output.log LD_PRELOAD=/builds/slave/c-cen-t-lnx/build/objdir/memory/replace/logalloc/replay/../liblogalloc.so ./logalloc-replay < /builds/slave/c-cen-t-lnx/build/mozilla/memory/replace/logalloc/replay/replay.log
#0 mapped: 1048576; allocated: 0; waste: 0; dirty: 0; bookkeep: 4096; binunused: 0
#10 mapped: 2097152; allocated: 10336; waste: 0; dirty: 0; bookkeep: 9280; binunused: 66400
sed -n '/jemalloc_stats/,$p' test_output.log | /builds/slave/c-cen-t-lnx/build/objdir/_virtualenv/bin/python /builds/slave/c-cen-t-lnx/build/mozilla/memory/replace/logalloc/replay/logalloc_munge.py | diff -w - expected_output.log
MALLOC_LOG=1 LD_PRELOAD=/builds/slave/c-cen-t-lnx/build/objdir/memory/replace/logalloc/replay/../minimal/liblogalloc_minimal.so ./logalloc-replay < /builds/slave/c-cen-t-lnx/build/mozilla/memory/replace/logalloc/replay/replay.log | sed -n '/jemalloc_stats/,$p' | /builds/slave/c-cen-t-lnx/build/objdir/_virtualenv/bin/python /builds/slave/c-cen-t-lnx/build/mozilla/memory/replace/logalloc/replay/logalloc_munge.py | diff -w - /builds/slave/c-cen-t-lnx/build/mozilla/memory/replace/logalloc/replay/expected_output_minimal.log
#0 mapped: 1048576; allocated: 0; waste: 0; dirty: 0; bookkeep: 4096; binunused: 0
#10 mapped: 2097152; allocated: 10336; waste: 0; dirty: 0; bookkeep: 9280; binunused: 66400
make[2]: Leaving directory `/builds/slave/c-cen-t-lnx/build/objdir/memory/replace/logalloc/replay'
make[1]: Target `recurse_check' not remade because of errors.
make[1]: Leaving directory `/builds/slave/c-cen-t-lnx/build/objdir'
make: *** [check] Error 2
DEBUG: shell: unmounting all filesystems

not really sure what's going on atm...   just filing this.

Comment 1

[Edmund Wong (:ewong)]

At the beginning there's also an error:

Starting 'mach python-test' with -j4
Error loading mozconfig: /builds/slave/c-cen-t-lnx-dbg/build/.mozconfig

Evaluation of your mozconfig exited with an error. This could be triggered
by a command inside your mozconfig failing. Please change your mozconfig
to not error and/or to catch errors in executed commands.

mozconfig output:

------BEGIN_MK_OPTION
AUTOCLOBBER=1
------END_MK_OPTION
------BEGIN_AC_OPTION
--enable-crashreporter
------END_AC_OPTION
------BEGIN_AC_OPTION
--enable-js-shell
------END_AC_OPTION
------BEGIN_MK_OPTION
export MOZ_AUTOMATION_BUILD_SYMBOLS=1
------END_MK_OPTION
------BEGIN_MK_OPTION
export MOZ_AUTOMATION_L10N_CHECK=1
------END_MK_OPTION
------BEGIN_MK_OPTION
export MOZ_AUTOMATION_PACKAGE=1
------END_MK_OPTION
------BEGIN_MK_OPTION
export MOZ_AUTOMATION_PACKAGE_TESTS=1
------END_MK_OPTION
------BEGIN_MK_OPTION
export MOZ_AUTOMATION_INSTALLER=0
------END_MK_OPTION
------BEGIN_MK_OPTION
export MOZ_AUTOMATION_UPDATE_PACKAGING=0
------END_MK_OPTION
------BEGIN_MK_OPTION
export MOZ_AUTOMATION_PACKAGE_GENERATED_SOURCES=1
------END_MK_OPTION
------BEGIN_MK_OPTION
export MOZ_AUTOMATION_UPLOAD=1
------END_MK_OPTION
------BEGIN_MK_OPTION
export MOZ_AUTOMATION_UPLOAD_SYMBOLS=0
------END_MK_OPTION
------BEGIN_AC_OPTION
--enable-rust-simd
------END_AC_OPTION
------BEGIN_MK_OPTION
PATH=/builds/slave/c-cen-t-lnx-dbg/build/gcc/bin:/tools/buildbot/bin:/usr/local/bin:/usr/lib/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin
------END_MK_OPTION
------BEGIN_AC_OPTION
--enable-elf-hack
------END_AC_OPTION
------BEGIN_AC_OPTION
--enable-stdcxx-compat
------END_AC_OPTION
------BEGIN_MK_OPTION
export LD_LIBRARY_PATH=objdir/mozilla/dist/bin:/builds/slave/c-cen-t-lnx-dbg/build/gcc/lib64:/builds/slave/c-cen-t-lnx-dbg/build/gcc/lib:/builds/slave/c-cen-t-lnx-dbg/build/gtk3/usr/local/lib:/builds/slave/c-cen-t-lnx-dbg/build/gcc/lib64:/builds/slave/c-cen-t-lnx-dbg/build/gcc/lib
------END_MK_OPTION
------BEGIN_AC_OPTION
--enable-default-toolkit=cairo-gtk3
------END_AC_OPTION
------BEGIN_MK_OPTION
export FONTCONFIG_PATH=/builds/slave/c-cen-t-lnx-dbg/build/gtk3/usr/local/etc/fonts
------END_MK_OPTION
------BEGIN_MK_OPTION
export PANGO_SYSCONFDIR=/builds/slave/c-cen-t-lnx-dbg/build/gtk3/usr/local/etc
------END_MK_OPTION
------BEGIN_MK_OPTION
export PANGO_LIBDIR=/builds/slave/c-cen-t-lnx-dbg/build/gtk3/usr/local/lib
------END_MK_OPTION
------BEGIN_MK_OPTION
export GDK_PIXBUF_MODULE_FILE=/builds/slave/c-cen-t-lnx-dbg/build/gtk3/usr/local/lib/gdk-pixbuf-2.0/2.10.0/loaders.cache
------END_MK_OPTION
------BEGIN_MK_OPTION
export GDK_PIXBUF_MODULEDIR=/builds/slave/c-cen-t-lnx-dbg/build/gtk3/usr/local/lib/gdk-pixbuf-2.0/2.10.0/loaders
------END_MK_OPTION
------BEGIN_MK_OPTION
export LD_LIBRARY_PATH=objdir/mozilla/dist/bin:/builds/slave/c-cen-t-lnx-dbg/build/gcc/lib64:/builds/slave/c-cen-t-lnx-dbg/build/gcc/lib:/builds/slave/c-cen-t-lnx-dbg/build/gtk3/usr/local/lib:/builds/slave/c-cen-t-lnx-dbg/build/gcc/lib64:/builds/slave/c-cen-t-lnx-dbg/build/gcc/lib:/builds/slave/c-cen-t-lnx-dbg/build/gtk3/usr/local/lib
------END_MK_OPTION
------BEGIN_MK_OPTION
CONFIG_GUESS=i686-pc-linux
------END_MK_OPTION
------BEGIN_AC_OPTION
--target=i686-pc-linux
------END_AC_OPTION
------BEGIN_AC_OPTION
--host=i686-pc-linux
------END_AC_OPTION
------BEGIN_AC_OPTION
--x-libraries=/usr/lib
------END_AC_OPTION
/builds/slave/c-cen-t-lnx-dbg/build/.mozconfig: line 3: /builds/slave/c-cen-t-lnx-dbg/build/mozilla/suite/config/mozconfigs/common: No such file or directory
make: *** [check] Error 1

And I think it's this that's the problem.  

looking at our .mozconfig, we use $topsrcdir and like everytime we use that variable, we're
screwed up as it would (during the build process) assume topsrcdir == ./mozilla instead of ./

Comment 2

[Edmund Wong (:ewong)]

Attachment: [puppet] need to update the mercurial tools

This should allow our puppet setup to grab the updated mercurial packages.
(currently at 3.1.2)

Comment 3

[Edmund Wong (:ewong)]

Attachment: [puppet] need to update the mercurial tools

helps if I got the right patch.

Comment 4

[Edmund Wong (:ewong)]

Attachment: [puppet] upgrade two python2.7.12 and use sea-puppet's yum repo and not mockbuild's repo

mockbuild's yum repo is seriously out of date.. sea-puppet's repo
has at least 2.7.12 python as well as the python27-mercurial version
of 3.9.1.

Comment 5

[Frank-Rainer Grahl (:frg)]

Comment on attachment 8927196 [details] [diff] [review]
[puppet] upgrade two python2.7.12 and use sea-puppet's yum repo and not mockbuild's repo

> s well as the python27-mercurial version of 3.9.1.

3.9.1 or 3.7.3? Is this a typo?

> b/modules/packages/manifests/mozilla/py27_mercurial-dmg.sh
> ...
> -version=3.2.1
> +version=3.7.3

r+ with question answered

Comment 6

[Edmund Wong (:ewong)]

(In reply to Frank-Rainer Grahl (:frg) from comment #5)
> Comment on attachment 8927196 [details] [diff] [review]
> [puppet] upgrade two python2.7.12 and use sea-puppet's yum repo and not
> mockbuild's repo
> 
> > s well as the python27-mercurial version of 3.9.1.
> 
> 3.9.1 or 3.7.3? Is this a typo?
> 
> > b/modules/packages/manifests/mozilla/py27_mercurial-dmg.sh
> > ...
> > -version=3.2.1
> > +version=3.7.3
> 
> r+ with question answered

My mistake in not specifying.  It is 3.9.1.. but we don't have 3.9.1 in the DMGs folder
of our puppet server for dmgs.. we only have 3.7.3 (max)  (we should build a 3.9.1 for
dmg someday).  That py27_mercurial-dmg.sh change was to make it updated.  Sorry for
not stating why that was changed.

Comment 7

[Edmund Wong (:ewong)]

Attachment: [puppet] use the correct mockbuild repo

the previous (and also a subsequent patch) were bad due to the
following reasons:

1) first patch : bad because python 2.7.12 didn't exist on the server.
   - Need to build a 2.7.12 (though I'd probably go with 2.7.14) build
     package

2) subsequent patch failed due to invalid repo url

this patch is basically changing the mockbuild url to use sea-puppet.

Comment 8

[Edmund Wong (:ewong)]

Backed out all previous pushes.

https://hg.mozilla.org/SeaMonkey/puppet/rev/6ef1f29c3db0
https://hg.mozilla.org/SeaMonkey/puppet/rev/2a1a6f2eb670
https://hg.mozilla.org/SeaMonkey/puppet/rev/aca04e7d5982
https://hg.mozilla.org/SeaMonkey/puppet/rev/e46cdae886a2
https://hg.mozilla.org/SeaMonkey/puppet/rev/1f4055951024
https://hg.mozilla.org/SeaMonkey/puppet/rev/351f814b9cbb
https://hg.mozilla.org/SeaMonkey/puppet/rev/08f096783e46
https://hg.mozilla.org/SeaMonkey/puppet/rev/56fd2e7f25a8
https://hg.mozilla.org/SeaMonkey/puppet/rev/0585d6064e8c
https://hg.mozilla.org/SeaMonkey/puppet/rev/288bd0511d7d
https://hg.mozilla.org/SeaMonkey/puppet/rev/776db6cf91df

Comment 9

[Edmund Wong (:ewong)]

Attachment: [puppet] Use the correct repos for mercurial 3.9.1

Comment 10

[Edmund Wong (:ewong)]

Attachment: [configs] specify the correct mercurial packages.

Comment 11

[Frank-Rainer Grahl (:frg)]

Comment on attachment 8927525 [details] [diff] [review]
[puppet] Use the correct repos for mercurial 3.9.1

modules/runner/files/mockbuild-config-templates/mozilla-centos6-i386.cfg

> +name=centos6-sea-mirros

mirros or mirrors?

modules/runner/files/mockbuild-config-templates/mozilla-centos6-x86_64.cfg

> +
> [centos6]

Is the extra line intentional?

> +
> for tpl in $TEMPLATES/*.cfg; do
> ...

If possible a comment line what the changed block actually does would be great. Otherwise I just believe you :)

r+ with questions answered

Comment 12

[Edmund Wong (:ewong)]

Attachment: [puppet] Use the correct repos for mercurial 3.9.1 (with nits fixed and comments)

Comment 13

[Frank-Rainer Grahl (:frg)]

Comment on attachment 8927552 [details] [diff] [review]
[puppet] Use the correct repos for mercurial 3.9.1 (with nits fixed and comments)

lgtm

Comment 14

[Edmund Wong (:ewong)]

Attachment: [puppet] forgot to also change i686 package repos.

Comment 15

[Edmund Wong (:ewong)]

Attachment: [configs] change to use 3.9.1 for i686 and use 1.7.1-2 git

cannot seem to use just 'git' as it takes the 'latest' one which is
2.x  and that messes up the lib requirements.

Comment 16

[Frank-Rainer Grahl (:frg)]

Comment on attachment 8927563 [details] [diff] [review]
[configs] change to use 3.9.1 for i686 and use 1.7.1-2 git

Already had a zlib-devel package: 'zlib-devel.i686',

I assume no zlib itself which was added for x86_64 is needed.

Comment 17

[Edmund Wong (:ewong)]

Attachment: [puppet] use sea-puppet repo first and foremost .. failover to mockbuild

backed out the previous patches

Comment 18

[Frank-Rainer Grahl (:frg)]

Comment on attachment 8927633 [details] [diff] [review]
[puppet] use sea-puppet repo first and foremost .. failover to mockbuild

we got an empty patch again :)

Comment 19

[Edmund Wong (:ewong)]

Attachment: [puppet] use the appropriate repos

Comment 20

[Edmund Wong (:ewong)]

Attachment: [config] use 3.9.1 python mercurial and 1.7.1 for git

Comment 21

[Edmund Wong (:ewong)]

Attachment: [puppet] ssh clients were lost somehow... need to add ssh binaries to chroot env

Comment 24

[Edmund Wong (:ewong)]

Attachment: [puppet] rsync needed as well.

Comment 25

[Edmund Wong (:ewong)]

We no longer use puppet.