932354 - mozilla-central patches applied twice on Thunderbird Windows try server, try build fails

Status: RESOLVED INVALID

(MailNews Core :: Build Config, defect)

Description

[Frank Wein [:mcsmurf]]

I've noticed that when you want to try a mozilla-central patch on the Thunderbird try server (see https://wiki.mozilla.org/Thunderbird/Infrastructure/TryServer#Pushing_mozilla-central_patches) it applies the patch twice with the hg import command. Depending on the patch this will usually fail or produce bad result (non-working build). 
I've created an example build at https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=8c22d6e37b4e to test this. I modified the client.py-args in comm-central and the AUTHORS file in mozilla-central to test this.
I think this here might be a bug in client.py, but maybe this is also a try server issue, not sure about that. I've filed it in the comm-central build system for now.

Comment 1

[Frank Wein [:mcsmurf]]

Attachment: Build log from try server (Windows XP)

This is the log file from try server. The output is a bit confusing (at least to me), it looks like stdout output appears before the "Executing command" output, but then stderr appears after it (at least it looks like that to me).
Important parts of the log file:
'python' 'e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build/build/pymake/make.py' '-f' 'client.mk' 'build' 'MOZ_BUILD_DATE=20131029060024'
[...]

applying e:\builds\moz2_slave\tb-try-c-cen-w32-0000000000000\build\mozilla-test2.patch
Executing command: ['hg', 'clone', '--time', 'http://hg.mozilla.org/mozilla-central/', 'e:\\builds\\moz2_slave\\tb-try-c-cen-w32-0000000000000\\build\\mozilla']
Executing command: ['hg', 'update', '-r', 'default', '-R', 'e:\\builds\\moz2_slave\\tb-try-c-cen-w32-0000000000000\\build\\mozilla', '--time']
[...]
Executing command: ['hg', 'import', '-R', 'mozilla', '-m', 'local patch from e:\\builds\\moz2_slave\\tb-try-c-cen-w32-0000000000000\\build\\mozilla-test2.patch', '--no-commit', '--force', 'e:\\builds\\moz2_slave\\tb-try-c-cen-w32-0000000000000\\build\\mozilla-test2.patch']
make.py[0]: Entering directory 'e:\builds\moz2_slave\tb-try-c-cen-w32-0000000000000\build'
e:\builds\moz2_slave\tb-try-c-cen-w32-0000000000000\build\config\makefiles\autotargets.mk:59:0$ pymake.builtins mkdir -p "objdir-tb/"
e:\builds\moz2_slave\tb-try-c-cen-w32-0000000000000\build\client.mk:193:0$ (  echo "export LIB=c:\\Program Files (x86)\\Windows Kits\\8.0\\Lib\\win8\\um\\x86;c:\\tools\\msvs10\\vc\\lib;c:\\tools\\msvs10\\vc\\atlmfc\\lib;c:\\tools\\sdks\\dx10\\lib";  echo "export LIBPATH=c:\\Program Files (x86)\\Windows Kits\\8.0\\Lib\\win8\\um\\x86;c:\\tools\\msvs10\\vc\\lib;c:\\tools\\msvs10\\vc\\atlmfc\\lib;c:\\tools\\sdks\\dx10\\lib";  echo "export PATH=c:\\Program Files (x86)\\Windows Kits\\8.0\\bin\\x86;c:\\tools\\msvs10\\Common7\\IDE;c:\\tools\\msvs10\\VC\\BIN;c:\\tools\\msvs10\\Common7\\Tools;c:\\tools\\msvs10\\VC\\VCPackages;c:\\mozilla-build\\moztools;c:\\mozilla-build\\python27;c:\\mozilla-build\\buildbotve\\scripts;C:\\mozilla-build\\msys\\local\\bin;c:\\mozilla-build\\wget;c:\\mozilla-build\\7zip;c:\\mozilla-build\\blat261\\full;c:\\mozilla-build\\python;c:\\mozilla-build\\svn-win32-1.6.3\\bin;c:\\mozilla-build\\upx203w;c:\\mozilla-build\\emacs-22.3\\bin;c:\\mozilla-build\\info-zip;c:\\mozilla-build\\nsis-2.22;c:\\mozilla-build\\nsis-2.33u;c:\\mozilla-build\\nsis-2.46u;c:\\mozilla-build\\wix-351728;c:\\mozilla-build\\hg;c:\\mozilla-build\\python\\Scripts;c:\\mozilla-build\\kdiff3;c:\\mozilla-build\\yasm;.;C:\\mozilla-build\\msys\\local\\bin;C:\\mozilla-build\\msys\\mingw\\bin;C:\\mozilla-build\\msys\\bin;c:\\Program Files (x86)\\Microsoft Visual Studio 9.0\\VC\\BIN\\amd64;c:\\Windows\\Microsoft.NET\\Framework64\\v3.5;c:\\Windows\\Microsoft.NET\\Framework64\\v3.5\\Microsoft .NET Framework 3.5 (Pre-Release Version);c:\\Windows\\Microsoft.NET\\Framework64\\v2.0.50727;c:\\Program Files (x86)\\Microsoft Visual Studio 9.0\\VC\\VCPackages;c:\\Tools\\msvs9\\Common7\\IDE;c:\\Tools\\msvs9\\Common7\\Tools;c:\\Tools\\msvs9\\Common7\\Tools\\bin;c:\\Tools\\sdks\\v7.0\\bin\\x64;c:\\Tools\\sdks\\v7.0\\bin\\win64\\x64;c:\\Tools\\sdks\\v7.0\\bin;c:\\Windows\\System32;c:\\Windows;c:\\Windows\\System32\\Wbem;c:\\mozilla-build\\moztools-x64\\bin;c:\\mozilla-build\\vim\\vim72";  echo "export INCLUDE=c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\shared;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\um;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt\\wrl;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt\\wrl\\wrappers;c:\\tools\\msvs10\\vc\\include;c:\\tools\\msvs10\\vc\\atlmfc\\include;c:\\tools\\sdks\\dx10\\include";  echo "export WIN32_REDIST_DIR=c:/tools/msvs10/VC/redist/x86/Microsoft.VC100.CRT";  echo "export MOZ_TOOLS=C:/mozilla-build/moztools";  echo "export INCLUDE=c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\shared;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\um;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt\\wrl;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt\\wrl\\wrappers;c:\\tools\\msvs10\\vc\\include;c:\\tools\\msvs10\\vc\\atlmfc\\include;c:\\tools\\sdks\\dx10\\include;c:\\Office 2010 Developer Resources\\Outlook 2010 MAPI Headers"; ) > objdir-tb/.mozconfig.mk
e:\builds\moz2_slave\tb-try-c-cen-w32-0000000000000\build\client.mk:455:0$ python e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build/client.py checkout --hg-options='--time' --apply-patches --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print
pulling from http://hg.mozilla.org/mozilla-central/
[...]

applying e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build\mozilla-test2.patch
patching file AUTHORS
Hunk #1 FAILED at 5
1 out of 1 hunks FAILED -- saving rejects to file AUTHORS.rej
abort: patch failed to apply
Executing command: ['hg', 'pull', '-R', 'e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build\\mozilla', '--time', 'http://hg.mozilla.org/mozilla-central/']
Executing command: ['hg', 'update', '-r', 'default', '-R', 'e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build\\mozilla', '--time']

Note how it seems to run client.py twice, at least it looks like this to me. Not sure why this happens.

Comment 2

[Frank Wein [:mcsmurf]]

I forgot the last line:
Executing command: ['hg', 'import', '-R', 'mozilla', '-m', 'local patch from e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build\\mozilla-test2.patch', '--no-commit', '--force', 'e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build\\mozilla-test2.patch']

This one is also important ;) this is the second patch apply that fails as the patch has already been applied before.

Comment 3

[Frank Wein [:mcsmurf]]

I think I have an idea now why this happens: The make.py file in this command
'python' 'e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build/build/pymake/make.py' '-f' 'client.mk' 'build' 'MOZ_BUILD_DATE=20131029060024'

is a wrapper for the mozilla-central pymake. It runs client.py checkout with the supplied client.py-args. This call will check out mozilla-central (to get the mozilla-central make.py!) and also apply the patches (this is not the desired behavior in this case). After it got the new make.py file, it will run client.py again because of the "ALWAYS_RUN_CLIENT_PY=1" directive in the official Thunderbird .mozconfig files.

Comment 4

[Frank Wein [:mcsmurf]]

So basically this issue is Windows only as the other platforms do not use pymake. Maybe the wrapper should use some hardcoded client.py-args and not the one supplied by the user? What would get lost is the ability to test mozilla-central make.py changes on the try server during source checkout I think (as then it would not run --apply-patches before running client.py again). The only other problem I see is when someone wants to build SeaMonkey with "python build/pymake/make.py -f client.mk" as this needs different client.py-args compared to Thunderbird. But then I wonder if building this way really needs to be supported.

Comment 5

[Frank Wein [:mcsmurf]]

Actually I think my comment on SeaMonkey is wrong here. After all it only uses those arguments to fetch the mozilla-central (make.py) repository to execute client.mk after that (which then checks out the source, if needed or requested). So I think setting static client.py-args inside comm-central make.py would be perfectly fine.

Comment 6

[Frank Wein [:mcsmurf]]

Attachment: Patch

This patch hardcodes the hg options used by comm-central make.py to check out the real mozilla-central make.py in case it's not there yet. This fixes the "mozilla-central patches applied twice on a try build" problem. This should not affect SeaMonkey in any bad way as client.mk itself takes care of running client.py to check out the SeaMonkey source repositories. It never relied on make.py for that (as make.py is Windows-only anyway).

Comment 7

[Joshua Cranmer [:jcranmer]]

Comment on attachment 825914 [details] [diff] [review]
Patch

I'm redirecting this to Mark, since this affects the release engineering side of things more than anything else.

Comment 8

[Mark Banner (:standard8)]

Comment on attachment 825914 [details] [diff] [review]
Patch

Aha, now I remember.

I think we should strip --apply-patches out of the arguments in make.py (i.e. still read from the client.py-args file), and then obviously leave client.mk to do the actual apply.