Closed Bug 932354 Opened 9 years ago Closed 3 years ago

mozilla-central patches applied twice on Thunderbird Windows try server, try build fails


(MailNews Core :: Build Config, defect)

Not set


(Not tracked)



(Reporter: mcsmurf, Assigned: mcsmurf)



(2 files)

I've noticed that when you want to try a mozilla-central patch on the Thunderbird try server (see it applies the patch twice with the hg import command. Depending on the patch this will usually fail or produce bad result (non-working build). 
I've created an example build at to test this. I modified the in comm-central and the AUTHORS file in mozilla-central to test this.
I think this here might be a bug in, but maybe this is also a try server issue, not sure about that. I've filed it in the comm-central build system for now.
This is the log file from try server. The output is a bit confusing (at least to me), it looks like stdout output appears before the "Executing command" output, but then stderr appears after it (at least it looks like that to me).
Important parts of the log file:
'python' 'e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build/build/pymake/' '-f' '' 'build' 'MOZ_BUILD_DATE=20131029060024'

applying e:\builds\moz2_slave\tb-try-c-cen-w32-0000000000000\build\mozilla-test2.patch
Executing command: ['hg', 'clone', '--time', '', 'e:\\builds\\moz2_slave\\tb-try-c-cen-w32-0000000000000\\build\\mozilla']
Executing command: ['hg', 'update', '-r', 'default', '-R', 'e:\\builds\\moz2_slave\\tb-try-c-cen-w32-0000000000000\\build\\mozilla', '--time']
Executing command: ['hg', 'import', '-R', 'mozilla', '-m', 'local patch from e:\\builds\\moz2_slave\\tb-try-c-cen-w32-0000000000000\\build\\mozilla-test2.patch', '--no-commit', '--force', 'e:\\builds\\moz2_slave\\tb-try-c-cen-w32-0000000000000\\build\\mozilla-test2.patch'][0]: Entering directory 'e:\builds\moz2_slave\tb-try-c-cen-w32-0000000000000\build'
e:\builds\moz2_slave\tb-try-c-cen-w32-0000000000000\build\config\makefiles\$ pymake.builtins mkdir -p "objdir-tb/"
e:\builds\moz2_slave\tb-try-c-cen-w32-0000000000000\build\$ (  echo "export LIB=c:\\Program Files (x86)\\Windows Kits\\8.0\\Lib\\win8\\um\\x86;c:\\tools\\msvs10\\vc\\lib;c:\\tools\\msvs10\\vc\\atlmfc\\lib;c:\\tools\\sdks\\dx10\\lib";  echo "export LIBPATH=c:\\Program Files (x86)\\Windows Kits\\8.0\\Lib\\win8\\um\\x86;c:\\tools\\msvs10\\vc\\lib;c:\\tools\\msvs10\\vc\\atlmfc\\lib;c:\\tools\\sdks\\dx10\\lib";  echo "export PATH=c:\\Program Files (x86)\\Windows Kits\\8.0\\bin\\x86;c:\\tools\\msvs10\\Common7\\IDE;c:\\tools\\msvs10\\VC\\BIN;c:\\tools\\msvs10\\Common7\\Tools;c:\\tools\\msvs10\\VC\\VCPackages;c:\\mozilla-build\\moztools;c:\\mozilla-build\\python27;c:\\mozilla-build\\buildbotve\\scripts;C:\\mozilla-build\\msys\\local\\bin;c:\\mozilla-build\\wget;c:\\mozilla-build\\7zip;c:\\mozilla-build\\blat261\\full;c:\\mozilla-build\\python;c:\\mozilla-build\\svn-win32-1.6.3\\bin;c:\\mozilla-build\\upx203w;c:\\mozilla-build\\emacs-22.3\\bin;c:\\mozilla-build\\info-zip;c:\\mozilla-build\\nsis-2.22;c:\\mozilla-build\\nsis-2.33u;c:\\mozilla-build\\nsis-2.46u;c:\\mozilla-build\\wix-351728;c:\\mozilla-build\\hg;c:\\mozilla-build\\python\\Scripts;c:\\mozilla-build\\kdiff3;c:\\mozilla-build\\yasm;.;C:\\mozilla-build\\msys\\local\\bin;C:\\mozilla-build\\msys\\mingw\\bin;C:\\mozilla-build\\msys\\bin;c:\\Program Files (x86)\\Microsoft Visual Studio 9.0\\VC\\BIN\\amd64;c:\\Windows\\Microsoft.NET\\Framework64\\v3.5;c:\\Windows\\Microsoft.NET\\Framework64\\v3.5\\Microsoft .NET Framework 3.5 (Pre-Release Version);c:\\Windows\\Microsoft.NET\\Framework64\\v2.0.50727;c:\\Program Files (x86)\\Microsoft Visual Studio 9.0\\VC\\VCPackages;c:\\Tools\\msvs9\\Common7\\IDE;c:\\Tools\\msvs9\\Common7\\Tools;c:\\Tools\\msvs9\\Common7\\Tools\\bin;c:\\Tools\\sdks\\v7.0\\bin\\x64;c:\\Tools\\sdks\\v7.0\\bin\\win64\\x64;c:\\Tools\\sdks\\v7.0\\bin;c:\\Windows\\System32;c:\\Windows;c:\\Windows\\System32\\Wbem;c:\\mozilla-build\\moztools-x64\\bin;c:\\mozilla-build\\vim\\vim72";  echo "export INCLUDE=c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\shared;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\um;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt\\wrl;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt\\wrl\\wrappers;c:\\tools\\msvs10\\vc\\include;c:\\tools\\msvs10\\vc\\atlmfc\\include;c:\\tools\\sdks\\dx10\\include";  echo "export WIN32_REDIST_DIR=c:/tools/msvs10/VC/redist/x86/Microsoft.VC100.CRT";  echo "export MOZ_TOOLS=C:/mozilla-build/moztools";  echo "export INCLUDE=c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\shared;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\um;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt\\wrl;c:\\Program Files (x86)\\Windows Kits\\8.0\\include\\winrt\\wrl\\wrappers;c:\\tools\\msvs10\\vc\\include;c:\\tools\\msvs10\\vc\\atlmfc\\include;c:\\tools\\sdks\\dx10\\include;c:\\Office 2010 Developer Resources\\Outlook 2010 MAPI Headers"; ) > objdir-tb/
e:\builds\moz2_slave\tb-try-c-cen-w32-0000000000000\build\$ python e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build/ checkout --hg-options='--time' --apply-patches --skip-chatzilla --skip-comm --skip-inspector --skip-venkman --tinderbox-print
pulling from

applying e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build\mozilla-test2.patch
patching file AUTHORS
Hunk #1 FAILED at 5
1 out of 1 hunks FAILED -- saving rejects to file AUTHORS.rej
abort: patch failed to apply
Executing command: ['hg', 'pull', '-R', 'e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build\\mozilla', '--time', '']
Executing command: ['hg', 'update', '-r', 'default', '-R', 'e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build\\mozilla', '--time']

Note how it seems to run twice, at least it looks like this to me. Not sure why this happens.
I forgot the last line:
Executing command: ['hg', 'import', '-R', 'mozilla', '-m', 'local patch from e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build\\mozilla-test2.patch', '--no-commit', '--force', 'e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build\\mozilla-test2.patch']

This one is also important ;) this is the second patch apply that fails as the patch has already been applied before.
I think I have an idea now why this happens: The file in this command
'python' 'e:/builds/moz2_slave/tb-try-c-cen-w32-0000000000000/build/build/pymake/' '-f' '' 'build' 'MOZ_BUILD_DATE=20131029060024'

is a wrapper for the mozilla-central pymake. It runs checkout with the supplied This call will check out mozilla-central (to get the mozilla-central!) and also apply the patches (this is not the desired behavior in this case). After it got the new file, it will run again because of the "ALWAYS_RUN_CLIENT_PY=1" directive in the official Thunderbird .mozconfig files.
So basically this issue is Windows only as the other platforms do not use pymake. Maybe the wrapper should use some hardcoded and not the one supplied by the user? What would get lost is the ability to test mozilla-central changes on the try server during source checkout I think (as then it would not run --apply-patches before running again). The only other problem I see is when someone wants to build SeaMonkey with "python build/pymake/ -f" as this needs different compared to Thunderbird. But then I wonder if building this way really needs to be supported.
Summary: mozilla-central patches applied twice on Thunderbird try server, try build fails → mozilla-central patches applied twice on Thunderbird Windows try server, try build fails
Actually I think my comment on SeaMonkey is wrong here. After all it only uses those arguments to fetch the mozilla-central ( repository to execute after that (which then checks out the source, if needed or requested). So I think setting static inside comm-central would be perfectly fine.
Attached patch PatchSplinter Review
This patch hardcodes the hg options used by comm-central to check out the real mozilla-central in case it's not there yet. This fixes the "mozilla-central patches applied twice on a try build" problem. This should not affect SeaMonkey in any bad way as itself takes care of running to check out the SeaMonkey source repositories. It never relied on for that (as is Windows-only anyway).
Assignee: nobody → bugzilla
Attachment #825914 - Flags: review?(Pidgeot18)
Comment on attachment 825914 [details] [diff] [review]

I'm redirecting this to Mark, since this affects the release engineering side of things more than anything else.
Attachment #825914 - Flags: review?(Pidgeot18) → review?(mbanner)
Comment on attachment 825914 [details] [diff] [review]

Aha, now I remember.

I think we should strip --apply-patches out of the arguments in (i.e. still read from the file), and then obviously leave to do the actual apply.
Attachment #825914 - Flags: review?(mbanner) → review-
Closed: 3 years ago
Resolution: --- → INACTIVE
You need to log in before you can comment on or make changes to this bug.