1429957 - searchfox: Run the win32 index job with 64-bit toolchains

Status: RESOLVED FIXED

(Webtools :: Searchfox, enhancement)

Description

[Ralph Giles (:rillian)]

All our builds run on 64-bit hosts, use 64-bit toolchains, and cross-compile for 32-bit targets.

The win32 searchfox indexing job is an exception. It would be better to use the same clang-cl and rust packages as the official builds to reduce both variance and dependencies.

Comment 1

[Ralph Giles (:rillian)]

Attachment: Bug 1429957 - searchfox: Use 64-bit toolchains for the win32 index task.

We generally build win32 cross compiling from win64. This
change uses the same toolchains for the index job as we
use for builds, reducing variance and letting us remove
a dependent packaging task.

Review commit: https://reviewboard.mozilla.org/r/212224/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/212224/

Comment 2

[Ralph Giles (:rillian)]

With this patch, the index job fails, but from the comments and tier-2 mark, maybe this is getting as far as it's expected to? I don't see it running on m-c commits, so I don't have a baseline.

> z:/task_1515708560/build/src/obj-firefox/_virtualenv/Scripts/python.exe -m mozbuild.action.cl z:/task_1515708560/build/src/clang/bin/clang.exe --driver-mode=cl -m32 -FoUnified_cpp_build_clang-plugin1.obj -c -DDEBUG=1 -Iz:/task_1515708560/build/src/build/clang-plugin -Iz:/task_1515708560/build/src/obj-firefox/build/clang-plugin -Iz:/task_1515708560/build/src/obj-firefox/dist/include -Iz:/task_1515708560/build/src/obj-firefox/dist/include/nspr -Iz:/task_1515708560/build/src/obj-firefox/dist/include/nss -MD -Qunused-arguments -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Iz:/task_1515708560/build/src/clang/include -fms-compatibility-version=19.11.25547 -Xclang -std=c++14 -Zc:inline -Zc:strictStrings -Oi -Zc:rvalueCast -Brepro -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wcovered-switch-default -Wdelete-non-virtual-dtor -Wstring-conversion -MT -O2 -Ob2 -DNDEBUG -EHs-c- -GR- -D_DEBUG_POINTER_IMPL= -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -DUNICODE -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DHAVE_NEW_ASTMATCHER_NAMES -DHAS_ACCEPTS_IGNORINGPARENIMPCASTS -GR- -EHsc -Oy-    -Fdgenerated.pdb -FS  z:/task_1515708560/build/src/obj-firefox/build/clang-plugin/Unified_cpp_build_clang-plugin1.cpp
>   In file included from z:/task_1515708560/build/src/obj-firefox/build/clang-plugin/Unified_cpp_build_clang-plugin1.cpp:137:
>   z:/task_1515708560/build/src/build/clang-plugin/mozsearch-plugin/FileOperations.cpp(53,27):  error: use of undeclared identifier 'filename'
>     std::string Hash = Hash(filename);

Comment 3

[Kartikaya Gupta (email:[e-mail])]

For reference: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ea6c7093f3b87f69782d4169429d84a13a029477

I don't know if billm is still doing reviews but this patch looks ok to me. We haven't yet gotten the plugin to run successfully on Windows so that's why it's not running by default there. I've been trying to get that green but haven't had much time to actually work on it.

Comment 4

[Kartikaya Gupta (email:[e-mail])]

Comment on attachment 8942019 [details]
Bug 1429957 - searchfox: Use 64-bit toolchains for the win32 index task.

https://reviewboard.mozilla.org/r/212224/#review218128

Comment 5

[Kartikaya Gupta (email:[e-mail])]

Actually I'm not sure why we are doing a 32-bit build at all, we should probably change that to 64-bit.

Comment 6

[Kartikaya Gupta (email:[e-mail])]

(In reply to Ralph Giles (:rillian) | needinfo me from comment #2)
> >     std::string Hash = Hash(filename);

Patch to fix the compilation issues is in bug 1430645.

Comment 7

[Kartikaya Gupta (email:[e-mail])]

Attachment: Move searchfox indexer to win64

Here's a patch that moves the searchfox indexing job from 32-bit to 64-bit. It is a superset of your patch, and I'd rather land this. However if you'd rather push this into another bug that's fine by me. Let me know either way and I'll clean up the patch and find somebody to review.

Comment 8

[Ralph Giles (:rillian)]

Comment on attachment 8942725 [details] [diff] [review]
Move searchfox indexer to win64

Review of attachment 8942725 [details] [diff] [review]:
-----------------------------------------------------------------

This is fine with me! I was just interested in removing the win32-rust dep, so I'm happy for you to take over the bug. r=me for the build parts.

Comment 9

[Kartikaya Gupta (email:[e-mail])]

Attachment: Move searchfox indexer to win64

r? to dustin and chmanchester since they reviewed the original landing of the win32 job in bug 1418188.

Carrying r=rillian from previous patch.

Comment 10

[Kartikaya Gupta (email:[e-mail])]

Attachment: Move searchfox indexer to win64

Whoops, wrong file

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/350aac55ed64
Move searchfox indexing job from win32 to win64. r=rillian,dustin,chmanchester

Comment 12

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/350aac55ed64