switch win64 builds to use mozharness mach

RESOLVED FIXED

Status

defect
RESOLVED FIXED
5 years ago
Last year

People

(Reporter: jlund, Assigned: jlund)

Tracking

Dependency tree / graph

Firefox Tracking Flags

(firefox38 fixed, firefox39 fixed)

Details

Attachments

(10 attachments)

5.08 KB, patch
catlee
: review+
jlund
: checked-in+
Details | Diff | Splinter Review
5.46 KB, patch
catlee
: review+
jlund
: checked-in+
Details | Diff | Splinter Review
669 bytes, patch
mshal
: review+
Details | Diff | Splinter Review
3.09 KB, patch
catlee
: review+
jlund
: checked-in+
Details | Diff | Splinter Review
309 bytes, patch
catlee
: review+
Details | Diff | Splinter Review
309 bytes, patch
catlee
: review+
Details | Diff | Splinter Review
541 bytes, patch
mshal
: review+
jlund
: checked-in+
Details | Diff | Splinter Review
308 bytes, patch
jlund
: review+
Details | Diff | Splinter Review
309 bytes, patch
RyanVM
: review+
Details | Diff | Splinter Review
616 bytes, patch
mshal
: review+
jlund
: checked-in+
Details | Diff | Splinter Review
Assignee

Description

5 years ago
No description provided.
Assignee

Comment 1

4 years ago
the buildbot parts needed to start playing with win64

note: I needed to add reboot and python interpreter to win64-debug only since it was already there (for l10n stuff) in win64 opt

[dev_master]jlund@Hastings163:~/devel/mozilla/dev_master/build-master
> peop ~/tmp/win64_mh_cedar-dump_master.diff
win64_mh_cedar-dump_master.diff
http://people.mozilla.org/~jlund/win64_mh_cedar-dump_master.diff

[dev_master]jlund@Hastings163:~/devel/mozilla/dev_master/build-master
> peop ~/tmp/win64_mh_cedar-builder_list.diff
win64_mh_cedar-builder_list.diff
http://people.mozilla.org/~jlund/win64_mh_cedar-builder_list.diff
Attachment #8559360 - Flags: review?(catlee)
Assignee

Comment 2

4 years ago
between this and the previous bbot-config attachment, we should have win64 builds on cedar. who knows, it could 'just work'.

Note: I removed the nsis refs from both the win64 and win64-debug PATHS even though nsis still exists in win64-debug.

here is how this differs from the win32 configs:
http://people.mozilla.org/~jlund/win64_config_differences.diff
Attachment #8559381 - Flags: review?(catlee)
Comment on attachment 8559360 [details] [diff] [review]
150204_1094364_win64_mh_cedar-bbot-configs.diff

Review of attachment 8559360 [details] [diff] [review]:
-----------------------------------------------------------------

<stamp />
Attachment #8559360 - Flags: review?(catlee) → review+
Comment on attachment 8559381 [details] [diff] [review]
150204_1094364_win64_mh_cedar-mh.diff

Review of attachment 8559381 [details] [diff] [review]:
-----------------------------------------------------------------

::: configs/builds/releng_base_windows_64_builds.py
@@ +59,5 @@
> +
> +
> +     #########################################################################
> +     ###### 64 bit specific ######
> +    'base_name': 'WINNT_6.1_%(branch)s',

is this right? should it be  WINNT_6.1_x86-64_branch?
Attachment #8559381 - Flags: review?(catlee) → review+
Assignee

Comment 5

4 years ago
Comment on attachment 8559381 [details] [diff] [review]
150204_1094364_win64_mh_cedar-mh.diff

Review of attachment 8559381 [details] [diff] [review]:
-----------------------------------------------------------------

https://hg.mozilla.org/build/mozharness/rev/d5c3f2d98f46

::: configs/builds/releng_base_windows_64_builds.py
@@ +59,5 @@
> +
> +
> +     #########################################################################
> +     ###### 64 bit specific ######
> +    'base_name': 'WINNT_6.1_%(branch)s',

yup. thanks for catch.
Attachment #8559381 - Flags: checked-in+
Assignee

Comment 6

4 years ago
Comment on attachment 8559360 [details] [diff] [review]
150204_1094364_win64_mh_cedar-bbot-configs.diff

https://hg.mozilla.org/build/buildbot-configs/rev/88116f993c6d
Attachment #8559360 - Flags: checked-in+
Assignee

Comment 7

4 years ago
poked this today. looks like we are failing to create a virtualenv: http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/cedar-win64/1424449078/cedar-win64-bm85-build1-build2.txt.gz

snippet:
09:27:46     INFO - buildids match.
09:27:46     INFO - Creating virtualenv c:\builds\moz2_slave\ced-w64-0000000000000000000000\build\venv
09:27:46    FATAL - The executable 'virtualenv' is not found; not creating virtualenv!
09:27:46    FATAL - Running post_fatal callback...
09:27:46    ERROR - setting return code to 2 because fatal was called
09:27:46    FATAL - Exiting -1
09:27:46     INFO - Running post-run listener: _summarize
09:27:46    ERROR - # TBPL FAILURE #

this must be because it landed beside mshal's s3 taskcluster bits and the virtualenv executable wasn't included.

I think this patch is all that's needed.

tested and now allows win64 to complete: https://treeherder.mozilla.org/#/jobs?repo=cedar&revision=d2d9810cf8d7
Assignee

Updated

4 years ago
Attachment #8570041 - Flags: review?(mshal)
Comment on attachment 8570041 [details] [diff] [review]
150225_mh_builds_win64_tc_fix-mh.patch

Looks good!
Attachment #8570041 - Flags: review?(mshal) → review+
Assignee

Comment 9

4 years ago
why not. We could do this more gradually by turning on a few branches at a time like I did initially for mh builds. However, seeing how mh builds have been working successfully for a while, I think the risk is less switching this over at once and backing out if needed.

note: this can't land till we update mozharness.json on trunk and m-a

builderlist diff: http://people.mozilla.org/~jlund/win64_mh_builds_m-c_and_m-a-builderlist.diff
Attachment #8570302 - Flags: review?(catlee)
Attachment #8570302 - Flags: review?(catlee) → review+
Assignee

Comment 10

4 years ago
will be switching this on today
Assignee

Comment 12

4 years ago
side note: we should probably stop asking for reviews of these. we still need a place to document the mh version bump on a branch as we are not only deploying revs that we need but also every rev in between current pin and this one. Reconfigs are merging mh to prod and updating bugs but most things are not going 'live' anymore.

this system of pinning by REV feels a little broken/awkward at the moment.
Attachment #8573086 - Flags: review?(catlee)
(In reply to Jordan Lund (:jlund) from comment #12)
> side note: we should probably stop asking for reviews of these. we still
> need a place to document the mh version bump on a branch as we are not only
> deploying revs that we need but also every rev in between current pin and
> this one. Reconfigs are merging mh to prod and updating bugs but most things
> are not going 'live' anymore.

I was wondering about this as well, especially since I'll need to bump it again for bug 1132123 :). Is the mh merge from default->production during a reconfig still useful when the version is pinned anyway? Maybe we should have a fool-proof mh bumper bot that updates the revision in m-c during a reconfig, and then nuke the production branch.
Attachment #8573083 - Flags: review?(catlee) → review+
Attachment #8573086 - Flags: review?(catlee) → review+
Assignee

Comment 15

4 years ago
Comment on attachment 8570302 [details] [diff] [review]
150226_win64_mh_builds_trunk_and_aurora-bbotcfgs.patch

Review of attachment 8570302 [details] [diff] [review]:
-----------------------------------------------------------------

the mh rev required is on m-c and m-a now.

some proj branches that are lazy at merging m-c may break for a short while till they actually merge.

on default: remote:   https://hg.mozilla.org/build/buildbot-configs/rev/b895db57d5a1
Attachment #8570302 - Flags: checked-in+
Assignee

Comment 17

4 years ago
reviewer: can I get your stamp for the subsequent mozharness.json bump too? It will need to land across trunk.

after landing, we had a regression for pgo builds: https://treeherder.mozilla.org/logviewer.html#?job_id=7373106&repo=mozilla-inbound

the reason is because I kept the max vsize linker check (and graph server post) enabled for win64 like it is for win32. However we don't do that for win64 buildbot factory case. You can see we only define enable_post_linker_size for the win32 case : http://mxr.mozilla.org/build/search?string=enable_post_linker_size&find=&findi=&filter=%5E%5B%5E%5C0%5D*%24&hitlimit=&tree=build


this patch disables the graph server post and linker check: http://mxr.mozilla.org/build/source/mozharness/mozharness/mozilla/building/buildbase.py#1518
Assignee

Comment 18

4 years ago
Comment on attachment 8574872 [details] [diff] [review]
150309_win64_mh_builds-disables_vsize_max_linker_check-mh.patch

can I get your approval here. we can figure out after we fix bustage whether we want to continue disabling win64 vsize linker checks or if we should update graph server to support win64 too
Attachment #8574872 - Flags: review?(mshal)
Attachment #8574872 - Flags: review?(mshal) → review+
Assignee

Comment 20

4 years ago
Comment on attachment 8574872 [details] [diff] [review]
150309_win64_mh_builds-disables_vsize_max_linker_check-mh.patch

thank you, this has landed and m-i mozharness.json has been updated. once merged, pgo win64 will be fixed

remote:   https://hg.mozilla.org/build/mozharness/rev/0230a275037f
Attachment #8574872 - Flags: checked-in+
https://hg.mozilla.org/mozilla-central/rev/1d8fe559384e
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Comment hidden (Legacy TBPL/Treeherder Robot)
Assignee

Comment 25

4 years ago
sigh. we are hitting failures with pgo win again.

looks like we lost my previous mozharness.json rev because we switched the pin rev from a default one to a prod one and the prod rev didn't have the latest default changes merged.

this points us to current tip
Attachment #8576194 - Flags: review?(coop)
Attachment #8576194 - Flags: review?(coop) → review+
Assignee

Updated

4 years ago
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee

Updated

4 years ago
Status: REOPENED → RESOLVED
Closed: 4 years ago4 years ago
Resolution: --- → FIXED
Attachment #8576989 - Flags: review?(mshal) → review+
Assignee

Comment 28

4 years ago
Comment on attachment 8576989 [details] [diff] [review]
150309_win64_mh_builds-add_to_percheckin_pgo_strategy-mh.patch

remote:   https://hg.mozilla.org/build/mozharness/rev/c89055ada676
Attachment #8576989 - Flags: checked-in+
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.