1473259 - Windows Updates for ASan reporter Nightly build

Status: RESOLVED FIXED

(Release Engineering :: Release Requests, enhancement)

Description

[Christian Holler (:decoder)]

The ASan reporter now supports Windows, so it would be nice if we could duplicate the Linux update channel work to produce builds and updates for Windows as well.

I have a WIP patch running on try right now, will post results as they come in.

Comment 1

[Christian Holler (:decoder)]

Attachment: WIP patch for build and updates

Here is a WIP patch with a try run:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4e61f531b5d016665a6ec24b57937954d7555fa3

The BM-R failure is due to required changes on the beetmover side.

:catlee, can you let me know if the patch looks otherwise ok so far? I am particularly unsure about the namings in taskcluster/taskgraph/util/partials.py because we don't have non-reporter equivalents on Windows listed there (e.g. should it be WINNT_x86_64-msvc-x64-asan instead?)

Also, should I make a PR for beetmover changes, or do you want to make these? Thanks!

Comment 2

[Mihai Tabara [:mtabara]⌚️GMT]

@decoder 
* hope you don't mind, I pushed a tiny commit in your PR[1] to fix the CHANGELOG vs newsfragment to keep things consistent with previous PRs
* Try branch doesn't use the production beetmoverworkers (for security reasons), but the dev pool. I pinned the whole pool to my environment and I pushed your changes there to test things out. All beetmoverworkers now have your changes applied.
* I couldn't rerun the beetmover-repackage job since it's way passed its deadline, but feel free to push again to try to see if things go smoothly. If it breaks again, I can follow-up with fixes and reruns to see it done today.

[1]: https://github.com/mozilla-releng/beetmoverscript/pull/170

Comment 3

[Christian Holler (:decoder)]

(In reply to Mihai Tabara [:mtabara]⌚️GMT from comment #2)

> * hope you don't mind, I pushed a tiny commit in your PR[1] to fix the
> CHANGELOG vs newsfragment to keep things consistent with previous PRs
> * Try branch doesn't use the production beetmoverworkers (for security
> reasons), but the dev pool. I pinned the whole pool to my environment and I
> pushed your changes there to test things out. All beetmoverworkers now have
> your changes applied.
> * I couldn't rerun the beetmover-repackage job since it's way passed its
> deadline, but feel free to push again to try to see if things go smoothly.
> If it breaks again, I can follow-up with fixes and reruns to see it done
> today.


Thanks a lot! I am preparing a repush to try to see if it is working now. I had to make slight adjustments to my patch anyway to match the architecture properly.

Comment 4

[Christian Holler (:decoder)]

Attachment: Bug 1473259 - Windows updates for ASan reporter nightly build.

Review commit: https://reviewboard.mozilla.org/r/257048/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/257048/

Comment 5

[Christian Holler (:decoder)]

(In reply to Christian Holler (:decoder) from comment #4)
> Created attachment 8992150 [details]
> Bug 1473259 - Windows updates for ASan reporter nightly build.
> 
> Review commit: https://reviewboard.mozilla.org/r/257048/diff/#index_header
> See other reviews: https://reviewboard.mozilla.org/r/257048/

This is green on try and has all the necessary changes already made in beetmover as well.

Comment 6

[Chris AtLee [:catlee]]

Comment on attachment 8992150 [details]
Bug 1473259 - Windows updates for ASan reporter nightly build.

https://reviewboard.mozilla.org/r/257048/#review265160

::: browser/config/mozconfigs/win64/nightly-asan-reporter:21
(Diff revision 1)
> +
> +export MOZ_PKG_SPECIAL=asan-reporter
> +
> +# Need this to add source information into platform.ini
> +export MOZILLA_OFFICIAL=1
> +

Do you need telemetry enabled here like was added to Linux in bug 1471647?

Comment 7

[Christian Holler (:decoder)]

Attachment: build-tools-win64-asan.patch

Necessary changes to build/tools repository for making the updates work.

Comment 8

[Christian Holler (:decoder)]

Mihai, could you put the beetmover PR for this and the patch for build/tools (once catlee r+'ed it) into production so I can land the m-c patch afterwards? Thanks in advance!

Comment 9

[Mihai Tabara [:mtabara]⌚️GMT]

Taking care of these now. Following-up in a bit with all the confirmations.

Comment 10

[Mihai Tabara [:mtabara]⌚️GMT]

Merged the PR: https://github.com/mozilla-releng/beetmoverscript/commit/6e1a66be08573e53e5b45f6a4512cac7cf798b4b
Bumped beetmoverscript to 7.6.0: https://github.com/mozilla-releng/beetmoverscript/commit/2d89210b85d081e6ee20d169ffa1c8b20907e068
Bumped beetmoverworkers: https://github.com/mozilla-releng/build-puppet/pull/126/files
Unpinned the dep beetmoverworkers as they'll pick the master anyway.

Comment 11

[Mihai Tabara [:mtabara]⌚️GMT]

All beetmoverworkers (production and dev pools) are now configured to use the latest 7.6.0 beetmoverscript that encompasses the asan changes. Give it a few more seconds/minutes for all changes to propagate but we should be good.

Comment 12

[Mihai Tabara [:mtabara]⌚️GMT]

Comment on attachment 8993490 [details] [diff] [review]
build-tools-win64-asan.patch

https://hg.mozilla.org/build/tools/rev/cadaac874297ba681839c3840d7b265b5ed45c99

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7c7794653326
Windows updates for ASan reporter nightly build. r=catlee

Comment 14

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/7c7794653326

Comment 15

[Pete Moore [:pmoore][:pete]]

Hi Christian,

Could this change be related to e.g. these failures?

https://taskcluster-artifacts.net/UqwGzpNJTn68x6PiSERtWA/1/public/logs/live_backing.log
https://taskcluster-artifacts.net/Z2jdeFttRp6e7IjNnxvo_w/0/public/logs/live_backing.log

Thanks,
Pete

Comment 16

[Christian Holler (:decoder)]

(In reply to Pete Moore [:pmoore][:pete] from comment #15)
> Hi Christian,
> 
> Could this change be related to e.g. these failures?
> 
> https://taskcluster-artifacts.net/UqwGzpNJTn68x6PiSERtWA/1/public/logs/
> live_backing.log
> https://taskcluster-artifacts.net/Z2jdeFttRp6e7IjNnxvo_w/0/public/logs/
> live_backing.log
> 
> Thanks,
> Pete

I think both are the result of me trying to manually trigger those builds on mozilla-central rather than waiting for cron to do so. That didn't work (and isn't intended to work either).

Are you still seeing the failures now on July 25 ongoing?

Comment 17

[Pete Moore [:pmoore][:pete]]

Thanks Christian for looking into it.

Then no doubt it was just a one-off and I'll let you know if it happens again.