1354804 - Ensure no talos regressions with Windows 2012 builds

Status: RESOLVED INCOMPLETE

(Firefox Build System :: Task Configuration, task)

Description

[Greg Arndt [:garndt]]

Sooner rather than later we should perform some talos tests against our Windows 2012 builds to ensure there are no regressions and we can move forward with these builds.

Comment 1

[Kim Moir [:kmoir] ET]

Joel, is this something than can someone from your team can run manually against a windows 2012 build since I don't see any talos tests enabled for windows tc builds on try etc in automation?

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

we can look into it, but right now we are focused on performance and quantum- it might be a few weeks.

Could you add a taskcluster config to use BBB and schedule jobs on win8 with the win 2012 build?  likewise we have win7 for the 32 bit builds.  If there were taskcluster jobs on a try pushing using BBB, that would help and we could analyze the data, etc. :)

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

as a note, I did a try push to attempt this:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1233f72b77cfb3813f944e48dc81027eedb88659&filter-tier=1&filter-tier=2&filter-tier=3

it looks like it needs a bit more hacking, probably one or two simple things to cleanup- maybe someone else might know how to make that work?  it appears getting data from tooltool isn't working, sounds sort of familiar that maybe we solved for linux and osx already.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

or...this could just be the general issue on the trees via bug 1355205.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

ok, got talos results for win7, but the win 2012 x64 builds do not work at all:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1233f72b77cfb3813f944e48dc81027eedb88659&filter-tier=1&filter-tier=2&filter-tier=3

on the base revision the builds worked fine, so either try is broken or very flaky- I will update with win7 results.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

there are no issues with the 32 bit builds, feel free to use my patch to get an idea of what to do when 64 bit builds are working on try.

Comment 7

[Kim Moir [:kmoir] ET]

thanks Joel!  I really appreciate it :-)

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

this morning I pushed to try and got a windows 8 build:
https://treeherder.mozilla.org/#/jobs?repo=try&author=[e-mail]&selectedJob=92058249

Looking at the performance comparison, I see 1 possible regression:
https://treeherder.mozilla.org/perf.html#/compare?originalProject=mozilla-inbound&originalRevision=f659555fc474&newProject=try&newRevision=ec9529f7afd58ed5a043862c437844b7de723aa7&framework=1

it is minor and fuzzy as there is a lot of overlap in the numbers from both builds.  In many cases the numbers follow a certain pattern and we still fit in the pattern, just seem to be on the higher end of the normal pattern.

Compared to OSX, I see no issues here and we should move forward assuming all is well.  I would prefer we do make a sanity pass over the build configs in general.

On a positive note, windows 8 has been crashing about 5-10% of the time on e10s.  I saw it 3 times in the base revision, and no times in the try push.  So something about the TC builds are allowing talos to run on win8-e10s more reliably.  As a note, I did 80 more retriggers to see if I can find a crash, if not, I would see if there is anything we can do to use the tc builds instead of the bb builds :)

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

all the retriggers are done, no specific change in the failure rate, maybe a slight reduction but still many failures.  I did notice a 404 on downloading the crash symbols.

In the tc jobs, I see a query to:
https://queue.taskcluster.net/v1/task/ZFjdypUbTsuaerRTTgxVWg/artifacts/public/build/target.crashreporter-symbols.zip

but the symbols are actually at:
https://queue.taskcluster.net/v1/task/ZFjdypUbTsuaerRTTgxVWg/runs/0/artifacts/public/build/firefox-55.0a1.en-US.win64.crashreporter-symbols.zip

and the runs/0/ is not needed, so it could be:
https://queue.taskcluster.net/v1/task/ZFjdypUbTsuaerRTTgxVWg/artifacts/public/build/firefox-55.0a1.en-US.win64.crashreporter-symbols.zip


the difference is in the filename, not target.crashreporter, but firefox-55.0a1.en-US.win64.crashreporter.  I am not sure where we need to fix that, but maybe in taskcluster, buildbotbridge, or some code in my patch wasn't right.

Comment 10

[Tom Prince [:tomprince]]

This migration is long past, and regressions from it are likely no longer relevant.