move mochitest-browser-chrome tests from win8->win10

RESOLVED FIXED in Firefox 57

Status

RESOLVED FIXED
a year ago
11 months ago

People

(Reporter: jmaher, Assigned: ahal)

Tracking

Trunk
mozilla57
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox57 fixed)

Details

MozReview Requests

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(4 attachments)

(Assignee)

Comment 1

a year ago
Sure! I'm really close to getting reftest selftests working, I'll take a look when I'm done that.
Assignee: nobody → ahalberstadt
Status: NEW → ASSIGNED
(Assignee)

Comment 2

a year ago
So this test was skipped on Windows 8 within the test itself:
http://searchfox.org/mozilla-central/source/browser/base/content/test/performance/browser_windowopen_reflows.js

I guess because manifests can't skip on specific windows versions? (we should maybe fix that)

That being said, digging into that test it looks like on Windows 10 there is 1 fewer reflow than before (which is a good thing). So I deleted that expected reflow and everything looks like it works:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b1c0739f6cc46c4218d4286fb6f9cd29dc307a02

I'll get review from mconley here to make sure this is ok. If it's not ok I guess we can just update the skip-if line to also include Win10.
Flags: needinfo?(ahalberstadt)
(Reporter)

Comment 3

a year ago
if needed in browser.ini we could:
skip-if = os == 'win' && bits=64
mozinfo adds the `os_version` key by default, and that's what we feed into the manifest parser, so you should be able to use `os_version == '10.0'` or similar.

On my local Windows 10 machine os_version == '10.0':
>>> import mozinfo
>>> mozinfo.os_version
StringVersion ('10.0')
(Assignee)

Comment 5

a year ago
True! I think the win8 tests are running on a win10 generated build though, so the mozinfo.json probably has the wrong os_version in it.

The fix worked though, so I think it's moot (for this case). The test can probably be re-enabled.
(Reporter)

Updated

11 months ago
Depends on: 1397879
Just for completeness, I want to note that we don't put os_version in the mozinfo.json that the build generates. Random example from an inbound Windows build:
https://public-artifacts.taskcluster.net/bXT8PmuISYGgwBiQo7ge3w/0/public/build/target.mozinfo.json

os_version will be determined by mozinfo on the test machine, so it should work fine.
(Assignee)

Comment 7

11 months ago
Ah, neat! I'll move the skip over to the manifest as part of this patch.
(Assignee)

Updated

11 months ago
Depends on: 1397829
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
(Reporter)

Comment 10

11 months ago
mozreview-review
Comment on attachment 8906731 [details]
Bug 1397229 - Schedule mochitest-browser-chrome tests on Windows 10 instead of Windows 8,

https://reviewboard.mozilla.org/r/178466/#review183466

this looks great; lets ensure we have capacity (i.e. bug 1398884)
Attachment #8906731 - Flags: review?(jmaher) → review+

Comment 11

11 months ago
mozreview-review
Comment on attachment 8906730 [details]
Bug 1397229 - Remove unused reflow from browser_windowopen_reflows.js on Windows,

https://reviewboard.mozilla.org/r/178464/#review183470

Great! I assume this change survives retriggers on try? If so, yeah, let's take it, and please WONTFIX bug 1381521. Thanks for your work here!
Attachment #8906730 - Flags: review?(mconley) → review+
(Assignee)

Comment 12

11 months ago
mozreview-review-reply
Comment on attachment 8906730 [details]
Bug 1397229 - Remove unused reflow from browser_windowopen_reflows.js on Windows,

https://reviewboard.mozilla.org/r/178464/#review183470

Yep, it's permanently fixed somehow!

Should I just remove the skip-if win 8 from the manifest then, given this won't be running anywhere on Windows 8? If for some weird reason we ever start running this there again, I guess we can just re-add the skip-if and file a new bug then.
(Assignee)

Comment 13

11 months ago
We're still waiting for the two blockers here to be fixed before landing this.

There were also a few intermittents I noticed in some of my try pushes. They all look known, but I'll do a bunch more retriggers to make sure they didn't become really frequent.
(Assignee)

Comment 15

11 months ago
There's two perma failures in that push:

1. browser/base/content/test/performance/browser_startup_images.js

This is a new test and is already skipped on Windows 7. It's unclear if it was running/working on Windows 8 or not, but maybe this Win10 issue is similar and it just needs to be skipped there too.


2. toolkit/components/tooltiptext/tests/browser_input_file_tooltips.js

This looks like a pre-existing intermittent turned perma-fail (bug 1295497). The test seems to be timing out here:
http://searchfox.org/mozilla-central/source/toolkit/components/tooltiptext/tests/browser_input_file_tooltips.js#38

It's worth noting this test also bypasses the no-arbitrary-setTimeout eslint rule, though looks unrelated to this specific issue.
(Assignee)

Comment 16

11 months ago
Hey Johann, we're in the process of migrating some tests from Windows 8 -> Windows 10. It looks like the hdpi version of browser_startup_images.js is failing on Windows 10:
https://treeherder.mozilla.org/logviewer.html#?job_id=131099613&repo=try&lineNumber=4225

I noticed this test was already disabled on Windows 7, does this look like a similar issue? Can we also disable it on Windows 10 for the time being? Let me know if you see a fix and/or how you'd like to proceed! Thanks
Flags: needinfo?(jhofmann)
(Assignee)

Comment 17

11 months ago
(In reply to Andrew Halberstadt [:ahal] from comment #15) 
> 2. toolkit/components/tooltiptext/tests/browser_input_file_tooltips.js
> 
> This looks like a pre-existing intermittent turned perma-fail (bug 1295497).
> The test seems to be timing out here:
> http://searchfox.org/mozilla-central/source/toolkit/components/tooltiptext/
> tests/browser_input_file_tooltips.js#38
> 
> It's worth noting this test also bypasses the no-arbitrary-setTimeout eslint
> rule, though looks unrelated to this specific issue.

Looking at bug 1295497, it looks like the timeout is happening at a different place, so this could be a different issue. Gijs, I saw you did a bit of digging in bug 1295497, do you have any idea what might be happening in:
https://treeherder.mozilla.org/logviewer.html#?job_id=131094283&repo=try&lineNumber=5406

For context, this started failing after migrating it from Win8 -> Win10.
Flags: needinfo?(gijskruitbosch+bugs)
(In reply to Andrew Halberstadt [:ahal] from comment #16)
> Hey Johann, we're in the process of migrating some tests from Windows 8 ->
> Windows 10. It looks like the hdpi version of browser_startup_images.js is
> failing on Windows 10:
> https://treeherder.mozilla.org/logviewer.
> html#?job_id=131099613&repo=try&lineNumber=4225
> 
> I noticed this test was already disabled on Windows 7, does this look like a
> similar issue? Can we also disable it on Windows 10 for the time being? Let
> me know if you see a fix and/or how you'd like to proceed! Thanks

I think that issue is slightly different than the one from Windows 7 (we probably just need to whitelist maximize.svg), but I don't have the time to deal with that immediately. I'm ok with disabling it for now if you open a bug for me to follow up on :)
Flags: needinfo?(jhofmann)
(Assignee)

Updated

11 months ago
See Also: → bug 1400357
(Assignee)

Updated

11 months ago
See Also: → bug 1400368
I replied on bug 1400368. We can disable the browser_input_file_tooltips.js test on win10 for now.
Flags: needinfo?(gijskruitbosch+bugs)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
(Assignee)

Comment 24

11 months ago
Here's a try run with those tests disabled:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b4d3794b17bf8b93c23eb0d50e8b54b7cf124946

There's a few intermittents, seem to all be on file. I'm sort of worried that the frequency has been increased, but I figured that's something we could make a decision on based on the orange factor data that comes in.
(Reporter)

Comment 25

11 months ago
mozreview-review
Comment on attachment 8909293 [details]
Bug 1397229 - Disable browser/base/content/test/performance/browser_startup_images.js hidpi on Windows 10,

https://reviewboard.mozilla.org/r/180868/#review185992

::: browser/base/content/test/performance/hidpi/browser.ini:6
(Diff revision 1)
>  [DEFAULT]
>  prefs =
>    layout.css.devPixelsPerPx='2'
>  
>  [../browser_startup_images.js]
> -skip-if = !debug || (os == 'win' && os_version == '6.1') # hidpi results in the toolbar overflowing on Win 7
> +skip-if = !debug || (os == 'win' && (os_version == '6.1' || os_version == '10.0'))  # hidpi results in the toolbar overflowing on Win 7; fails on Win 10 (bug 1400357)

have you confirmed that our version os '10.0' ?
Attachment #8909293 - Flags: review?(jmaher) → review+
(Reporter)

Comment 26

11 months ago
mozreview-review
Comment on attachment 8909294 [details]
Bug 1397229 - Disable toolkit/browser/components/tooltiptext/browser_input_file_tooltips.js on Win10,

https://reviewboard.mozilla.org/r/180870/#review185994

assuming we match the os_version == 10.0 exactly.
Attachment #8909294 - Flags: review?(jmaher) → review+
(Assignee)

Comment 27

11 months ago
mozreview-review-reply
Comment on attachment 8909293 [details]
Bug 1397229 - Disable browser/base/content/test/performance/browser_startup_images.js hidpi on Windows 10,

https://reviewboard.mozilla.org/r/180868/#review185992

> have you confirmed that our version os '10.0' ?

Yep!
Backed out for failing browser-chrome's browser/base/content/test/performance/browser_windowopen_reflows.js on Windows 7:

https://hg.mozilla.org/integration/autoland/rev/93ac511811516cfcd3e3c759f0ce3afef8a6b371

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=cb09c5099e519f615038109f0da3da190555775b&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=131749901&repo=autoland

14:41:29     INFO -  Buffered messages finished
14:41:29    ERROR -  774 INFO TEST-UNEXPECTED-FAIL | browser/base/content/test/performance/browser_windowopen_reflows.js | unexpected uninterruptible reflow
14:41:29     INFO -  [
14:41:29     INFO -  	"handleEvent@chrome://browser/content/tabbrowser.xml:6864:19",
14:41:29     INFO -  	"EventListener.handleEvent*tabbrowser-tabs_XBL_Constructor@chrome://browser/content/tabbrowser.xml:6282:11",
14:41:29     INFO -  	""
14:41:29     INFO -  ]
14:41:29     INFO -   - false == true - JS frame :: chrome://mochitests/content/browser/browser/base/content/test/performance/head.js :: reflow :: line 115
14:41:29     INFO -  Stack trace:
14:41:29     INFO -  chrome://mochitests/content/browser/browser/base/content/test/performance/head.js:reflow:115
14:41:29     INFO -  chrome://browser/content/tabbrowser.xml:handleEvent:6864
Flags: needinfo?(ahalberstadt)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
Comment hidden (mozreview-request)
(Assignee)

Comment 33

11 months ago
Comment on attachment 8906730 [details]
Bug 1397229 - Remove unused reflow from browser_windowopen_reflows.js on Windows,

This fixes the Windows 7 error from the backout:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b8d7a18ccc0bdfdd3bd50f1e9271140a69324320

Hey Mike, I'd appreciate a (trivial) re-review of this latest change. Using the user agent feels hacky, but this is how the test was previously being skipped and I don't know of a better way.

Alternatively if this test doesn't need to run on both Win7 and Win10, we could disable it on Win7.
Flags: needinfo?(ahalberstadt)
Attachment #8906730 - Flags: review+ → review?(mconley)

Comment 34

11 months ago
mozreview-review
Comment on attachment 8906730 [details]
Bug 1397229 - Remove unused reflow from browser_windowopen_reflows.js on Windows,

https://reviewboard.mozilla.org/r/178464/#review186192

Yeah, let's do this. Thanks!
Attachment #8906730 - Flags: review?(mconley) → review+

Comment 35

11 months ago
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d928ad6475d2
Remove unused reflow from browser_windowopen_reflows.js on Windows, r=mconley
https://hg.mozilla.org/integration/autoland/rev/fed3a10c6ffc
Disable browser/base/content/test/performance/browser_startup_images.js hidpi on Windows 10, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/1c9f780c4e9c
Disable toolkit/browser/components/tooltiptext/browser_input_file_tooltips.js on Win10, r=jmaher

Comment 36

11 months ago
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/9f07c6210b34
Remove unused reflow from browser_windowopen_reflows.js on Windows, r=mconley
https://hg.mozilla.org/integration/autoland/rev/2671b6644f50
Disable browser/base/content/test/performance/browser_startup_images.js hidpi on Windows 10, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/bb89cd6f5507
Disable toolkit/browser/components/tooltiptext/browser_input_file_tooltips.js on Win10, r=jmaher
https://hg.mozilla.org/integration/autoland/rev/008e877ffdaa
Schedule mochitest-browser-chrome tests on Windows 10 instead of Windows 8, r=jmaher

Comment 37

11 months ago
Backout by kwierso@gmail.com:
https://hg.mozilla.org/mozilla-central/rev/87b0dfdc4832
Backed out changeset cb09c5099e51 
https://hg.mozilla.org/mozilla-central/rev/1eaf47154762
Backed out changeset 1c9f780c4e9c 
https://hg.mozilla.org/mozilla-central/rev/9f2bddf428df
Backed out changeset fed3a10c6ffc
You need to log in before you can comment on or make changes to this bug.