1286435 - browser_parsable_css.js is failing but not turning orange on infra

Status: RESOLVED FIXED

(Firefox :: General, defect)

Description

[Jared Wein [:jaws] (please needinfo? me)]

I ran browser_parsable_css.js locally and got the following error:

102 INFO TEST-UNEXPECTED-FAIL | browser/base/content/test/general/browser_parsable_css.js | Got error message for chrome://loop/content/shared/css/common.css: Unknown property ΓÇÿuser-selectΓÇÖ.  Declaration dropped. -

The unknown property is 'user-select' (the weird characters above are due to an encoding mismatch in my terminal).

The test is failing because it has an exception for:
  {sourceName: /loop\/.*shared\/css\/common.css$/i,
   errorMessage: /Unknown property 'user-select'/i},
except the error messages have been changed to use Unicode curly single quotes whereas the error message embedded in the test uses straight single quotes.

It will be easy to update the browser_parsable_css.js test to use the curly quotes, but we need to figure out why this failure wasn't making the test turn orange on infra.

Gijs, as you wrote this test do you have any idea here?

Comment 1

[:Gijs (he/him)]

Not really much without a full log from infra. And even then...

Does running the test in the directory fail locally? What about running it on its own? You can also use the mochitest against a build from infra to see if there's something funny going on with the stuff it's being tested against there...

The one thing we've seen before is error messages not making it to the terminal before the test finishes, but we fixed that, so not sure if that's likely to still be interfering...

Comment 2

[Jared Wein [:jaws] (please needinfo? me)]

The test fails if I run it through `mach test browser/base/content/test/general` and also through `mach test browser/base/content/test/general/browser_parsable_css.js`.

http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-win32/1468412425/mozilla-inbound_win7_ix_test-mochitest-browser-chrome-7-bm127-tests1-windows-build144.txt.gz shows that the test passes on infra. Does it fail for you if you run the test with either of the commands above?

Comment 3

[:Gijs (he/him)]

(In reply to Jared Wein [:jaws] (please needinfo? me) from comment #2)
> The test fails if I run it through `mach test
> browser/base/content/test/general` and also through `mach test
> browser/base/content/test/general/browser_parsable_css.js`.
> 
> http://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-inbound-
> win32/1468412425/mozilla-inbound_win7_ix_test-mochitest-browser-chrome-7-
> bm127-tests1-windows-build144.txt.gz shows that the test passes on infra.

Yes, but it has no logs, so no idea what's happening.

> Does it fail for you if you run the test with either of the commands above?

Sure, but that doesn't help us track down why it doesn't fail on infra.

Comment 4

[Jared Wein [:jaws] (please needinfo? me)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=df7341c3c14b&exclusion_profile=false

Comment 5

[Jared Wein [:jaws] (please needinfo? me)]

Attachment: parsable_css.log

This is the relevant part of the log from https://treeherder.mozilla.org/#/jobs?repo=try&revision=df7341c3c14b&filter-resultStatus=success&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=running&filter-resultStatus=pending&filter-resultStatus=runnable&selectedJob=24139621.

There are only 69 files parsed when run on tryserver but locally it parses 94 files. Locally it runs a second pass where it looks at 100 more files. I'm not sure why it does two passes locally, but the log from the try push only shows one pass.

Comment 6

[Jared Wein [:jaws] (please needinfo? me)]

Attachment: parsable_css.diff

Comment 7

[:Gijs (he/him)]

Attachment: Bug 1286435 - include extension zip/xpi files in JS and CSS parsing tests,

Review commit: https://reviewboard.mozilla.org/r/65338/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/65338/

Comment 8

[:Gijs (he/him)]

Comment on attachment 8772593 [details]
Bug 1286435 - include extension zip/xpi files in JS and CSS parsing tests,

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/65338/diff/1-2/

Comment 9

[Jared Wein [:jaws] (please needinfo? me)]

Comment on attachment 8772593 [details]
Bug 1286435 - include extension zip/xpi files in JS and CSS parsing tests,

https://reviewboard.mozilla.org/r/65338/#review62362

Interesting!

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4425ad9e4849
include extension zip/xpi files in JS and CSS parsing tests, r=jaws

Comment 11

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/4425ad9e4849