Closed Bug 1444203 Opened 6 years ago Closed 6 years ago

[tier-2] Test-Verify permafailing /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- img elements - assert_not_equals: got disallowed value 2236

Categories

(Core :: CSS Parsing and Computation, defect, P5)

Product:
Core
Component:
CSS Parsing and Computation
Type:
defect

Tracking

()

Status:
RESOLVED DUPLICATE of bug 1445164

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell disabled]) 

Filed by: ncsoregi [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=166842285&repo=autoland

https://queue.taskcluster.net/v1/task/SwaT_f_eQKmluugM6NWHLg/runs/0/artifacts/public/logs/live_backing.log

[task 2018-03-08T20:30:34.306Z] 20:30:34     INFO - TEST-START | /css/css-tables/table-model-fixup-2.html
[task 2018-03-08T20:30:35.675Z] 20:30:35     INFO - ...
[task 2018-03-08T20:30:35.676Z] 20:30:35     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements inside a table cannot be table-column and are considered inline -- img elements 
[task 2018-03-08T20:30:35.676Z] 20:30:35     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements inside a table cannot be table-cell and are considered inline -- input elements 
[task 2018-03-08T20:30:35.677Z] 20:30:35     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements inside a table cannot be table-cell and are considered inline -- img elements 
[task 2018-03-08T20:30:35.678Z] 20:30:35     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be inline-table and are considered inline -- input=text elements 
[task 2018-03-08T20:30:35.679Z] 20:30:35     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be inline-table and are considered inline -- input=button elements 
[task 2018-03-08T20:30:35.679Z] 20:30:35     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be inline-table and are considered inline -- input=file elements 
[task 2018-03-08T20:30:35.679Z] 20:30:35     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be inline-table and are considered inline -- img elements 
[task 2018-03-08T20:30:35.680Z] 20:30:35     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- input=text elements 
[task 2018-03-08T20:30:35.681Z] 20:30:35     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- input=button elements 
[task 2018-03-08T20:30:35.682Z] 20:30:35     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- input=file elements 
[task 2018-03-08T20:30:35.683Z] 20:30:35     INFO - TEST-UNEXPECTED-FAIL | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- img elements - assert_not_equals: got disallowed value 2236
[task 2018-03-08T20:30:35.684Z] 20:30:35     INFO - assertUnequalOffsetTops@http://web-platform.test:8000/css/css-tables/table-model-fixup-2.html:47:9
[task 2018-03-08T20:30:35.684Z] 20:30:35     INFO - @http://web-platform.test:8000/css/css-tables/table-model-fixup-2.html:252:9
[task 2018-03-08T20:30:35.692Z] 20:30:35     INFO - ................
[task 2018-03-08T20:30:35.693Z] 20:30:35     INFO - TEST-OK | /css/css-tables/table-model-fixup-2.html | took 1376ms
[task 2018-03-08T20:30:36.208Z] 20:30:36     INFO - PID 856 | *** UTM:SVC TimerManager:registerTimer called after profile-before-change notification. Ignoring timer registration for id: telemetry_modules_ping
[task 2018-03-08T20:30:37.441Z] 20:30:37  WARNING - u'runner_teardown': ()
[task 2018-03-08T20:30:37.522Z] 20:30:37     INFO - certutil | 
[task 2018-03-08T20:30:37.578Z] 20:30:37     INFO - certutil | 
[task 2018-03-08T20:30:37.615Z] 20:30:37     INFO - certutil | 
[task 2018-03-08T20:30:37.615Z] 20:30:37     INFO - Certificate Nickname                                         Trust Attributes
[task 2018-03-08T20:30:37.615Z] 20:30:37     INFO -                                                              SSL,S/MIME,JAR/XPI
[task 2018-03-08T20:30:37.615Z] 20:30:37     INFO - 
[task 2018-03-08T20:30:37.615Z] 20:30:37     INFO - web-platform-tests                                           CT,, 
[task 2018-03-08T20:30:37.616Z] 20:30:37     INFO - 
[task 2018-03-08T20:30:42.995Z] 20:30:42     INFO - PID 1006 | 1520541042986	Marionette	INFO	Listening on port 2828
Blocks: 1411778
Component: CSS Parsing and Computation → web-platform-tests
Product: Core → Testing
This bug has been filed yesterday and already has 54 failures.

Summary: Permafailing /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- img elements - assert_not_equals: got disallowed value 2236

Failures per platform:
- Linux x64: 21
- OS X 10.10: 9
- Windows 7: 9
- windows10-64: 14
- Linux: 1

Failures per build type:
- opt: 19
- debug: 20
- pgo: 9
- asan: 6

Here is a relevant log file and a snippet with the failure:
https://treeherder.mozilla.org/logviewer.html#?repo=autoland&job_id=166796325&lineNumber=1285

[task 2018-03-08T18:19:41.322Z] 18:19:41     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- input=button elements 
[task 2018-03-08T18:19:41.322Z] 18:19:41     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- input=file elements 
[task 2018-03-08T18:19:41.323Z] 18:19:41     INFO - TEST-UNEXPECTED-FAIL | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- img elements - assert_not_equals: got disallowed value 2236
[task 2018-03-08T18:19:41.324Z] 18:19:41     INFO - assertUnequalOffsetTops@http://web-platform.test:8000/css/css-tables/table-model-fixup-2.html:47:9
[task 2018-03-08T18:19:41.325Z] 18:19:41     INFO - @http://web-platform.test:8000/css/css-tables/table-model-fixup-2.html:252:9
[task 2018-03-08T18:19:41.331Z] 18:19:41     INFO - ................
[task 2018-03-08T18:19:41.331Z] 18:19:41     INFO - TEST-OK | /css/css-tables/table-model-fixup-2.html | took 1381ms
Flags: needinfo?(james)
Whiteboard: [stockwell needswork]
This is a test-verify failure. I don't think it should be in the testing/web-platform-tests component unless it's a problem with the harness. It can be reproduced locally with |mach wpt --verify /css/css-tables/table-model-fixup-2.html| and also by running the test directly in the browser using

cd testing/web-platform/tests
./wpt serve

<load http://web-platform.test:8000/css/css-tables/table-model-fixup-2.html in Firefox and reload>

I don't see the same reload-dependent failure in Chrome so it's not obviously a test issue.
Component: web-platform-tests → CSS Parsing and Computation
Flags: needinfo?(james)
Product: Testing → Core
Given that this is a regression from bug 1411778... could you annotate this as an expected failure?  It's not generally OK to land things that make tests orange and leave it for others to clean up as an urgent matter.
Flags: needinfo?(james)
(In reply to James Graham [:jgraham] from comment #3)
> and also by running the test
> directly in the browser using
> 
> cd testing/web-platform/tests
> ./wpt serve
> 
> <load http://web-platform.test:8000/css/css-tables/table-model-fixup-2.html
> in Firefox and reload>

When I do this I don't see any differences on reload; I see 21 pass and 9 fail.
> It's not generally OK to land things that make tests orange and leave it for others to clean up as an urgent matter.

This mostly affects a tier-2 verify job that only runs when this file is changed. Therefore it didn't obviously seem like an urgent fix. It does also seem to be failing occasionally on other pushes, so I think there's a real intermittent here, but the rate is vastly overstated because all the verify failures were also starred.

So, I can look at the intermittent, but unless it happens to clearly be a test problem it's unlikely that I can do much except disable the test. That doesn't seem like an optimal approach compared to a layout developer investigating which will likely be faster and much more likely to end up with a useful fix.

> When I do this I don't see any differences on reload; I see 21 pass and 9 fail.

I 100% reproducably see 8 fails on the first load and 9 on subsequent loads. Even loading the test in a new tab is enough to get back to 8. The rendering looks identical in each case, and the test looks like it should be passing, so I suspect we're getting the wrong getBoundingClientRect() value in some cases for some reason.
Flags: needinfo?(james)
It would be nice if we actually distinguished things that were tier-2 from those that were tier-1.
Summary: Permafailing /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- img elements - assert_not_equals: got disallowed value 2236 → [tier-2] Permafailing /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- img elements - assert_not_equals: got disallowed value 2236
:dbaron this occurs on tier1 as well, but it's not permafailing, please see https://bugzilla.mozilla.org/show_bug.cgi?id=1445164
Blocks: 1445164
No longer blocks: 1445164
Summary: [tier-2] Permafailing /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- img elements - assert_not_equals: got disallowed value 2236 → [tier-2] Test-Verify permafailing /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- img elements - assert_not_equals: got disallowed value 2236
Blocks: 1445164
There have been 32 total failures in the last week.
Almost all the failures are on linux64-qr / opt now.
This started to occur again on March 20th and it seems that almost are failures are tier-2.

Recent relevant log file and log snippet:
https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=169992731&lineNumber=2061

[task 2018-03-23T23:07:58.951Z] 23:07:58     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- input=button elements 
[task 2018-03-23T23:07:58.951Z] 23:07:58     INFO - TEST-PASS | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- input=file elements 
[task 2018-03-23T23:07:58.952Z] 23:07:58     INFO - TEST-UNEXPECTED-FAIL | /css/css-tables/table-model-fixup-2.html | Replaced elements outside a table cannot be table and are considered block -- img elements - assert_not_equals: got disallowed value 2236
[task 2018-03-23T23:07:58.953Z] 23:07:58     INFO - assertUnequalOffsetTops@http://web-platform.test:8000/css/css-tables/table-model-fixup-2.html:47:9
[task 2018-03-23T23:07:58.953Z] 23:07:58     INFO - @http://web-platform.test:8000/css/css-tables/table-model-fixup-2.html:252:9
Flags: needinfo?(james)
Whiteboard: [stockwell unknown] → [stockwell needswork]
Flags: needinfo?(james)
In the last 7 days we have 57 failures.
They occur on linux64-qr and the affected builds type are opt and debug.
Here is a recent failure log example: https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=171345951&lineNumber=2124
Starting with 1st of April we dont have any failures in O.F. but this is probably related with this weekend's holiday.
:jet, Hi, can you have a look at this?
Flags: needinfo?(bugs)
(In reply to Eliza Balazs [:ebalazs_] from comment #16)
> :jet, Hi, can you have a look at this?

The link in comment 13 explains why this test is flaky. We'll need a fix from upstream to repair it.
Flags: needinfo?(bugs)
:jmaher Should this test be disabled or turned off? Taking in consideration that this test is buggy. See comment 13 https://bugzilla.mozilla.org/show_bug.cgi?id=1444203#c13
Flags: needinfo?(jmaher)
Arthur- lets disable this test- without people fixing this upstream we need to make our test results sane in the short term.
Flags: needinfo?(jmaher)
patch to disable this on bug 1445164
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → DUPLICATE
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]
You need to log in before you can comment on or make changes to this bug.