Intermittent parser/htmlparser/tests/reftest/bug582940-1.html == parser/htmlparser/tests/reftest/bug582940-1-ref.html | image comparison, max difference: 255, number of differing pixels: 647

NEW
Unassigned

Status

()

P2
normal
a year ago
8 months ago

People

(Reporter: intermittent-bug-filer, Unassigned)

Tracking

({intermittent-failure, leave-open})

unspecified
intermittent-failure, leave-open
Points:
---

Firefox Tracking Flags

(firefox58 disabled, firefox59 verified)

Details

(Whiteboard: [stockwell disabled])

Attachments

(1 attachment)

3 failures in 1008 pushes (0.003 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 3

Platform breakdown:
* macosx64-stylo: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-07-24&endday=2017-07-30&tree=all
10 failures in 888 pushes (0.011 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 7
* mozilla-central: 2
* mozilla-inbound: 1

Platform breakdown:
* macosx64-stylo: 10

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-07-31&endday=2017-08-06&tree=all
12 failures in 901 pushes (0.013 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 7
* try: 4
* mozilla-central: 1

Platform breakdown:
* macosx64-stylo: 12

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-08-07&endday=2017-08-13&tree=all
12 failures in 949 pushes (0.013 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 8
* try: 2
* mozilla-central: 2

Platform breakdown:
* macosx64-stylo: 12

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-08-14&endday=2017-08-20&tree=all
12 failures in 908 pushes (0.013 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 7
* mozilla-central: 3
* try: 2

Platform breakdown:
* macosx64-stylo: 11
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-08-21&endday=2017-08-27&tree=all
17 failures in 939 pushes (0.018 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 9
* try: 5
* mozilla-central: 3

Platform breakdown:
* macosx64-stylo: 17

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-08-28&endday=2017-09-03&tree=all
21 failures in 924 pushes (0.023 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 10
* mozilla-central: 5
* try: 3
* mozilla-inbound: 3

Platform breakdown:
* osx-10-10: 10
* macosx64-stylo: 9
* macosx64-nightly: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-09-04&endday=2017-09-10&tree=all
19 failures in 1032 pushes (0.018 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 7
* autoland: 7
* mozilla-inbound: 4
* try: 1

Platform breakdown:
* osx-10-10: 19

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-09-11&endday=2017-09-17&tree=all
25 failures in 943 pushes (0.027 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 9
* autoland: 7
* mozilla-central: 4
* mozilla-beta: 4
* try: 1

Platform breakdown:
* osx-10-10: 22
* macosx64-devedition: 2
* macosx64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-09-18&endday=2017-09-24&tree=all
22 failures in 885 pushes (0.025 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 5
* mozilla-beta: 5
* mozilla-central: 4
* autoland: 4
* oak: 3
* try: 1

Platform breakdown:
* osx-10-10: 17
* macosx64-nightly: 4
* macosx64-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-09-25&endday=2017-10-01&tree=all
35 failures in 824 pushes (0.042 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* mozilla-inbound: 15
* autoland: 11
* mozilla-beta: 5
* mozilla-central: 3
* try: 1

Platform breakdown:
* osx-10-10: 30
* macosx64-devedition: 3
* macosx64-nightly: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-10-02&endday=2017-10-08&tree=all
Priority: P5 → --
Whiteboard: [stockwell needswork:owner]

Comment 13

10 months ago
Alphan, would you like to give it a try?
Flags: needinfo?(alchen)

Updated

10 months ago
Assignee: nobody → alchen
Flags: needinfo?(alchen)
38 failures in 947 pushes (0.04 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 16
* mozilla-inbound: 11
* try: 6
* mozilla-beta: 3
* mozilla-central: 2

Platform breakdown:
* osx-10-10: 35
* macosx64-devedition: 2
* macosx64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-10-09&endday=2017-10-15&tree=all

Updated

10 months ago
Priority: -- → P2
thanks for working on this :alchen :)
41 failures in 864 pushes (0.047 failures/push) were associated with this bug in the last 7 days. 

This is the #29 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* mozilla-inbound: 18
* autoland: 17
* mozilla-central: 3
* mozilla-beta: 3

Platform breakdown:
* osx-10-10: 38
* macosx64-nightly: 2
* macosx64-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-10-16&endday=2017-10-22&tree=all
this has 156 failures in the last 30 days, any update or eta?
Flags: needinfo?(alchen)
21 failures in 196 pushes (0.107 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 15
* mozilla-inbound: 3
* try: 2
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 20
* macosx64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-10-26&endday=2017-10-26&tree=all

Comment 19

10 months ago
(In reply to Joel Maher ( :jmaher) (UTC-5) from comment #17)
> this has 156 failures in the last 30 days, any update or eta?

Sorry about the delay. I will have some updates next week.
Flags: needinfo?(alchen)

Comment 20

10 months ago
After investigation, the result is positive for parser part.
They both go to the right anchor. (Ref!)

However, there is displacement in showing the second "Filler" after "Ref!" which cause the image comparison failed.
It should be related to layout or render part.

Comment 21

10 months ago
(In reply to Alphan Chen [:alchen] from comment #20)
> After investigation, the result is positive for parser part.
> They both go to the right anchor. (Ref!)
>
The result can be found in the following link. 
https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/bKW5HaPOSjSZpyZHx9mKfw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

Comment 22

10 months ago
Created attachment 8922663 [details]
Failure Screen Shot

Two images are scrolled to the right anchor.

Comment 23

10 months ago
Hi Astley,
as I said in previous comments, the failure is not caused by purpose of this test.
There is displacement in showing some elements.
could you take a look about this bug?


The failure only happened on macosx.
https://brasstacks.mozilla.com/orangefactor/?display=BugCount&includefiltertype=quicksearch&includefilterdetailsexcludeResolved=false&includefilterdetailsexcludeDisabled=false&includefilterdetailsquicksearch=bug582940-1.html%20&includefilterdetailsnumbugs=0&includefilterdetailsresolvedIds=&excludefiltertype=quicksearch&excludefilterdetailsquicksearch=&excludefilterdetailsnumbugs=0&excludefilterdetailsresolvedIds=&key=platform&tree=all&startday=2017-10-19&endday=2017-10-26

In the beginning, only on macosx64-stylo.
(In reply to OrangeFactor Robot from comment #1)
> 3 failures in 1008 pushes (0.003 failures/push) were associated with this
> bug in the last 7 days.   
> 
> Platform breakdown:
> * macosx64-stylo: 3

(In reply to OrangeFactor Robot from comment #3)
> 10 failures in 888 pushes (0.011 failures/push) were associated with this
> bug in the last 7 days.   
> 
> Platform breakdown:
> * macosx64-stylo: 10

(In reply to OrangeFactor Robot from comment #4)
> 12 failures in 901 pushes (0.013 failures/push) were associated with this
> bug in the last 7 days.   
> 
> Platform breakdown:
> * macosx64-stylo: 12

(In reply to OrangeFactor Robot from comment #5)
> 12 failures in 949 pushes (0.013 failures/push) were associated with this
> bug in the last 7 days.   
> 
> Platform breakdown:
> * macosx64-stylo: 12

(In reply to OrangeFactor Robot from comment #7)
> 17 failures in 939 pushes (0.018 failures/push) were associated with this
> bug in the last 7 days.   
> 
> Platform breakdown:
> * macosx64-stylo: 17
Assignee: alchen → nobody
Component: XML → Layout
Flags: needinfo?(bmo)
16 failures in 151 pushes (0.106 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 13
* mozilla-inbound: 3

Platform breakdown:
* osx-10-10: 16

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-10-27&endday=2017-10-27&tree=all
70 failures in 912 pushes (0.077 failures/push) were associated with this bug in the last 7 days. 

This is the #16 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 47
* mozilla-inbound: 15
* mozilla-central: 4
* try: 3
* mozilla-beta: 1

Platform breakdown:
* osx-10-10: 66
* macosx64-nightly: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-10-23&endday=2017-10-29&tree=all

Comment 26

10 months ago
(In reply to OrangeFactor Robot from comment #25)
> 70 failures in 912 pushes (0.077 failures/push) were associated with this
> bug in the last 7 days. 
> 
> This is the #16 most frequent failure this week.  
> 
> ** This failure happened more than 30 times this week! Resolving this bug is
> a high priority. **

Xidorn, do you have any idea why stylo could cause this failure intermittently?
Flags: needinfo?(bmo) → needinfo?(xidorn+moz)
Had a brief look at the comparison result, and don't immediately have any thought on why this happens. This may be related to font metrics being rounded differently, although I'm not sure why that could be nondeterministic. Maybe Manish has some insight about whether there is anything could go wrong with font-size.

If this is easy to reproduce on try server, maybe we can test with different font and see if this still happens for mitigation.
Flags: needinfo?(xidorn+moz)
Yeah, I don't know why this would happen.
23 failures in 171 pushes (0.135 failures/push) were associated with this bug yesterday.    

** This test has failed more than 200 times in the last 30 days. It should be disabled until it can be fixed. ** 

Repository breakdown:
* autoland: 12
* mozilla-inbound: 10
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 23

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-10-30&endday=2017-10-30&tree=all
Whiteboard: [stockwell needswork:owner] → [stockwell disable-recommended]

Comment 30

10 months ago
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4f39d5874392
Disable parser/htmlparser/tests/reftest/bug582940-1.html on osx debug for frequent failures. r=me, a=testonly
I disabled this test for having >200 failures in the last 30 days, please remember to enable this when testing a fix
Keywords: leave-open
Whiteboard: [stockwell disable-recommended] → [stockwell disabled]
15 failures in 169 pushes (0.089 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 8
* mozilla-inbound: 4
* try: 2
* mozilla-beta: 1

Platform breakdown:
* osx-10-10: 14
* macosx64-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-10-31&endday=2017-10-31&tree=all
18 failures in 180 pushes (0.1 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 11
* mozilla-inbound: 3
* mozilla-beta: 2
* try: 1
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 16
* macosx64-nightly: 1
* macosx64-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-11-01&endday=2017-11-01&tree=all

Comment 35

10 months ago
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/d3f290bc08ee
Disable parser/htmlparser/tests/reftest/bug582940-1.html on osx for frequent failures. r=me, a=testonly
106 failures in 857 pushes (0.124 failures/push) were associated with this bug in the last 7 days. 

This is the #16 most frequent failure this week. 

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **   

Repository breakdown:
* autoland: 59
* mozilla-inbound: 30
* try: 7
* mozilla-central: 7
* mozilla-beta: 3

Platform breakdown:
* osx-10-10: 103
* macosx64-devedition: 2
* macosx64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-10-30&endday=2017-11-05&tree=all
There are 90 failures in the last week.
All the failures occur on the OS X 10.10 platform.
The most affected build type is debug, but there are also 10 failures on opt.
Here is a recent log file:
https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-central&job_id=143972900&lineNumber=34410

And a snippet from it:
02:55:33     INFO - [Child 738, Main Thread] WARNING: Unsupported TextureClient backend type: 'aMoz2DBackend != gfx::BackendType::SKIA && aMoz2DBackend != gfx::BackendType::DIRECT2D && aMoz2DBackend != gfx::BackendType::DIRECT2D1_1', file /builds/worker/workspace/build/src/gfx/layers/client/TextureClient.cpp, line 1255
34407
02:55:33     INFO - [Child 738, Main Thread] WARNING: Unsupported TextureClient backend type: 'aMoz2DBackend != gfx::BackendType::SKIA && aMoz2DBackend != gfx::BackendType::DIRECT2D && aMoz2DBackend != gfx::BackendType::DIRECT2D1_1', file /builds/worker/workspace/build/src/gfx/layers/client/TextureClient.cpp, line 1255
34408
02:55:33     INFO - [Child 738, Main Thread] WARNING: Unsupported TextureClient backend type: 'aMoz2DBackend != gfx::BackendType::SKIA && aMoz2DBackend != gfx::BackendType::DIRECT2D && aMoz2DBackend != gfx::BackendType::DIRECT2D1_1', file /builds/worker/workspace/build/src/gfx/layers/client/TextureClient.cpp, line 1255
34409
02:55:34     INFO - REFTEST INFO | REFTEST fuzzy test (0, 0) <= (255, 647) <= (2, 5)
34410
02:55:34     INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///Users/cltbld/tasks/task_1510396492/build/tests/reftest/tests/parser/htmlparser/tests/reftest/bug582940-1.html == file:///Users/cltbld/tasks/task_1510396492/build/tests/reftest/tests/parser/htmlparser/tests/reftest/bug582940-1-ref.html | image comparison, max difference: 255, number of differing pixels: 647
Flags: needinfo?(bugs)
it looks like my attempt to disable this test didn't work:
https://hg.mozilla.org/mozilla-central/rev/d3f290bc08ee

I will figure it out, I think there is an issue with skip-if and fuzzy-if in the reftest manifest where fuzzy-if will negate any skip-if.
87 failures in 849 pushes (0.102 failures/push) were associated with this bug in the last 7 days. 

This is the #16 most frequent failure this week. 

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **   

Repository breakdown:
* autoland: 39
* mozilla-inbound: 31
* mozilla-central: 14
* mozilla-release: 2
* try: 1

Platform breakdown:
* osx-10-10: 85
* macosx64-nightly: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-11-06&endday=2017-11-12&tree=all

Updated

9 months ago
Flags: needinfo?(bugs)
ok, we cannot skip-if and fuzzy-if on the same line:
https://searchfox.org/mozilla-central/source/layout/tools/reftest/globals.jsm#42

fuzzy-if will take precedence and be used throwing out the skip-if clause- so the only thing that I know is to disable fully on all tests.

Comment 42

9 months ago
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/dc331b8af3a7
Disable parser/htmlparser/tests/reftest/bug582940-1.html on osx for frequent failures. r=me, a=testonly
tried flipping skip-if and fuzzy-if, I am skeptical, but will watch and disable fully if this doesn't work.
Flags: needinfo?(jmaher)
my skepticism was uncalled for, this worked:
0:01:42     INFO - REFTEST TEST-START | file:///Users/cltbld/tasks/task_1510594843/build/tests/reftest/tests/parser/htmlparser/tests/reftest/bug582940-1.html == file:///Users/cltbld/tasks/task_1510594843/build/tests/reftest/tests/parser/htmlparser/tests/reftest/bug582940-1-ref.html
10:01:42     INFO - REFTEST TEST-SKIP | file:///Users/cltbld/tasks/task_1510594843/build/tests/reftest/tests/parser/htmlparser/tests/reftest/bug582940-1.html == file:///Users/cltbld/tasks/task_1510594843/build/tests/reftest/tests/parser/htmlparser/tests/reftest/bug582940-1-ref.html | 
10:01:42     INFO - REFTEST TEST-END | file:///Users/cltbld/tasks/task_1510594843/build/tests/reftest/tests/parser/htmlparser/tests/reftest/bug582940-1.html == file:///Users/cltbld/tasks/task_1510594843/build/tests/reftest/tests/parser/htmlparser/tests/reftest/bug582940-1-ref.html

I am going to assume this is righfully disabled and the failures should be gone now.
Flags: needinfo?(jmaher)
https://hg.mozilla.org/releases/mozilla-beta/rev/70b42ae89f23
status-firefox58: --- → disabled
status-firefox59: --- → verified
8 failures in 762 pushes (0.01 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 5
* mozilla-inbound: 2
* try: 1

Platform breakdown:
* osx-10-10: 8

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-11-13&endday=2017-11-19&tree=all
1 failures in 744 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-release: 1

Platform breakdown:
* macosx64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-11-20&endday=2017-11-26&tree=all
2 failures in 792 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-release: 2

Platform breakdown:
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-11-27&endday=2017-12-03&tree=all
1 failures in 889 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-release: 1

Platform breakdown:
* macosx64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1385288&startday=2017-12-04&endday=2017-12-10&tree=all
You need to log in before you can comment on or make changes to this bug.