1331866 - Same result on stress testing website with the HW acceleration ON or OFF

Status: REOPENED

(Core :: Graphics: Canvas2D, defect, P3)

Description

[Mihai Boldan, Desktop QA [:mboldan]]

[Affected versions]:
- Firefox 53.0a1 (2017-01-17)

[Affected platforms]:
- Mac OS X 10.12.1, Mac OS X 10.10.4

[Steps to reproduce]:
1. Launch Firefox.
2. Make sure that the HW acceleration is enabled(Preferences->Advanced).
3. Open http://www.kevs3d.co.uk/dev/canvasmark/ and run the test.
4. Disable HW acceleration and restart FF.
5. Run again the test from http://www.kevs3d.co.uk/dev/canvasmark/. 

[Expected result]:
- The results from the tests are different (at least 4000 points).

[Actual result]:
- The results are almost the same.

[Regression range]:
- Not quite sure if this is a regression. I will investigate asap.

[Additional notes]:
- This issue is not reproducible on Latest Aurora build.

Comment 1

[George Wright (:gw280) (needinfo me!)]

I can't reproduce this on either current nightly, or build id 20170117030218. I see ~5000-6000 for accelerated, and ~10,000 for unaccelerated.

Comment 2

[Oana Botisan, Desktop QA]

I ran the test on macOS 10.12. 
I tested on several releases versions and the results were as they follow: 
 * 52.0 - pass,
 * 53.0 - pass,  
 * 54.0 - pass, 
 * 55.0 - pass, 
I also tested on some versions of Nightly and the results are:
 * 2016-09-20 - failed
 * 2016-10-15 - failed
 * 2016-11-08 - pass
 * latest Nightly - failed

I ran the tests with mozregression too, but I don't think they were accurate, because it said that the Nightly build from 2016-11-08 was a fail and as you can see from the test I did manually that one passed. 
In conclusion, I don't think that we can run this test with mozregression and only some builds are affected. I think it's impossible to narrow it down. But the best part is that the release versions seem not to be affected by this bug.

Comment 3

[Oana Botisan, Desktop QA]

I ran the test on Firefox 56.0, beta 57.0b5 and latest Nightly 58 and the bug is not reproducing anymore. I think somewhere along the way, the problem was solved. 
I am going to close the bug as worksforme, but if there are any similar issues you can reopen it.

Comment 4

[Oana Botisan, Desktop QA]

After more investigations (and rewiring my brain) I found that the bug is still reproducing. The difference between the results is smaller than 4000. The difference is around 3000 for all the three versions of Firefox.
Under this circumstances I am reopening the bug.

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee didn't login in Bugzilla in the last 7 months, so the assignee is being reset.