758654 - BrowserMark benchmark test fails on Fennec

Status: RESOLVED WONTFIX

(Firefox for Android Graveyard :: Testing, defect, P3)

Description

[Thomas G Eaton]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Build ID: 20111008085652

Steps to reproduce:

I installed fennec and then tried to run the BrowserMark benchmark.  I tried this on an Intel Tablet, Phone, and Samsung Galaxy Nexus.  I installed a version I built from the trunk on the Intel phones and the Beta release from the app store on the Nexus.  


Actual results:

It ran fine until it got to "RUNNING SUITE STRING 5 of 6."  At that point the benchmark errors out and goes to a screen stating, "An unexpected error occurred.  Please make sure you have enabled cookies in your browser and try again."  I verified on both devices that cookies were enabled.  Subsequent runs report "There was something wrong with your benchmark results, Please try again."  Those runs are still crashing on the string test though.  


Expected results:

The benchmark should have run to completion with no issues.

Comment 1

[Thomas G Eaton]

I continuously ran the benchmark while filing this bug to make sure everything was reported correctly.  It suddenly completed successfully on the x86 tablet.  I tried a few more runs on the Nexus and it eventually completed as well.  The x86 phone still hasn't completed the benchmark.

Comment 2

[Aaron Train [:aaronmt]]

Mark/Brad, thoughts?

Comment 3

[(not currently active) Ted Mielczarek]

Probably needs someone to figure out where inside the benchmark we're failing. It could be something being miscompiled by the NDK compiler. I'd find it harder to believe it was a platform issue, since lots of the core Gecko code doesn't care about what platform it's on.

Comment 4

[Kevin Brosnan [Ex-Mozilla]]

This has failed for me on ARM hardware as well. A debug build might spit out enough info to get a lead.

Comment 5

[Rob]

I was going to file on this also, but I checked for Dupes.

I did get one successful(?) run of BrowserMark with a reasonable result.

Other runs usually fail at the 'Spinning Cube' (first Tests) and do not get to the 'Web Page' (with an Article and Photos about Space). Rarely the second Test will succeed, and we get to the 'CSS Tests'. Usually that is about all you will see.

Sometimes (despite seeing 'broken Tests' and not seeing some of the Tests run) the Result is given (and reasonable) and other times the Result is not given (and it 'crashes back' (goes back one Link, without using the Back Button) to the Main Page.

In one instance the Tests completed (with some Tests not shown), and it produced a Benchmark Result that was DOUBLE of what I expected (now we are causing problems for others).

One Example Result was like this:

"Phone Unknown 1 with Unknown browser score was:
3223
Phone Unknown 1 with Unknown Browser is superior to 100% of all Phone browsers"
.

As you can see there is a different (unimportant to this Bug) issue in the 'Hardware and Browser Identification' which does not know my Phone is an "HTC One". Sometimes when the Test completes the Browser CAN be identified, and in those instances it is thought to be "Firefox for Mobile 24".


I am using today's Nightly version of Firefox for Android.


Since Nightly renders the first Test incorrectly (and often crashes there) that should reduce the effort needed to start on this Bug.

Comment 6

[Rob]

Rebooting the Phone and running Nightly first greatly increases the chances of BowserMark completing all tests. I did try BrowserMark with Google's Browser and they do no better than us, the Tests often do not run correctly there either.

I notice the 'Ancient City' is rendered upsidedown, that should be easy to fix.

Comment 7

[Rob]

Nightly 31.0a1 (2014-03-18) on Android (Google Street Map) causing GL errors also:
bp-9f773aba-3573-4999-91c8-81b7221-40319
bp-f28a55a6-841d-4b56-b9f6-9430021-40316

Note: The [Preview] Tab does not interpret BPs (into Links), and that was a complex number to type. I hope they are correct, but I will test the BP's (on Desktop) to ensure they are correct. I hate sending double mails (to ~3 dozen people).

Comment 8

[Rob]

(In reply to Rob from comment #7)
> Nightly 31.0a1 (2014-03-18) on Android (Google Street Map) causing GL errors
> also:
> bp-9f773aba-3573-4999-91c8-81b7221-40319
> bp-f28a55a6-841d-4b56-b9f6-9430021-40316
> 
> Note: The [Preview] Tab does not interpret BPs (into Links), and that was a
> complex number to type. I hope they are correct, but I will test the BP's
> (on Desktop) to ensure they are correct. I hate sending double mails (to ~3
> dozen people).

Ah, that explains why (BPs where not previewed). I need a magnifier to read my Phone's Screen even when fully Zoomed in (it is 1080P). The [Help][About] does not zoom at all (will file seperately).

Correct BPs:

bp-9f773aba-3573-4999-91c8-81b722140319
bp-f28a55a6-841d-4b56-b9f6-943002140316

Comment 9

[Kevin Brosnan [Ex-Mozilla]]

Both are bug 978492.

Comment 10

[Rob]

(In reply to Kevin Brosnan [:kbrosnan] from comment #9)
> Both are bug 978492.

And now they are Bug 952721 - crash in gfxContext::gfxContext(mozilla::gfx::DrawTarget*)  .

It seems to change around a fair bit.

Comment 11

[Geoff Brown [:gbrown]]

Are we still interested in this test on Fennec? Are there still problems?