Closed Bug 493057 Opened 12 years ago Closed 12 years ago

Run fennecmark regularly


(Firefox for Android Graveyard :: General, defect)

Not set


(Not tracked)



(Reporter: taras.mozilla, Assigned: jmaher)



(3 files)

No description provided.
Here is a blog post describing it

Basically it is a benchmark that measures and outputs interesting performance numbers, would like to get those numbers graphed for easy comparison.
Attached patch FennecMark 1.0Splinter Review
This is the first round of fennecmark 1.0.  This reports two numbers:
1) median pan lag time
2) median zoom lag time

There is an option to report a third number: total pan time.

This patch needs to be applied at the root of this repository:

Developed and tested on Ubuntu Linux.  For the .jar file, this makes a clean install, but we need to resolve the core source code location.
Assignee: nobody → jmaher
Joel: have you run these at all on an n810?
I'm hitting timeouts for both tests.  Bumped the timeout to 300, still timing out; trying 1200 now.
Ok, now I'm getting a 404 for http://localhost/page_load_test/pages/en.wikipedia.org_wiki_Main_Page/

This page doesn't exist in the talos pageset on the devices. Did you get this from somewhere or is it in the standalone pageset?

Is there something special about this page or could I choose any page off the existing pageset?
updating the loadURI to point to a page on your device works just fine!  The one problem I do have (while running with -d -n) could be this:

EBUG: Working with test: tzoom
DEBUG: Sending results: 855
Generating results file: tzoom: 
		Started Wed, 10 Jun 2009 17:52:42
Traceback (most recent call last):
  File "", line 541, in <module>
  File "", line 520, in test_file
    links = send_to_graph(results_server, results_link, title, date, browser_config, results)
  File "", line 229, in send_to_graph
    files.append(construct_file(machine, fullname, browser_config['branch_name'], browser_config['sourcestamp'], browser_config['buildid'], date, vals))
KeyError: 'branch_name'
Ok, I'm not sure if you got this via IRC; putting here for clarity.

[10:54]	<aki>	i don't think i set branchName
[10:56]	oh, apparently -b sets both branch and branch_name
[10:58]	jmaher: i run |python -v -e /media/mmc1/fennec/fennec -t `hostname` -b mobile-browser --activeTests tzoom --sampleConfig mobile.config --browserWait 60 --noChrome --oldresultsServer --oldresultsLink /server/bulk.cgi --output local.config|
[11:02]	[ERROR]	Connection to ircs:// (ircs:// timed out. [Help]
I will poke around the PerfConfigurator a bit to see if there is something we can do.  Is there a chance that the server needs to be configured for the new test name?
I *think* the current graph server will take new tests without any manual intervention. Alice?

We are moving to a new graph server for Firefox talos.  There's a whole bunch of stuff we'll have to do before we can report there from the mobile devices... maybe Q3, maybe not.  To be tracked in bug 487335.
in #perfomatic, alice helped me debug the issue.  bug 497922 address the issue required to fix this.

Basically in there is a hardcoded list of tests 'ts' and 'twinopen' that are used to match tests that report in a certain method.

modifying this list of tests to include tzoom and tpan appear to work: <- tzoom.

I also increased the cycles from 1 to 5 in order to get that result.  I am going to recommend for 10 cycles.

Unfortunately this requires another file change which will need to get rolled out to all the talos devices.
this patch applies to the tree.  if you run ./ inside of tpan/fennecmark, you will get a .jar file.  Copy the fennecmark.jar and fennecmark.manifest to the tpan/chrome folder.

Also I have fixed the noisy output of tpan.js. I suspect this might fix the results with tpan
This is checked in and I've pushed the new jar to the n810s.
When running, I don't get a fennecmark.manifest, however.
This adds fennecbench.manifest to the tpan/fennecmark directory.  This will be inline with the instructions that display.
That's checked in, and tests are running.
Closed: 12 years ago
Resolution: --- → FIXED
verfied with beta3.  There is a bug tracking the issues with the latest updates.
You need to log in before you can comment on or make changes to this bug.