Closed Bug 649155 Opened 14 years ago Closed 13 years ago

Investigate FastestFox results

Categories

(Testing :: Talos, defect)

Product:
Testing
Component:
Talos
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: fligtar, Unassigned)

References

Details

Automated testing of FastestFox shows a start-up impact of 32% with the following data: mysql> select * from perf_results where addon_id=9825; +------+----------+---------------+--------------+---------+------+---------------------+---------------------+ | id | addon_id | appversion_id | osversion_id | average | test | created | modified | +------+----------+---------------+--------------+---------+------+---------------------+---------------------+ | 429 | 9825 | 23 | 1 | 771.32 | ts | 2011-03-19 07:15:29 | 2011-03-19 07:15:29 | | 401 | 9825 | 23 | 2 | 599.37 | ts | 2011-03-19 07:12:22 | 2011-03-19 07:12:22 | | 705 | 9825 | 23 | 3 | 451.47 | ts | 2011-03-19 07:50:33 | 2011-03-19 07:50:33 | | 471 | 9825 | 23 | 4 | 526.47 | ts | 2011-03-19 07:21:02 | 2011-03-19 07:21:02 | | 1570 | 9825 | 24 | 1 | 819.47 | ts | 2011-04-09 07:26:17 | 2011-04-09 07:26:17 | | 1448 | 9825 | 24 | 2 | 678.79 | ts | 2011-04-09 07:15:51 | 2011-04-09 07:15:51 | | 1750 | 9825 | 24 | 3 | 566.26 | ts | 2011-04-09 07:49:26 | 2011-04-09 07:49:26 | | 1535 | 9825 | 24 | 4 | 732.95 | ts | 2011-04-09 07:22:39 | 2011-04-09 07:22:39 | +------+----------+---------------+--------------+---------+------+---------------------+---------------------+ 8 rows in set (0.19 sec) Manually testing start-up time we haven't been able to obtain similar results. We've removed the FastestFox results so that it doesn't appear on the slow performers page until we can figure out whether there's a problem in the automated testing. Please investigate. Thanks.
Depends on: 649153
I'm seeing the slow startup as reported by talos. FastestFox is opening a second tab: http://smarterfox.com/smarterwiki/installed/?from_ver=firstrun&to_ver=4.3.7 It brings up a 404. I'm not sure if it is just the overhead of the 404 affecting the results or if there is something else going on here.
I'm also seeing these console errors: Could not read chrome manifest file '/Users/cltbld/talos-slave/talos-data/Firefox.app/Contents/MacOS/extensions/{972ce4c6-7e08-4474-a285-3208198ce6fd}/chrome.manifest'. While creating services from category 'profile-after-change', service for entry '@smarterwiki/http-observer;1', contract ID '@smarterwiki/http-observer;1' does not implement nsIObserver.
Not sure where else to investigate here... FastestFox doesn't appear to be adding any unusual prefs to the browser - not sure if it attempting to preload something or other as that seems to be part of its modus operandi. I think that being able to disable the first run tab would be a good start.
I ran some tests on my Mac OS machine and they also seem consistent with the results on the performance page. I also tried setting the preference that prevents the first run page from opening, but that didn't make a big difference in startup performance. I'm also curious as to why the performance overhead on Mac is roughly half the one on Windows. Alice, can you test on Windows with and without this preference set: extensions.smarterwiki.last-version : 4.3.7 I'm interested in knowing if this makes a major difference in the measured impact.
I'm trying to get those tests run, but having some difficulty with our staging boxes. I'll get the data posted as quickly as I can gather it.
Using nightly firefox-4.0.2pre.en-US.win32.zip (pulled today) on winxp: without pref: i,val 0,712 1,539 2,590 3,576 4,522 5,526 6,592 7,542 8,544 9,516 10,536 11,516 12,544 13,672 14,574 15,537 16,520 17,531 18,537 19,525 Average excluding max: ts: 549.42 With pref: i,val 0,685 1,532 2,516 3,523 4,678 5,527 6,541 7,551 8,519 9,577 10,531 11,556 12,525 13,511 14,542 15,537 16,592 17,539 18,526 19,512 Average excluding max: ts-pref: 543.95 Any help here? I can run the tests against another build if you need more info.
Aren't these averages much lower than those on comment #0, and more in line with the results in other systems?
It could be in line with the results in comment #0, I'm just not sure what os_version maps to which os.
mysql> select id,os,version,name from perf_osversions; +----+--------+------------------+-----------------+ | id | os | version | name | +----+--------+------------------+-----------------+ | 1 | MacOSX | 10.5.8 | Mac OS X 10.5.8 | | 2 | Fedora | 12 - Constantine | Fedora 12 | | 3 | WINNT | 5.1 | Windows XP | | 4 | WINNT | 6.1 | Windows 7 | +----+--------+------------------+-----------------+ 4 rows in set (0.00 sec) mysql> select id,app,version from perf_appversions; +----+------+-----------------+ | id | app | version | +----+------+-----------------+ | 1 | | browser_version | | 2 | | browser_version | | 3 | | browser_version | | 4 | | browser_version | | 5 | | browser_version | | 6 | | browser_version | | 7 | | browser_version | | 8 | | browser_version | | 9 | | browser_version | | 10 | | browser_version | | 11 | | browser_version | | 12 | | browser_version | | 13 | | browser_version | | 14 | | browser_version | | 15 | | browser_version | | 16 | | browser_version | | 17 | | browser_version | | 18 | | browser_version | | 19 | | browser_version | | 20 | | browser_version | | 21 | fx | 3.6.15pre | | 22 | fx | 4.0b13pre | | 23 | fx | 3.6.15 | | 24 | fx | 4.0 | | 25 | fx | 4.0.1 | +----+------+-----------------+ 25 rows in set (0.00 sec)
Thanks, Wil. I assumed XP was 1, so I was wrong in Comment #7. It also looks like the difference with and without the preference on Windows is not that big, so it wouldn't be worth including it in the default configuration. I don't know what else we can do here. We should probably close this bug since it appears that Talos is doing the right thing.
Based upon comment 10, closing.
Status: NEW → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.