394517 - Brute Force Leak Detection [meta]

Status: RESOLVED FIXED

(Core :: General, defect)

Description

[Robert Sayre]

Attachment: wip python script

Use scripted visits to random URLs to detect leaks.

Just doing this serially right now, to work out obvious leaks and perfect the scripts. Should probably add Jesse's multi_timed_run.py. Planning on running this in parallel on Amazon EC2.

Comment 1

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Comment on attachment 279176 [details]
wip python script

Just FYI, 'XPCOM_NO_REMOTE' doesn't do anything that I'm aware of. I think you want 'MOZ_NO_REMOTE'.

Comment 2

[Robert Sayre]

(In reply to comment #1)
> (From update of attachment 279176 [details])
> Just FYI, 'XPCOM_NO_REMOTE' doesn't do anything that I'm aware of. I think you
> want 'MOZ_NO_REMOTE'.
> 

huh, I guess that's why I needed to pass the command line param too. Oops.

Comment 3

[Bug Filler]

It should be able to read a user's history database and loop all the urls. 

That way, a user can find out if the url's he normally visit leaks anything.

Comment 4

[Ray Kiddy]

FYI, I had done some testing. The report of that testing is at:

http://www.wykiwyk.com/mozilla/leakTesting/testlog_20070921_summary.html

These tests are from all the nightly builds between Minefield_3.0a4pre_20070413.app and Minefield_3.0a8pre_20070908.app.

The testing I just did on Minefield_3.0a9pre_20070922.app is at:

http://www.wykiwyk.com/mozilla/leakTesting/testlog_20070928_all.html

Leaks in 09/22 nightly were:

http://www.5show.com/
http://www.flogao.com.br/
http://www.grono.net/
http://www.hinet.net/
http://www.msn.ca/
http://www.msn.com/
http://www.tudou.com/

My script launches the app, loads the URL and then shuts down the app. Or rather, the app is shut down by an extension I made for just that purpose. Are your tests shutting down the app after each load? If not, how are you guaranteed that there are not interaction effects caused by loading more than one page?

Also, you are testing only local content? I am testing the Alexa 500 list of URLs.

FYI, I would be interested in helping to create a systematic way of tracking these test results, but if people do not talk to me, it won't happen in a way that helps MoCo. Hearing nothing, I do only what I feel like doing.

Comment 5

[Robert Sayre]

(In reply to comment #4)
> FYI, I had done some testing. The report of that testing is at:
> 
> http://www.wykiwyk.com/mozilla/leakTesting/testlog_20070921_summary.html
> 
> These tests are from all the nightly builds between
> Minefield_3.0a4pre_20070413.app and Minefield_3.0a8pre_20070908.app.
> 
> The testing I just did on Minefield_3.0a9pre_20070922.app is at:
> 
> http://www.wykiwyk.com/mozilla/leakTesting/testlog_20070928_all.html
> 
> Leaks in 09/22 nightly were:

We've been fixing leaks at a pretty fast clip. For example, we fixed grono.net in Bug 397386 on 9/24. I don't see leaks on any of your examples except for 

> http://www.tudou.com/

which is known and filed. Other examples available at http://wiki.mozilla.org/LeakingPages.

> 
> My script launches the app, loads the URL and then shuts down the app. Or
> rather, the app is shut down by an extension I made for just that purpose. Are
> your tests shutting down the app after each load? 

Yeah, I use a variation on your loadAndDie extension.

> 
> Also, you are testing only local content? I am testing the Alexa 500 list of
> URLs.

See the script attachment. It uses random URLs from a variety of services.

Comment 6

[Bug Filler]

Attachment: fetch url from a places.sqlite file

This module can be used to fetch visited urls from an user's profile so that an user can test all his/her favorite sites.

Comment 7

[Ray Kiddy]

See http://www.wykiwyk.com/mozilla/leakTesting/testlog_20071004_summary.html

This was with the nightly:

Minefield_3.0a9pre_20071001.app/Contents/MacOS/firefox

Here were the leaks in this build:

http://www.hinet.net/ leaks: DOMWindows:6/7 documents:29/40 docshells:3/3
http://www.gamer.com.tw/ leaks: DOMWindows:0/9 documents:0/38 docshells:1/4
http://www.mapy.cz/ leaks: DOMWindows:8/9 documents:32/41 docshells:4/4
http://www.msn.ca/ leaks: DOMWindows:2/7 documents:0/38 docshells:0/3
http://www.msn.com/ leaks: DOMWindows:2/7 documents:0/38 docshells:0/3
http://www.xunlei.com/ leaks: DOMWindows:6/7 documents:29/37 docshells:3/3