384269 - Enabling libxul regressed Tp2 on Mac badly

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Adam Guthrie]

Note that the Linux tinderbox switched what build it was testing, but it's still the same box testing the builds.

Linux:
Tp: ~635ms -> ~700ms = +10%
Tp2: ~475ms -> ~525ms = +10%
Tdhtml: ~1180ms -> ~1240ms = +5%
Txul: ~730ms -> ~765ms = +4.7%
Ts: sorta noisy, but it went down a bit

Mac:
Tp: no regression
Tdhtml: ~410ms -> ~425ms = +3.6%
Txul: ~145ms -> 150ms = +3%
Ts: ~840ms -> 900ms = +7%

Windows seems to have improved on all fronts.

Comment 1

[Stuart Parmenter]

We should back out the libxul changes until this can be resolved.

Comment 2

[Benjamin Smedberg]

The Linux changes were a conscious decision. The old argo-vm tinderbox has an old compiler that doesn't do visibility properly. That's one of the reasons we switched to fx-linux-tbox where --enable-libxul was a performance improvement across the board.

I'll be working with stan on the mac issues: there is some issue with mach-o "external" symbols.

Comment 3

[Benjamin Smedberg]

Attachment: Disable libxul for mac, rev. 1

Comment 4

[(not currently active) Ted Mielczarek]

Comment on attachment 268239 [details] [diff] [review]
Disable libxul for mac, rev. 1

Yuck, shell script!

Comment 5

[Adam Guthrie]

Just to clarify bsmedberg's comment 2: bl-blxlnx01 is a tinderbox machine that downloads builds and runs tests on them. Benjamin set up argo-vm and fx-linux-tbox with identical (static) configurations and ran perf. tests on them using bl-bldlnx01. For some reason, fx-linux-tbox was much slower than argo-vm in this case, BUT when fx-linux-tbox switched over to libxul, it's performance numbers went down. The reason why fx-linux-tbox was slower is unknown.

Comment 6

[Benjamin Smedberg]

mac-specific backout landed on trunk

Comment 7

[Phil Ringnalda (:philor)]

Post-backout:

Tdhtml: ~425ms -> ~466ms
Txul: ~150ms -> ~170ms
Ts: ~900ms -> ~870ms

Comment 8

[Phil Ringnalda (:philor)]

And then, after two cycles with worse numbers than the ones *with* libxul, they dropped back to what they should be on the first new-textframe cycle, and stayed there after its backout. I guess making sense is too much to ask of them.

Comment 9

[Robert Sayre]

Looks like this regressed Tp2 from ~115ms -> ~155ms

Comment 10

[Robert Sayre]

Adding dependency on Bug 384513.

Comment 11

[Benjamin Smedberg]

ok, this landed 2007-06-19 08:02

graphs:
Tp:
http://build-graphs.mozilla.org/graph/query.cgi?tbox=xserve08.build.mozilla.org_fx-trunk&testname=pageload&autoscale=1&size=&units=ms&ltype=&points=&showpoint=2007%3A06%3A19%3A17%3A37%3A14%2C161&avg=1&days=2
Tp2:
http://build-graphs.mozilla.org/graph/query.cgi?tbox=xserve08.build.mozilla.org_fx-trunk&testname=pageload2&autoscale=1&size=&units=ms&ltype=&points=&showpoint=&avg=1&days=2
dhtml:
http://build-graphs.mozilla.org/graph/query.cgi?tbox=xserve08.build.mozilla.org_fx-trunk&testname=dhtml&autoscale=1&size=&units=ms&ltype=&points=&showpoint=2007%3A06%3A19%3A17%3A43%3A29%2C517&avg=1&days=2
Ts:
http://build-graphs.mozilla.org/graph/query.cgi?tbox=xserve08.build.mozilla.org_fx-trunk&testname=startup&autoscale=1&size=&units=ms&ltype=&points=&showpoint=2007%3A06%3A19%3A17%3A49%3A46%2C896&avg=1&days=2

Tp bumped up a bit at 02:00
Tp2 jumped a lot at 08:00
dhtml jumped a lot at 02:00
Ts went down and up seemingly randomly

Is there a way to tell whether paricular Tp2 tests went up a lot or if they all went up a little? 

Comment 12

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Not easily; you really want alice's talos tests.  If you look at the raw build logs, you'll see per-page timings in there, though; so you can manually/visually take a look at what's going on...

Comment 13

[Benjamin Smedberg]

This was fixed, oddly enough, by the single_module change in bug 386445. Go figure.