6% Sunspider Regression between 1-29 and 1-30 builds

RESOLVED INVALID

Status

()

P2
normal
RESOLVED INVALID
11 years ago
11 years ago

People

(Reporter: mtschrep, Assigned: crowderbt)

Tracking

({perf, regression})

Trunk
perf, regression
Points:
---
Bug Flags:
blocking1.9 +

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

11 years ago
1-29 Nightly:

http://people.mozilla.com/~schrep/SunSpider/hosted/sunspider-results.html?%7B%223d-cube%22:%5B450,443,442,441,442%5D,%223d-morph%22:%5B594,605,603,599,599%5D,%223d-raytrace%22:%5B286,294,285,282,287%5D,%22access-binary-trees%22:%5B151,127,124,138,138%5D,%22access-fannkuch%22:%5B383,382,383,384,385%5D,%22access-nbody%22:%5B460,443,455,437,452%5D,%22access-nsieve%22:%5B164,161,164,163,162%5D,%22bitops-3bit-bits-in-byte%22:%5B144,144,144,145,145%5D,%22bitops-bits-in-byte%22:%5B154,153,155,152,152%5D,%22bitops-bitwise-and%22:%5B657,661,650,647,648%5D,%22bitops-nsieve-bits%22:%5B211,211,209,209,215%5D,%22controlflow-recursive%22:%5B80,80,80,79,82%5D,%22crypto-aes%22:%5B221,237,224,228,225%5D,%22crypto-md5%22:%5B131,124,122,123,139%5D,%22crypto-sha1%22:%5B138,144,143,144,143%5D,%22date-format-tofte%22:%5B385,385,381,385,381%5D,%22date-format-xparb%22:%5B214,231,217,212,215%5D,%22math-cordic%22:%5B378,353,363,372,351%5D,%22math-partial-sums%22:%5B430,409,411,419,435%5D,%22math-spectral-norm%22:%5B167,175,164,163,166%5D,%22regexp-dna%22:%5B341,342,337,336,334%5D,%22string-base64%22:%5B414,396,393,401,396%5D,%22string-fasta%22:%5B335,349,334,347,349%5D,%22string-tagcloud%22:%5B277,253,274,257,257%5D,%22string-unpack-code%22:%5B447,418,420,451,446%5D,%22string-validate-input%22:%5B174,172,176,179,200%5D%7D

1-30 Nightly

http://people.mozilla.com/~schrep/SunSpider/hosted/sunspider-results.html?%7B%223d-cube%22:%5B442,431,428,428,440%5D,%223d-morph%22:%5B694,690,682,688,686%5D,%223d-raytrace%22:%5B299,293,300,308,298%5D,%22access-binary-trees%22:%5B140,139,136,135,136%5D,%22access-fannkuch%22:%5B383,382,383,382,383%5D,%22access-nbody%22:%5B550,543,544,564,540%5D,%22access-nsieve%22:%5B165,162,163,162,163%5D,%22bitops-3bit-bits-in-byte%22:%5B204,206,201,204,208%5D,%22bitops-bits-in-byte%22:%5B190,189,188,191,190%5D,%22bitops-bitwise-and%22:%5B715,690,678,678,663%5D,%22bitops-nsieve-bits%22:%5B207,209,210,224,209%5D,%22controlflow-recursive%22:%5B82,83,81,83,84%5D,%22crypto-aes%22:%5B238,226,227,228,228%5D,%22crypto-md5%22:%5B125,133,130,130,125%5D,%22crypto-sha1%22:%5B133,135,132,137,136%5D,%22date-format-tofte%22:%5B400,397,392,395,392%5D,%22date-format-xparb%22:%5B216,223,237,222,221%5D,%22math-cordic%22:%5B399,389,373,391,389%5D,%22math-partial-sums%22:%5B404,410,406,407,419%5D,%22math-spectral-norm%22:%5B174,181,175,174,173%5D,%22regexp-dna%22:%5B354,356,353,355,353%5D,%22string-base64%22:%5B393,397,394,394,396%5D,%22string-fasta%22:%5B347,340,345,346,342%5D,%22string-tagcloud%22:%5B294,292,289,288,290%5D,%22string-unpack-code%22:%5B503,505,502,505,506%5D,%22string-validate-input%22:%5B176,186,202,186,183%5D%7D

Updated

11 years ago
Keywords: perf, regression

Comment 1

11 years ago
Did you see this on Windows?
(Reporter)

Comment 2

11 years ago
Yes - closer to 3% there:

1-29: 

http://people.mozilla.com/~schrep/SunSpider/hosted/sunspider-results.html?{%223d-cube%22:[497,537,508,517,504],%223d-morph%22:[686,684,701,697,681],%223d-raytrace%22:[341,336,365,360,331],%22access-binary-trees%22:[151,137,163,138,152],%22access-fannkuch%22:[552,556,569,554,555],%22access-nbody%22:[530,522,536,515,515],%22access-nsieve%22:[221,216,230,215,214],%22bitops-3bit-bits-in-byte%22:[201,200,201,203,199],%22bitops-bits-in-byte%22:[259,247,256,255,248],%22bitops-bitwise-and%22:[872,828,788,838,849],%22bitops-nsieve-bits%22:[323,298,314,327,298],%22controlflow-recursive%22:[99,102,97,100,100],%22crypto-aes%22:[266,267,269,273,270],%22crypto-md5%22:[161,162,161,166,162],%22crypto-sha1%22:[187,184,185,184,184],%22date-format-tofte%22:[404,410,401,408,407],%22date-format-xparb%22:[219,221,224,238,221],%22math-cordic%22:[531,540,501,525,533],%22math-partial-sums%22:[488,475,495,472,481],%22math-spectral-norm%22:[213,222,216,212,213],%22regexp-dna%22:[623,712,637,621,615],%22string-base64%22:[431,467,444,448,442],%22string-fasta%22:[400,398,414,381,386],%22string-tagcloud%22:[293,366,403,317,330],%22string-unpack-code%22:[685,538,548,528,533],%22string-validate-input%22:[320,428,321,307,308]}

1-30: 

http://people.mozilla.com/~schrep/SunSpider/hosted/sunspider-results.html?{%223d-cube%22:[532,519,480,481,501],%223d-morph%22:[806,796,798,784,785],%223d-raytrace%22:[333,352,331,339,340],%22access-binary-trees%22:[161,133,143,145,132],%22access-fannkuch%22:[596,588,570,572,570],%22access-nbody%22:[646,663,571,621,622],%22access-nsieve%22:[222,222,226,220,227],%22bitops-3bit-bits-in-byte%22:[282,276,274,275,274],%22bitops-bits-in-byte%22:[327,310,308,309,310],%22bitops-bitwise-and%22:[806,786,795,795,809],%22bitops-nsieve-bits%22:[334,328,325,326,324],%22controlflow-recursive%22:[95,99,96,97,96],%22crypto-aes%22:[282,284,281,267,268],%22crypto-md5%22:[171,172,171,169,168],%22crypto-sha1%22:[178,179,181,178,178],%22date-format-tofte%22:[432,412,416,416,419],%22date-format-xparb%22:[216,213,214,211,215],%22math-cordic%22:[530,521,504,498,512],%22math-partial-sums%22:[460,465,480,453,473],%22math-spectral-norm%22:[213,217,199,198,200],%22regexp-dna%22:[649,626,623,627,612],%22string-base64%22:[424,419,444,430,430],%22string-fasta%22:[392,387,390,395,388],%22string-tagcloud%22:[340,341,314,316,317],%22string-unpack-code%22:[632,624,621,618,643],%22string-validate-input%22:[306,299,301,296,301]}
What's on the hook? Does the js shell show the same regression, or is this browser-only?

/be
(Reporter)

Comment 4

11 years ago
Can't detect the problem in the shell. Get same results from 1-29, 1-30 and current trunk shell runs.

Hook is friggen huge.  Including a number of possibilities from Brendan and JST among others:

http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=HEAD&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2008-01-29+04%3A00%3A00&maxdate=2008-01-30+04%3A00%3A00&cvsroot=%2Fcvsroot

Including bug 4144452, bug 404735 and bug 413097 from Brendan and 

Bug 413774, bug 414658, bug 413767 and bug 412491 from JST
(Reporter)

Comment 5

11 years ago
> Including bug 4144452, bug 404735 and bug 413097 from Brendan and 

That's bug 414452
Probably a GC scheduling thing, possibly bug 414452 or bug 413097 -- or both. Jst's should not have made things slower, though.

I'm under the weather and would appreciate it if someone tried a local back-out of bug 414452's patch to rule it out or confirm it. If it is to blame, then the fix to bug 408871 should be attached and tested (it's straightforward). If that doesn't do the trick, the further idea in bug 414452 comment 26 last paragraph should be tried.

/be
(Reporter)

Comment 8

11 years ago
(In reply to comment #7)
> Can definitely spot this on tjss and tsspider thanks to qm-mini-vista05
> reporting to the Mozilla test tree.
> 
> http://graphs-stage.mozilla.org/graph.html#spst=range&spstart=0&spend=1202155731&bpst=cursor&bpstart=0&bpend=1202155731&m1tid=141517&m1bl=0&m1avg=0
> 
> Bonsai hook for this range:
> 
> http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=HEAD&branchtype=match&dir=&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2008-01-29+19%3A59&maxdate=2008-01-29+21%3A38&cvsroot=%2Fcvsroot
> 

Based on that hook bug 414452 is not there but bug 413097 is.  Since we've got talos coverage here a simple test backout of 413097 would quickly confirm whether it is the issue.

(Assignee)

Comment 9

11 years ago
I attached the results of backing out the patch(es) from bug 413097 to that bug.  Bug 413097, comment 10

Comment 10

11 years ago
I agree that's not convincing. I guess we need to pull by date. Testing against tjss would probably be better, because the regression there is very clear cut.
(Reporter)

Updated

11 years ago
Assignee: general → crowder
Flags: blocking1.9+
Priority: -- → P2
(Assignee)

Comment 11

11 years ago
http://graphs-stage.mozilla.org/graph.html#spst=range&spstart=1201311352&spend=1202805788&bpst=Cursor&bpstart=1201311352&bpend=1202805788&m1tid=140263&m1bl=0&m1avg=0

This graph shows the regression well.  Based on this range, Bonsai has the following hook:
http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=HEAD&branchtype=match&dir=mozilla%2Fjs%2Fsrc&filetype=match&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2008-01-29+18%3A50&maxdate=2008-01-29+17%3A48&cvsroot=%2Fcvsroot

I think the most likely culprit is the ComputeGlobalThis change by mrbkap.  I think this is an unfortunate but necessary regression and we should close this bug out, unless Brendan or mrbkap have ideas about how to improve this situation further.

Enough has changed about how ComputeThis works, since this regression, that I think we'll have better luck looking elsewhere more broadly.
(Assignee)

Comment 12

11 years ago
Since no other comments have followed-up my last one, I'm closing this one INVA; reopen with more info.
Status: NEW → RESOLVED
Last Resolved: 11 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.