Closed Bug 1079952 Opened 10 years ago Closed 8 years ago

Default browser on Android is 230% faster in sunspider

Categories

(Core :: JavaScript Engine: JIT, defect)

Product:
Core
Component:
JavaScript Engine: JIT
Platform:
ARM
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WONTFIX

People

(Reporter: speciesx, Unassigned)

References

Details

Tested on Galaxy S4 with Android 4.4.2 Firefox nightly https://www.webkit.org/perf/sunspider-1.0.2/sunspider-1.0.2/results.html?%7B%22v%22:%20%22sunspider-1.0.2%22,%20%223d-cube%22:%5B117,104,115,86,144,86,96,88,92,104%5D,%223d-morph%22:%5B41,40,29,27,29,36,30,31,28,41%5D,%223d-raytrace%22:%5B104,146,99,108,98,121,98,116,86,106%5D,%22access-binary-trees%22:%5B21,16,23,16,16,17,15,18,14,16%5D,%22access-fannkuch%22:%5B46,42,33,37,33,37,32,33,26,42%5D,%22access-nbody%22:%5B20,18,23,17,18,19,17,16,16,40%5D,%22access-nsieve%22:%5B16,13,14,12,11,13,11,11,12,16%5D,%22bitops-3bit-bits-in-byte%22:%5B6,9,6,7,6,6,5,7,5,7%5D,%22bitops-bits-in-byte%22:%5B9,10,9,8,8,12,8,17,8,10%5D,%22bitops-bitwise-and%22:%5B10,6,7,6,6,10,11,8,6,10%5D,%22bitops-nsieve-bits%22:%5B73,21,21,19,20,24,33,21,19,22%5D,%22controlflow-recursive%22:%5B16,12,11,10,11,16,14,10,13,19%5D,%22crypto-aes%22:%5B83,71,52,77,71,76,53,43,76,73%5D,%22crypto-md5%22:%5B35,32,28,29,30,28,46,34,30,32%5D,%22crypto-sha1%22:%5B24,21,21,27,23,21,36,22,20,20%5D,%22date-format-tofte%22:%5B114,73,87,77,91,92,106,71,72,97%5D,%22date-format-xparb%22:%5B104,88,143,103,112,97,104,124,122,133%5D,%22math-cordic%22:%5B20,14,30,27,25,24,16,18,18,26%5D,%22math-partial-sums%22:%5B39,35,31,40,39,36,47,32,36,53%5D,%22math-spectral-norm%22:%5B13,10,19,11,18,10,15,10,17,10%5D,%22regexp-dna%22:%5B44,24,34,26,45,26,30,24,28,28%5D,%22string-base64%22:%5B41,35,54,31,50,50,54,30,47,32%5D,%22string-fasta%22:%5B67,39,41,33,42,62,44,34,40,35%5D,%22string-tagcloud%22:%5B150,192,154,119,137,124,127,118,154,147%5D,%22string-unpack-code%22:%5B234,195,215,160,211,181,144,113,129,161%5D,%22string-validate-input%22:%5B85,49,73,63,53,49,45,37,67,43%5D%7D Default Browser https://www.webkit.org/perf/sunspider-1.0.2/sunspider-1.0.2/results.html?{%22v%22:%20%22sunspider-1.0.2%22,%20%223d-cube%22:%5B20,20,20,21,21,21,21,22,21,23%5D,%223d-morph%22:%5B23,21,21,25,21,21,21,21,25,23%5D,%223d-raytrace%22:%5B30,32,30,32,32,32,31,31,32,32%5D,%22access-binary-trees%22:%5B5,5,4,5,5,5,5,5,5,4%5D,%22access-fannkuch%22:%5B11,11,13,12,12,13,11,10,10,11%5D,%22access-nbody%22:%5B7,8,8,7,7,7,7,8,7,8%5D,%22access-nsieve%22:%5B8,7,8,7,7,8,9,9,7,8%5D,%22bitops-3bit-bits-in-byte%22:%5B2,1,1,2,2,2,1,2,2,1%5D,%22bitops-bits-in-byte%22:%5B7,5,8,6,7,7,8,6,5,7%5D,%22bitops-bitwise-and%22:%5B11,9,10,10,10,10,10,10,11,10%5D,%22bitops-nsieve-bits%22:%5B17,16,17,15,16,16,15,16,15,16%5D,%22controlflow-recursive%22:%5B5,10,5,5,8,8,5,5,6,5%5D,%22crypto-aes%22:%5B18,16,14,16,15,15,15,18,16,18%5D,%22crypto-md5%22:%5B8,8,8,9,10,9,8,8,8,8%5D,%22crypto-sha1%22:%5B17,16,17,17,17,17,18,16,19,19%5D,%22date-format-tofte%22:%5B51,51,54,52,54,53,53,58,73,51%5D,%22date-format-xparb%22:%5B49,52,50,48,49,50,49,49,50,50%5D,%22math-cordic%22:%5B14,12,12,12,12,14,12,12,12,12%5D,%22math-partial-sums%22:%5B21,18,17,18,18,17,18,19,18,18%5D,%22math-spectral-norm%22:%5B7,5,4,6,4,6,3,6,7,4%5D,%22regexp-dna%22:%5B20,20,19,20,19,20,20,21,20,20%5D,%22string-base64%22:%5B14,10,10,10,9,13,9,9,10,11%5D,%22string-fasta%22:%5B45,47,47,48,49,45,46,47,49,47%5D,%22string-tagcloud%22:%5B48,43,43,43,43,43,46,43,44,44%5D,%22string-unpack-code%22:%5B78,80,82,81,83,83,83,80,84,79%5D,%22string-validate-input%22:%5B18,20,19,17,18,19,18,18,17,19%5D} TEST COMPARISON FROM TO DETAILS =============================================================================== ** TOTAL **: 2.34x as fast 1284.2ms +/- 6.9% 549.6ms +/- 1.2% significant =============================================================================== 3d: 3.28x as fast 244.6ms +/- 7.3% 74.6ms +/- 2.4% significant cube: 4.91x as fast 103.2ms +/- 12.6% 21.0ms +/- 3.2% significant morph: 1.50x as fast 33.2ms +/- 12.3% 22.2ms +/- 5.4% significant raytrace: 3.45x as fast 108.2ms +/- 10.9% 31.4ms +/- 1.9% significant access: 2.76x as fast 86.6ms +/- 11.4% 31.4ms +/- 2.7% significant binary-trees: 3.58x as fast 17.2ms +/- 11.6% 4.8ms +/- 6.3% significant fannkuch: 3.17x as fast 36.1ms +/- 11.7% 11.4ms +/- 6.7% significant nbody: 2.76x as fast 20.4ms +/- 25.2% 7.4ms +/- 5.0% significant nsieve: 1.65x as fast 12.9ms +/- 10.6% 7.8ms +/- 7.2% significant bitops: 1.51x as fast 51.6ms +/- 24.2% 34.2ms +/- 3.5% significant 3bit-bits-in-byte: 4.00x as fast 6.4ms +/- 13.1% 1.6ms +/- 23.1% significant bits-in-byte: 1.50x as fast 9.9ms +/- 20.3% 6.6ms +/- 11.6% significant bitwise-and: *1.26x as slow* 8.0ms +/- 18.4% 10.1ms +/- 4.0% significant nsieve-bits: 1.72x as fast 27.3ms +/- 43.4% 15.9ms +/- 3.3% significant controlflow: 2.13x as fast 13.2ms +/- 16.3% 6.2ms +/- 20.9% significant recursive: 2.13x as fast 13.2ms +/- 16.3% 6.2ms +/- 20.9% significant crypto: 2.95x as fast 123.4ms +/- 7.9% 41.8ms +/- 2.9% significant aes: 4.19x as fast 67.5ms +/- 14.0% 16.1ms +/- 6.4% significant md5: 3.86x as fast 32.4ms +/- 11.8% 8.4ms +/- 5.9% significant sha1: 1.36x as fast 23.5ms +/- 14.8% 17.3ms +/- 4.4% significant date: 1.92x as fast 201.0ms +/- 7.8% 104.6ms +/- 4.6% significant format-tofte: 1.60x as fast 88.0ms +/- 12.1% 55.0ms +/- 8.7% significant format-xparb: 2.28x as fast 113.0ms +/- 10.8% 49.6ms +/- 1.5% significant math: 2.06x as fast 73.9ms +/- 9.2% 35.8ms +/- 5.4% significant cordic: 1.76x as fast 21.8ms +/- 17.4% 12.4ms +/- 4.9% significant partial-sums: 2.13x as fast 38.8ms +/- 12.4% 18.2ms +/- 4.5% significant spectral-norm: 2.56x as fast 13.3ms +/- 19.6% 5.2ms +/- 19.2% significant regexp: 1.55x as fast 30.9ms +/- 17.9% 19.9ms +/- 2.0% significant dna: 1.55x as fast 30.9ms +/- 17.9% 19.9ms +/- 2.0% significant string: 2.28x as fast 459.0ms +/- 11.2% 201.1ms +/- 0.8% significant base64: 4.04x as fast 42.4ms +/- 16.4% 10.5ms +/- 11.7% significant fasta: ?? 43.7ms +/- 18.9% 47.0ms +/- 2.2% not conclusive: might be *1.076x as slow* tagcloud: 3.23x as fast 142.2ms +/- 11.3% 44.0ms +/- 2.8% significant unpack-code: 2.14x as fast 174.3ms +/- 16.3% 81.3ms +/- 1.8% significant validate-input: 3.08x as fast 56.4ms +/- 19.2% 18.3ms +/- 3.7% significant
Component: JavaScript Engine → JavaScript Engine: JIT
Hannes what is the V8 browser we're testing on the Android 4.3 Flame device on AWFY? Chrome? Any idea why this doesn't match the results in comment 0?
Flags: needinfo?(hv1989)
All tested on the flame running android 4.3 - SS: > Default browser: 1249.5ms > Chrome Nightly: ChromeShell: 1814.9ms > ContentShell: 1872.1ms > Nightly: 1622.3ms > Chrome Beta: 1742.8ms > * lower is better I use ChromeShell to measure performance on AWFY. Which seems to correspond to actual chrome performance. If we look to beta, it is a bit faster, but this corresponds to chrome degrading performance of ss every release. Here we also see that the Default browser is much faster on SS. Note that SS doesn't run long enough for the JIT's to help a lot. It is always a balancing to not regress SS with new performance optimizations. So if the Default browser doesn't do fancy tricks as jit compiler etc. It might be faster on SS, but it should be way slower on Octane ... Which is also the case: > Richards Deltablue Crypto Raytrace > Default browser: 1186 1455 1993 1284 > ChromeShell: 2485 3699 2436 3739 > Nightly: 3237 3779 2438 4076 > * higher is better Note: I only used the first few benchmarks. Since I got the unresponsive dialog on the "Default browser". So the numbers during that popup will definitely be slower. So I only used the first 4 scripts before the unresponsive dialog. Sunspider isn't really a good benchmark to measure performance. One of the reasons we have these newer ones, like kraken and octane. Chrome and Nightly are both starting to optimize code, which takes time. As a result we are slower on SS. The Default browser doesn't optimize so takes less time, because it isn't optimizing its code.
Flags: needinfo?(hv1989)
Default browser is also faster in Octane and Kraken (but only 3-4% faster). Default browser vs Nightly Score 3098 2861 Richards 4981 6591 Deltablue 2583 2061 Crypto 5055 5049 Raytrace 3619 5715 EarleyBoyer 4611 4890 Regsp 604 376 Splay 2027 864 SplayLatency 6297 890 NavierStokes 8310 9863 Pdf.js 2034 1629 Mandreel 2708 2601 MandreelLatency 2140 2801 GB Emulator 3986 6755 CodeLoad 2740 2572 Box2DWeb 1925 2914 zlib 4079 9231 Typescript 3178 1902
(In reply to SpeciesX from comment #3) > Default browser is also faster in Octane and Kraken (but only 3-4% faster). > > Default browser vs Nightly > Score 3098 2861 > > Richards 4981 6591 > Deltablue 2583 2061 > Crypto 5055 5049 > Raytrace 3619 5715 > EarleyBoyer 4611 4890 > Regsp 604 376 > Splay 2027 864 > SplayLatency 6297 890 > NavierStokes 8310 9863 > Pdf.js 2034 1629 > Mandreel 2708 2601 > MandreelLatency 2140 2801 > GB Emulator 3986 6755 > CodeLoad 2740 2572 > Box2DWeb 1925 2914 > zlib 4079 9231 > Typescript 3178 1902 That would make me assume this browser is actually chrome (and not stock browser). Which is possible, since on more and more phones the stack browser is replaced by chrome. (Why is the SS score that much better :0) Can you find out which browser is the stock browser on your device?
(In reply to Hannes Verschore [:h4writer] from comment #4) > That would make me assume this browser is actually chrome (and not stock > browser). Which is possible, since on more and more phones the stack browser > is replaced by chrome. (Why is the SS score that much better :0) > > Can you find out which browser is the stock browser on your device? It's not Chrome, the name of the Browser App is "Internet" Version 1.5.28
That seems to be "com.sec.android.app.sbrowser". Not sure what it is based on yet. And I'll try to get that on the flame to see. And also try to find out what the roots of this "stock browser" is and how it comes it is so good at SS and octane.
Flags: needinfo?(hv1989)
Chromium/Blink has become the engine running the WebView in Android 4.4, which is what the "Internet" browser uses. So it's most likely running Chromium 30, since that was the version it used for 4.4.2.
yea, it is based on chrome. Mozilla/5.0 (Linux; Android 4.4.2; de-de; SAMSUNG GT-I9505 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/1.5 Chrome/28.0.1500.94 Mobile Safari/537.36 { "ua": "Mozilla/5.0 (Linux; Android 4.4.2; de-de; SAMSUNG GT-I9505 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/1.5 Chrome/28.0.1500.94 Mobile Safari/537.36", "browser": { "name": "Chrome", "version": "28.0.1500.94", "major": "28" }, "engine": { "name": "WebKit", "version": "537.36" }, "os": { "name": "Android", "version": "4.4.2" }, "device": { "model": "GT-I9505", "vendor": "Samsung", "type": "mobile" }, "cpu": {} }
Oh that means chrome 28, released on 2013-07-10. Looking at AWFY that is a sweetspot for chrome on SS. Afterwards they started degrading the SS score a lot. Btw. can you run octane a few times. I think you might see a big variation. (Due to octane-splay). Which would make nightly atleast sometimes be faster than chrome on that benchmark. (This issue is being tracked).
Flags: needinfo?(hv1989)
(In reply to Hannes Verschore [:h4writer] from comment #9) > Btw. can you run octane a few times. I think you might see a big variation. > (Due to octane-splay). Which would make nightly atleast sometimes be faster > than chrome on that benchmark. (This issue is being tracked). After 10 runs i get in average 2700-2900 and only once 3300.
Depends on: 1078437
We (V8 and SM) have been regressing (not on purpose) sunspider every release when improve performance on longer running benchmarks. The stock browser on android was just a very old "V8" version, just before v8 started regressing SS. That is the reason of the performance difference. Closing since this is a know fact and I don't think the stock browser is still used on android?
Status: UNCONFIRMED → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.