Intermittent stylebench | invalid literal for int() with base 10: '<random text>'

RESOLVED FIXED in Firefox 63

Status

defect
P5
normal
RESOLVED FIXED
11 months ago
11 months ago

People

(Reporter: intermittent-bug-filer, Assigned: emilio)

Tracking

({intermittent-failure})

Version 3
mozilla63
Points:
---

Firefox Tracking Flags

(firefox63 fixed)

Details

Attachments

(1 attachment)

Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=192914099&repo=autoland

https://queue.taskcluster.net/v1/task/N61Yxu7RREuoh1W5VLsiMQ/runs/0/artifacts/public/logs/live_backing.log

20:21:32     INFO -  PID 19548 | |337;Before and after pseudo elements/Removing classes - 2/Sync;2.660000000000764;1.800000000000182;2.039999999999054;1.819999999999709;1.680000000000291;1.820000000003347;2.0200000000040745;1.5999999999985448;1.6999999999970896;1.8999999999996362;1.5799999999999272;2.459999999999127;2.280000000002474;1.8400000000001455;2.1599999999998545;1.6999999999970896;2.599999999998545;1.9599999999991269;2.1599999999998545;1.9600000000000364;2.0800000000017462;2.680000000000291;1.9399999999986903;2.0200000000004366;2.639999999999418;1.7200000000011642;1.5199999999967986;1.6599999999998545;1.639999999999418;2.5;1.6199999999989814;1.7399999999979627;1.639999999999418;1.8000000000029104;1.6600000000034925;2.139999999999418;1.7199999999993452;1.9399999999995998;2.319999999999709;1.4400000000005093;2.040000000000873;2.099999999998545;1.639999999999418;1.5999999999985448;2.0599999999976717
20:21:32     INFO -  PID 19548 | |338;Before and after pseudo elements/Removing classes - 2/Async;39.42000000000007;32.97999999999956;22.68000000000029;40.600000000000364;42.599999999998545;22.959999999999127;45.04000000000087;52.080000000001746;45.45999999999913;28.31999999999971;42.9399999999996;33.460000000000946;38.7599999999984;26.200000000000728;46.919999999998254;33.36000000000058;23.840000000003783;24.779999999998836;36.159999999999854;52.73999999999978;36.63999999999942;46.219999999999345;40.780000000002474;40.95999999999913;47.0199999999968;33.81999999999971;36.37999999999738;40.26000000000022;39.400000000000546;43.659999999999854;35.06000000000131;54.20000000000073;52.68000000000029;37.95999999999913;32.279999999998836;33.61999999999534;28.24000000000069;44.220000000000255;38.399999999999636;30.920000000000073;40.39999999999782;35.54000000000087;33.299999999995634;40.18000000000029;43.400000000001455
20:21:32     INFO -  PID 19548 | |339;Before and after pseudo elements/Removing classes - 2;42.08000000000084;34.779999999999745;24.719999999999345;42.42000000000007;44.279999999998836;24.780000000002474;47.06000000000495;53.68000000000029;47.15999999999622;30.219999999999345;44.51999999999953;35.92000000000007;41.04000000000087;28.040000000000873;49.07999999999811;35.05999999999767;26.44000000000233;26.739999999997963;38.31999999999971;54.69999999999982;38.720000000001164;48.899999999999636;42.720000000001164;42.97999999999956;49.65999999999622;35.54000000000087;37.89999999999418;41.92000000000007;41.039999999999964;46.159999999999854;36.68000000000029;55.93999999999869;54.31999999999971;39.76000000000204;33.94000000000233;35.75999999999476;29.960000000000036;46.159999999999854;40.719999999999345;32.36000000000058;42.43999999999869;37.63999999999942;34.93999999999505;41.779999999998836;45.45999999999913
20:21:32     INFO -  PID 19548 | |340;Before and after pseudo elements/Mutating attributes - 2/Sync;1.480000000000473;1.180000000000291;1.2800000000006548;1.2399999999997817;1.6599999999998545;1.5200000000004366;1.2600000000020373;1.2599999999947613;1.7399999999979627;1.360000000000582;1.3599999999996726;2.959999999999127;1.679999999996653;1.680000000000291;0.9200000000018917;1.5400000000008731;1.8000000000029104;1.5600000000049477;1.5799999999999272;1.4600000000000364;2.319999999999709;1.139999999999418;2.899999999997817;1.319999999999709;1.4599999999991269;1.9400000000023283;1.5400000000008731;1.680000000000291;1.6400000000003274;1.3799999999991996;1.2800000000006548;1.4199999999982538;1.2199999999975262;0.8400000000037835;0.8399999999965075;1.7200000000011642;1.3999999999996362;1.9600000000000364;1.7600000000002183;1.639999999999418;1.6599999999998545;1.5;1.9000000000014552;1.3999999999941792;2.099999999998545
20:21:32    ERROR -  PID 19548 | |341JavaScript error: http://localhost:35956/tests/webkit/PerformanceTests/StyleBench/resources/benchmark-report.js, line 49: TypeError: Argument 1 of Node.removeChild is not an object.
20:21:32     INFO -  PID 19548 | ;Before and after pseudo elements/Mutating attributes - 2/Async;20.18000000000029;30.300000000000182;19.920000000000073;30.68000000000029;10.219999999997526;26.68000000000029;28.260000000002037;15;32.13999999999942;9.659999999999854;14.44000000000051;21.280000000002474;28.18000000000029;45.279999999998836;10.479999999999563;14.12000000000262;20.799999999995634;11.659999999996217;16.859999999999673;9.800000000000182;50.42000000000007;19.159999999999854;42.960000000002765;18.020000000000437;10.159999999996217;23.68000000000029;29.81999999999971;14.859999999999673;20.44000000000051;11.06000000000131;17.19999999999891;9.619999999998981;14.520000000000437;10.239999999997963;17.360000000000582;21.839999999996508;42.88000000000011;38.86000000000058;10.039999999999054;32.98000000000138;36.68000000000029;27.93999999999869;19.81999999999971;32.78000000000611;14.940000000002328
20:21:32     INFO -  PID 19548 | |342;Before and after pseudo elements/Mutating attributes - 2;21.660000000000764;31.480000000000473;21.200000000000728;31.920000000000073;11.87999999999738;28.200000000000728;29.520000000004075;16.25999999999476;33.87999999999738;11.020000000000437;15.800000000000182;24.2400000000016;29.859999999996944;46.95999999999913;11.400000000001455;15.660000000003492;22.599999999998545;13.220000000001164;18.4399999999996;11.260000000000218;52.73999999999978;20.299999999999272;45.86000000000058;19.340000000000146;11.619999999995343;25.62000000000262;31.360000000000582;16.539999999999964;22.080000000000837;12.44000000000051;18.479999999999563;11.039999999997235;15.739999999997963;11.080000000001746;18.19999999999709;23.55999999999767;44.279999999999745;40.82000000000062;11.799999999999272;34.6200000000008;38.340000000000146;29.43999999999869;21.720000000001164;34.18000000000029;17.040000000000873

20:21:32     INFO -  PID 19548 | Values: 29.7  191.6  45.3  28.3  10.7  26.4  12.6  15.5  32.4  40.3  183.0  42.7  18.2  27.7  11.8  37.8  20.4  40.6  27.5  179.3  10.0  9.8  30.0  182.5  45.3  35.8  14.0  17.0  180.2  40.3  86.3  30.3  98.2  32.9  10.3  9.8  11.1  21.5  43.8  179.1  37.5  31.8  39.9  11.9  9.6
20:21:32     INFO -  PID 19548 |
20:21:32     INFO -  PID 19548 | [#378] Before and after pseudo elements/Removing leaf elements - 4  Cycles:45  Average:51.59  Median:28.30  stddev:55.65 (196.7%)  stddev-sans-first:56.21
20:21:32     INFO -  PID 19548 | Values: 31.2  193.6  47.1  30.1  12.2  28.3  14.4  17.4  34.0  41.9  184.6  44.5  20.6  29.6  13.7  39.7  22.0  42.7  29.4  181.2  11.4  11.6  31.7  184.4  46.9  37.6  15.6  18.4  181.9  42.1  88.7  32.0  99.8  34.7  12.0  11.8  12.5  23.9  45.3  181.0  39.9  33.6  41.7  13.4  11.5
20:21:32     INFO -  PID 19548 |
20:21:32     INFO -  PID 19548 | [#379] Before and after pseudo elements  Cycles:45  Average:810.76  Median:666.48  stddev:185.20 (27.8%)  stddev-sans-first:186.73
20:21:32     INFO -  PID 19548 | Values: 712.8  888.5  636.2  831.3  642.3  950.5  1074.0  1142.8  634.0  617.5  973.9  985.9  666.5  716.8  649.5  1284.9  651.4  726.1  1026.8  785.9  632.0  697.0  605.6  911.5  702.8  812.9  954.5  1170.8  775.0  588.2  841.3  1020.7  770.5  596.1  760.8  1128.4  1019.1  625.2  602.0  939.3  984.6  625.1  777.6  696.3  649.2
20:21:32     INFO -  PID 19548 | -------- Summary: end --------
20:21:32     INFO -  PID 19548 |
20:21:33     INFO -  TEST-INFO | 19548: exit 0
20:21:33     INFO -  TEST-UNEXPECTED-ERROR | stylebench | invalid literal for int() with base 10: 'Before and after pseudo elements/Mutating attributes - 2/Async'
20:21:33    ERROR -  Traceback (most recent call last):
20:21:33     INFO -    File "/home/cltbld/workspace/build/tests/talos/talos/run_tests.py", line 297, in run_tests
20:21:33     INFO -      talos_results.add(mytest.runTest(browser_config, test))
20:21:33     INFO -    File "/home/cltbld/workspace/build/tests/talos/talos/ttest.py", line 63, in runTest
20:21:33     INFO -      return self._runTest(browser_config, test_config, setup)
20:21:33     INFO -    File "/home/cltbld/workspace/build/tests/talos/talos/ttest.py", line 270, in _runTest
20:21:33     INFO -      else None)
20:21:33     INFO -    File "/home/cltbld/workspace/build/tests/talos/talos/results.py", line 97, in add
20:21:33     INFO -      results = browserLog.results()
20:21:33     INFO -    File "/home/cltbld/workspace/build/tests/talos/talos/results.py", line 403, in results
20:21:33     INFO -      format_pagename=self.format_pagename)
20:21:33     INFO -    File "/home/cltbld/workspace/build/tests/talos/talos/results.py", line 246, in __init__
20:21:33     INFO -      result['index'] = int(r[0])
20:21:33     INFO -  ValueError: invalid literal for int() with base 10: 'Before and after pseudo elements/Mutating attributes - 2/Async'
20:21:33     INFO -  TEST-INFO took 227216ms
20:21:33     INFO -  SUITE-END | took 417s
20:21:33    ERROR - Return code: 2
20:21:33  WARNING - setting return code to 2
20:21:33    ERROR - # TBPL FAILURE #
20:21:33     INFO - Running post-action listener: _package_coverage_data
Emilio, can you please take a look at this?
Flags: needinfo?(emilio)
Summary: Intermittent stylebench | invalid literal for int() with base 10: 'Before and after pseudo elements/Mutating attributes - 2/Async' → Intermittent stylebench | invalid literal for int() with base 10: '<random text>'
Yes, I'm looking. Looks like some measurements are not present:

| ;Before and after pseudo elements/Mutating attributes - 2/Async;20.18000000000029;30.300000000000182;19.920000000000073;30.68000000000029;10.219999999997526;26.68000000000029;28.260000000002037;15;32.13999999999942;9.659999999999854;14.44000000000051;21.280000000002474;28.18000000000029;45.279999999998836;10.479999999999563;14.12000000000262;20.799999999995634;11.659999999996217;16.859999999999673;9.800000000000182;50.42000000000007;19.159999999999854;42.960000000002765;18.020000000000437;10.159999999996217;23.68000000000029;29.81999999999971;14.859999999999673;20.44000000000051;11.06000000000131;17.19999999999891;9.619999999998981;14.520000000000437;10.239999999997963;17.360000000000582;21.839999999996508;42.88000000000011;38.86000000000058;10.039999999999054;32.98000000000138;36.68000000000029;27.93999999999869;19.81999999999971;32.78000000000611;14.940000000002328

(There should be a number before the `;`)

I'll try to figure out.
Sorry for the lag here, got carried away.
Assignee: nobody → emilio
Flags: needinfo?(emilio)
benchmar-report.js already has code like:

    window.onload = function () {
        document.body.removeChild(document.querySelector('main'));
        startBenchmark();
    }

So we avoid starting the test on load twice.

The throw when the style element was not there was saving us from reporting
twice, but the message interfered with the output that talos tries to parse.

At first sight, looks like this could also be a problem for speedometer, but
turns out it's not, since speedometer uses `<body onload="">`, and window.onload
overrides it.

That means that
https://hg.mozilla.org/mozilla-central/rev/162593bda5e167bea16b28637e85cc9d23f19c7c
is useless for Talos / Raptor, but it may be needed for PGO stuff, so I've kept
it for now.
Comment on attachment 8999001 [details]
Avoid starting the benchmark twice with the same client. r=jmaher

Joel Maher ( :jmaher ) (UTC+2) has approved the revision.
Attachment #8999001 - Flags: review+
Pushed by emilio@crisal.io:
https://hg.mozilla.org/integration/autoland/rev/2bbf606a0b68
Avoid starting the benchmark twice with the same client. r=jmaher
https://hg.mozilla.org/mozilla-central/rev/2bbf606a0b68
Status: NEW → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla63
Push from comment 9 updated our baselines:

== Change summary for alert #14868 (as of Fri, 10 Aug 2018 09:48:08 GMT) ==

Improvements:

 85%  stylebench osx-10-10 opt e10s stylo     9.68 -> 17.86
 77%  stylebench windows10-64 opt e10s stylo  23.37 -> 41.30
 76%  stylebench windows7-32 pgo e10s stylo   24.90 -> 43.86
 76%  stylebench windows10-64 pgo e10s stylo  25.34 -> 44.48
 74%  stylebench windows7-32 opt e10s stylo   24.06 -> 41.94
 73%  stylebench linux64 opt e10s stylo       22.43 -> 38.82
 70%  stylebench linux64 pgo e10s stylo       24.19 -> 41.17
 70%  stylebench windows10-64-qr opt e10s stylo25.36 -> 43.05
 64%  stylebench linux64-qr opt e10s stylo    23.71 -> 38.85

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=14868
Duplicate of this bug: 1471747
You need to log in before you can comment on or make changes to this bug.