Closed
Bug 1665911
Opened 4 years ago
Closed 3 years ago
Node crashes on Windows 7 browsertime tests
Categories
(Testing :: Performance, defect, P2)
Testing
Performance
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: sparky, Unassigned)
References
(Blocks 1 open bug)
Details
Node is currently crashing on our Windows 7 browsertime tests. This push is testing with node 12: https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=781c3f519c966e4e1b1327538ec1f89fbf7595b2&selectedTaskRun=czDvHp0lQWedEuIzfLe8aw.0
Failure on Node 12:
[task 2020-09-17T20:13:14.750Z] 20:13:14 INFO - raptor-browsertime Info: Using Firefox Window Recorder for videos
[task 2020-09-17T20:13:14.750Z] 20:13:14 INFO - raptor-browsertime Info: timeout (s): 60000
[task 2020-09-17T20:13:14.750Z] 20:13:14 INFO - raptor-browsertime Info: browsertime cwd: C:\Users\task_1600369927\build
[task 2020-09-17T20:13:14.751Z] 20:13:14 INFO - raptor-browsertime Info: browsertime cmd: C:/Users/task_1600369927/fetches/node/node.exe C:/Users/task_1600369927/fetches/browsertime/node_modules/browsertime/bin/browsertime.js --firefox.geckodriverPath C:/Users/task_1600369927/fetches/geckodriver.exe C:\Users\task_1600369927\build\tests\raptor\raptor\browsertime\..\..\browsertime\browsertime_pageload.js --browser firefox --firefox.binaryPath C:\Users\task_1600369927\build\application\firefox\firefox.exe --browsertime.page_cycles 2 --browsertime.url https://en.wikipedia.org/wiki/Barack_Obama --browsertime.page_cycle_delay 1000 --browsertime.post_startup_delay 1000 --firefox.profileTemplate c:\users\task_1600369927\appdata\local\temp\tmpp5aobs\profile --skipHar --viewPort 1366x695 --pageLoadStrategy none --firefox.disableBrowsertimeExtension true --pageCompleteCheckStartWait 5000 --pageCompleteCheckPollTimeout 1000 --visualMetrics false --timeouts.pageLoad 60000 --timeouts.script 120000 --resultDir C:\Users\task_1600369927\build\blobber_upload_dir\browsertime-results\wikipedia --video true --firefox.windowRecorder true -n 25
[task 2020-09-17T20:13:14.751Z] 20:13:14 INFO - raptor-browsertime Info: browsertime_ffmpeg: C:/Users/task_1600369927/fetches/ffmpeg-4.1.1-win64-static/bin/ffmpeg.exe
[task 2020-09-17T20:13:14.751Z] 20:13:14 INFO - raptor-browsertime Info: PATH: C:\Users\task_1600369927\fetches\ffmpeg-4.1.1-win64-static\bin;C:\Users\task_1600369927\build\venv\Scripts;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\windows\System32\OpenSSH\;C:\Program Files\Puppet Labs\Puppet\bin;C:\Program Files\Mellanox\MLNX_VPI\IB\Tools;C:\Program Files\Mellanox\MLNX_CIMProvider\lib\mft;C:\Program Files\Git\cmd;C:\Program Files (x86)\GNU\GnuPG\pub;C:\Program Files\Mercurial\;C:\Program Files\Mercurial;C:\mozilla-build\bin;C:\mozilla-build\kdiff;C:\mozilla-build\moztools-x64\bin;C:\mozilla-build\mozmake;C:\mozilla-build\nsis-3.01;C:\mozilla-build\python;C:\mozilla-build\python\Scripts;C:\mozilla-build\python3;C:\mozilla-build\msys\bin;C:\mozilla-build\msys\local\bin;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Users\task_1600369927\AppData\Local\Microsoft\WindowsApps
[task 2020-09-17T20:13:15.454Z] 20:13:15 INFO - raptor-browsertime Info: Running tests using Firefox - 25 iteration(s)
[task 2020-09-17T20:13:17.711Z] 20:13:17 INFO - raptor-browsertime Info: C:\windows\system32\cmd.exe[1728]: c:\ws\src\env-inl.h:889: Assertion `(ret) != (nullptr)' failed.
[task 2020-09-11T23:18:02.707Z] 23:18:02 INFO - raptor-browsertime-desktop Info: binary_path: C:\Users\task_1599865692\build\application\firefox\firefox.exe
[task 2020-09-11T23:18:02.707Z] 23:18:02 INFO - raptor-browsertime Info: Using Firefox Window Recorder for videos
[task 2020-09-11T23:18:02.707Z] 23:18:02 INFO - raptor-browsertime Info: timeout (s): 60000
[task 2020-09-11T23:18:02.707Z] 23:18:02 INFO - raptor-browsertime Info: browsertime cwd: C:\Users\task_1599865692\build
[task 2020-09-11T23:18:02.708Z] 23:18:02 INFO - raptor-browsertime Info: browsertime cmd: C:/Users/task_1599865692/fetches/node/node.exe C:/Users/task_1599865692/fetches/browsertime/node_modules/browsertime/bin/browsertime.js --firefox.geckodriverPath C:/Users/task_1599865692/fetches/geckodriver.exe C:\Users\task_1599865692\build\tests\raptor\raptor\browsertime\..\..\browsertime\browsertime_pageload.js --browser firefox --firefox.binaryPath C:\Users\task_1599865692\build\application\firefox\firefox.exe --browsertime.page_cycles 25 --browsertime.url https://en.wikipedia.org/wiki/Barack_Obama --browsertime.page_cycle_delay 1000 --browsertime.post_startup_delay 1000 --firefox.profileTemplate c:\users\task_1599865692\appdata\local\temp\tmpsm5zy4\profile --skipHar --viewPort 1366x695 --pageLoadStrategy none --firefox.disableBrowsertimeExtension true --pageCompleteCheckStartWait 1000 --pageCompleteCheckPollTimeout 1000 --visualMetrics false --timeouts.pageLoad 60000 --timeouts.script 1500000 --resultDir C:\Users\task_1599865692\build\blobber_upload_dir\browsertime-results\wikipedia --video true --firefox.windowRecorder true -n 1
[task 2020-09-11T23:18:02.708Z] 23:18:02 INFO - raptor-browsertime Info: browsertime_ffmpeg: C:/Users/task_1599865692/fetches/ffmpeg-4.1.1-win64-static/bin/ffmpeg.exe
[task 2020-09-11T23:18:02.708Z] 23:18:02 INFO - raptor-browsertime Info: PATH: C:\Users\task_1599865692\fetches\ffmpeg-4.1.1-win64-static\bin;C:\Users\task_1599865692\build\venv\Scripts;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\windows\System32\OpenSSH\;C:\Program Files\Puppet Labs\Puppet\bin;C:\Program Files\Mellanox\MLNX_VPI\IB\Tools;C:\Program Files\Mellanox\MLNX_CIMProvider\lib\mft;C:\Program Files\Git\cmd;C:\Program Files (x86)\GNU\GnuPG\pub;C:\Program Files\Mercurial\;C:\Program Files\Mercurial;C:\mozilla-build\bin;C:\mozilla-build\kdiff;C:\mozilla-build\moztools-x64\bin;C:\mozilla-build\mozmake;C:\mozilla-build\nsis-3.01;C:\mozilla-build\python;C:\mozilla-build\python\Scripts;C:\mozilla-build\python3;C:\mozilla-build\msys\bin;C:\mozilla-build\msys\local\bin;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Users\task_1599865692\AppData\Local\Microsoft\WindowsApps
[task 2020-09-11T23:18:03.511Z] 23:18:03 INFO - raptor-browsertime Info: Running tests using Firefox - 1 iteration(s)
[task 2020-09-11T23:18:05.645Z] 23:18:05 INFO - raptor-browsertime Info: C:\windows\system32\cmd.exe[7524]: c:\ws\src\util-inl.h:382: Assertion `!(n > 0) || (ret != nullptr)' failed.
[task 2020-09-11T23:18:05.816Z] 23:18:05 INFO - raptor-browsertime Info: 1: 0023D65E node::MakeCallback+3982
|
Reporter | |
Comment 1•4 years ago
|
||
I'll test with the latest node version next.
|
|
Reporter | |
Comment 2•4 years ago
|
||
|
|
Reporter | |
Comment 3•4 years ago
|
||
Ah this is an interesting error from node 14:
[task 2020-09-18T16:55:24.324Z] 16:55:24 INFO - raptor-browsertime Info: Running tests using Firefox - 25 iteration(s)
[task 2020-09-18T16:55:26.619Z] 16:55:26 INFO - raptor-browsertime Info:
[task 2020-09-18T16:55:26.619Z] 16:55:26 INFO - raptor-browsertime Info: <--- Last few GCs --->
[task 2020-09-18T16:55:26.620Z] 16:55:26 INFO - raptor-browsertime Info:
[task 2020-09-18T16:55:26.620Z] 16:55:26 INFO - raptor-browsertime Info: [7132:04261710] 2974 ms: Mark-sweep (reduce) 321.4 (324.1) -> 321.4 (325.1) MB, 5.6 / 0.0 ms (average mu = 0.070, current mu = 0.017) low memory notification GC in old space requested
[task 2020-09-18T16:55:26.620Z] 16:55:26 INFO - raptor-browsertime Info: [7132:04261710] 2981 ms: Mark-sweep (reduce) 321.4 (324.1) -> 321.4 (325.1) MB, 6.2 / 0.0 ms (average mu = 0.036, current mu = 0.003) low memory notification GC in old space requested
[task 2020-09-18T16:55:26.620Z] 16:55:26 INFO - raptor-browsertime Info:
[task 2020-09-18T16:55:26.620Z] 16:55:26 INFO - raptor-browsertime Info:
[task 2020-09-18T16:55:26.620Z] 16:55:26 INFO - raptor-browsertime Info: <--- JS stacktrace --->
[task 2020-09-18T16:55:26.621Z] 16:55:26 INFO - raptor-browsertime Info:
[task 2020-09-18T16:55:26.621Z] 16:55:26 INFO - raptor-browsertime Info: FATAL ERROR: v8::ArrayBuffer::NewBackingStore Allocation failed - process out of memory
|
|
Reporter | |
Comment 4•4 years ago
|
||
Testing with node 14 and an increased memory limit: https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=8a70a2d0a2e5cc5270bbd71a2cc9492eaf9bdde5
|
|
Reporter | |
Comment 5•4 years ago
|
||
With 2Gb available for memory, it is still failing: https://treeherder.mozilla.org/#/jobs?repo=try&selectedTaskRun=aRoilum0T_mMPSKxkEwqqA.0&tier=1%2C2%2C3&revision=b5b2b08016b1e066d780e5558d9994e2478ac931
That said, it makes me wonder if the option --max-old-space-size=2048 is even being applied because I'm seeing the same sort of memory levels in the failure message:
[task 2020-09-21T19:19:03.656Z] 19:19:03 INFO - raptor-browsertime Info: Running tests using Firefox - 25 iteration(s)
[task 2020-09-21T19:19:05.871Z] 19:19:05 INFO - raptor-browsertime Info:
[task 2020-09-21T19:19:05.871Z] 19:19:05 INFO - raptor-browsertime Info: <--- Last few GCs --->
[task 2020-09-21T19:19:05.871Z] 19:19:05 INFO - raptor-browsertime Info:
[task 2020-09-21T19:19:05.871Z] 19:19:05 INFO - raptor-browsertime Info: [8732:03A615D0] 2908 ms: Mark-sweep (reduce) 318.3 (320.9) -> 318.3 (321.9) MB, 6.2 / 0.0 ms (average mu = 0.067, current mu = 0.015) low memory notification GC in old space requested
[task 2020-09-21T19:19:05.871Z] 19:19:05 INFO - raptor-browsertime Info: [8732:03A615D0] 2913 ms: Mark-sweep (reduce) 318.3 (320.9) -> 318.3 (321.9) MB, 5.5 / 0.0 ms (average mu = 0.037, current mu = 0.003) low memory notification GC in old space requested
[task 2020-09-21T19:19:05.871Z] 19:19:05 INFO - raptor-browsertime Info:
[task 2020-09-21T19:19:05.871Z] 19:19:05 INFO - raptor-browsertime Info:
[task 2020-09-21T19:19:05.871Z] 19:19:05 INFO - raptor-browsertime Info: <--- JS stacktrace --->
[task 2020-09-21T19:19:05.871Z] 19:19:05 INFO - raptor-browsertime Info:
[task 2020-09-21T19:19:05.872Z] 19:19:05 INFO - raptor-browsertime Info: FATAL ERROR: v8::ArrayBuffer::NewBackingStore Allocation failed - process out of memory
|
|
Reporter | |
Comment 6•4 years ago
|
||
I'm going to figure out exactly where we start crashing and/or running out of memory now.
|
|
Reporter | |
Comment 7•4 years ago
|
||
So we're crashing as soon as we try opening the browser:
[task 2020-09-22T13:18:54.192Z] 13:18:54 INFO - raptor-browsertime Info: Running tests using Firefox - 25 iteration(s)
[task 2020-09-22T13:18:54.192Z] 13:18:54 INFO - raptor-browsertime Info: Starting...
[task 2020-09-22T13:18:54.192Z] 13:18:54 INFO - raptor-browsertime Info: making ss
[task 2020-09-22T13:18:54.192Z] 13:18:54 INFO - raptor-browsertime Info: making opts
[task 2020-09-22T13:18:54.192Z] 13:18:54 INFO - raptor-browsertime Info: making selenium runner
[task 2020-09-22T13:18:54.192Z] 13:18:54 INFO - raptor-browsertime Info: making new video
[task 2020-09-22T13:18:54.193Z] 13:18:54 INFO - raptor-browsertime Info: extension server
[task 2020-09-22T13:18:54.193Z] 13:18:54 INFO - raptor-browsertime Info: engine delegate
[task 2020-09-22T13:18:54.193Z] 13:18:54 INFO - raptor-browsertime Info: starting browser
[task 2020-09-22T13:18:56.511Z] 13:18:56 INFO - raptor-browsertime Info:
[task 2020-09-22T13:18:56.511Z] 13:18:56 INFO - raptor-browsertime Info: <--- Last few GCs --->
[task 2020-09-22T13:18:56.511Z] 13:18:56 INFO - raptor-browsertime Info:
[task 2020-09-22T13:18:56.511Z] 13:18:56 INFO - raptor-browsertime Info: [9756:039B0508] 3042 ms: Mark-sweep (reduce) 325.2 (327.4) -> 325.2 (328.4) MB, 5.7 / 0.0 ms (average mu = 0.075, current mu = 0.025) low memory notification GC in old space requested
[task 2020-09-22T13:18:56.511Z] 13:18:56 INFO - raptor-browsertime Info: [9756:039B0508] 3048 ms: Mark-sweep (reduce) 325.2 (327.4) -> 325.2 (328.4) MB, 5.9 / 0.0 ms (average mu = 0.042, current mu = 0.006) low memory notification GC in old space requested
[task 2020-09-22T13:18:56.511Z] 13:18:56 INFO - raptor-browsertime Info:
[task 2020-09-22T13:18:56.511Z] 13:18:56 INFO - raptor-browsertime Info:
[task 2020-09-22T13:18:56.512Z] 13:18:56 INFO - raptor-browsertime Info: <--- JS stacktrace --->
[task 2020-09-22T13:18:56.512Z] 13:18:56 INFO - raptor-browsertime Info:
[task 2020-09-22T13:18:56.512Z] 13:18:56 INFO - raptor-browsertime Info: FATAL ERROR: v8::ArrayBuffer::NewBackingStore Allocation failed - process out of memory
|
|
Reporter | |
Updated•4 years ago
|
Assignee: gmierz2 → nobody
Priority: P1 → P2
|
|
Reporter | |
Comment 8•4 years ago
|
||
I found something interesting related to this: https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=74d7a53b6330e17a1bd5f33ff95c503fffd0a07e
Note how chromium and chrome tests on Windows 7 don't fail. This issue is specific to Firefox.
|
|
Reporter | |
Comment 9•3 years ago
|
||
We don't need to worry about this anymore since we no longer test on windows 7.
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → WONTFIX
You need to log in
before you can comment on or make changes to this bug.
Description
•