1512472 - Intermittent leakcheck | default 12 bytes leaked (nsStringBuffer)

Status: RESOLVED FIXED

(Core :: Networking: HTTP, defect, P2)

Description

[Treeherder Bug Filer]

Filed by: ncsoregi [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=215765867&repo=mozilla-central

https://queue.taskcluster.net/v1/task/K2vOCi6XSSCZQ6StNkfGeA/runs/0/artifacts/public/logs/live_backing.log

https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/K2vOCi6XSSCZQ6StNkfGeA/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

16:51:48     INFO - TEST-INFO | leakcheck | default leaked 1 nsStringBuffer
16:51:48     INFO - TEST-UNEXPECTED-FAIL | leakcheck | default 12 bytes leaked (nsStringBuffer)
16:51:48     INFO - 
16:51:48     INFO - leakcheck | Processing leak log file c:\users\task_1544113902\appdata\local\temp\tmp1aclsc.mozrunner\runtests_leaks_1496_tab_pid1360.log
16:51:48     INFO - 
16:51:48     INFO - == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, tab process 1360
16:51:48     INFO - 
16:51:48     INFO -      |<----------------Class--------------->|<-----Bytes------>|<----Objects---->|
16:51:48     INFO -      |                                      | Per-Inst   Leaked|   Total      Rem|
16:51:48     INFO -    0 |TOTAL                                 |       27        0|   41611        0|
16:51:48     INFO - 
16:51:48     INFO - nsTraceRefcnt::DumpStatistics: 911 entries
16:51:48     INFO - TEST-PASS | leakcheck | tab no leaks detected!
16:51:48     INFO - leakcheck | Processing leak log file c:\users\task_1544113902\appdata\local\temp\tmp1aclsc.mozrunner\runtests_leaks_1496_tab_pid1716.log
16:51:48     INFO - 
16:51:48     INFO - == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, tab process 1716
16:51:48     INFO - 
16:51:48     INFO -      |<----------------Class--------------->|<-----Bytes------>|<----Objects---->|
16:51:48     INFO -      |                                      | Per-Inst   Leaked|   Total      Rem|
16:51:48     INFO -    0 |TOTAL                                 |       27        0|   42374        0|
16:51:48     INFO - 
16:51:48     INFO - nsTraceRefcnt::DumpStatistics: 928 entries
16:51:48     INFO - TEST-PASS | leakcheck | tab no leaks detected!
16:51:48     INFO - leakcheck | Processing leak log file c:\users\task_1544113902\appdata\local\temp\tmp1aclsc.mozrunner\runtests_leaks_1496_tab_pid3024.log
16:51:48     INFO - 
16:51:48     INFO - == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, tab process 3024
16:51:48     INFO - 
16:51:48     INFO -      |<----------------Class--------------->|<-----Bytes------>|<----Objects---->|
16:51:48     INFO -      |                                      | Per-Inst   Leaked|   Total      Rem|
16:51:48     INFO -    0 |TOTAL                                 |       31        0|   16203        0|
16:51:48     INFO - 
16:51:48     INFO - nsTraceRefcnt::DumpStatistics: 421 entries
16:51:48     INFO - TEST-PASS | leakcheck | tab no leaks detected!
16:51:48     INFO - leakcheck | Processing leak log file c:\users\task_1544113902\appdata\local\temp\tmp1aclsc.mozrunner\runtests_leaks_1496_tab_pid3560.log
16:51:48     INFO - 
16:51:48     INFO - == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, tab process 3560
16:51:48     INFO - 
16:51:48     INFO -      |<----------------Class--------------->|<-----Bytes------>|<----Objects---->|
16:51:48     INFO -      |                                      | Per-Inst   Leaked|   Total      Rem|
16:51:48     INFO -    0 |TOTAL                                 |       26        0| 1034333        0|
16:51:48     INFO - 
16:51:48     INFO - nsTraceRefcnt::DumpStatistics: 1152 entries
16:51:48     INFO - TEST-PASS | leakcheck | tab no leaks detected!

Comment 1

[Intermittent Failures Robot]

3 failures in 1862 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1
* autoland: 2

Platform breakdown:
* windows7-32: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2018-12-03&endday=2018-12-09&tree=all

Comment 2

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Last test ran: css/css-paint-api/valid-image-before-load.https.html

Comment 3

[Intermittent Failures Robot]

8 failures in 2485 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-beta: 1
* autoland: 3
* mozilla-inbound: 3
* mozilla-central: 1

Platform breakdown:
* windows7-32: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2018-12-10&endday=2018-12-16&tree=all

Comment 4

[Intermittent Failures Robot]

8 failures in 2648 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1
* try: 1
* mozilla-inbound: 2
* autoland: 4

Platform breakdown:
* windows7-32: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2018-12-17&endday=2018-12-23&tree=all

Comment 5

[Intermittent Failures Robot]

3 failures in 1008 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-beta: 1
* mozilla-central: 2

Platform breakdown:
* windows7-32: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2018-12-24&endday=2018-12-30&tree=all

Comment 6

[Intermittent Failures Robot]

11 failures in 1668 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 4
* autoland: 4
* mozilla-central: 3

Platform breakdown:
* windows10-64: 3
* windows7-32: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2018-12-31&endday=2019-01-06&tree=all

Comment 7

[Intermittent Failures Robot]

7 failures in 2627 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-inbound: 4
• mozilla-central: 2

Platform breakdown:

• windows7-32: 6
• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-01-07&endday=2019-01-13&tree=all

Comment 8

[Intermittent Failures Robot]

18 failures in 2378 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4
• mozilla-inbound: 10
• mozilla-central: 4

Platform breakdown:

• windows10-64: 9
• windows7-32: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-01-14&endday=2019-01-20&tree=all

Comment 9

[Intermittent Failures Robot]

19 failures in 2383 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 2
• mozilla-central: 12
• autoland: 5

Platform breakdown:

• windows10-64-ccov: 10
• windows10-64: 3
• windows7-32: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-01-21&endday=2019-01-27&tree=all

Comment 10

[Dorel Luca [:dluca]]

This bug has failed 55 times in the last 7 days. Occurs mostly on windows10-64-ccov and a few occurrences on windows7-32 on debug build types.

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=225632331&repo=mozilla-central&lineNumber=17500

mattwoodrow: Can you please take a look at this bug?

Comment 11

[Intermittent Failures Robot]

54 failures in 2628 pushes (0.021 failures/push) were associated with this bug in the last 7 days.

This is the #31 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 6
• mozilla-inbound: 2
• mozilla-beta: 1
• mozilla-central: 45

Platform breakdown:

• windows7-32: 10
• windows10-64-ccov: 44

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-01-28&endday=2019-02-03&tree=all

Comment 12

[Matt Woodrow (:mattwoodrow)]

This doesn't look like a Web Painting bug. The css-paint-api folder is for 3rd party tests for the Houdini Paint spec, which we don't implement and isn't related to Web Painting. I also don't see that as the last failing test for other failing runs.

I'm not sure what component this belongs in, do we have anywhere for undiagnosed leaks?

Comment 13

[Andreea Pavel [:apavel]]

Hi Matt, not that I know of. As you can see in comment 2, the component was based on the last test that ran.

Last failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=226807620&repo=mozilla-central&lineNumber=17680

Test: 12:40:47 INFO - TEST-START | devtools/client/webconsole/test/mochitest/browser_webconsole_output_copy_newlines.js

Product and component Devtools::Console

Comment 14

[Intermittent Failures Robot]

41 failures in 2658 pushes (0.015 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-inbound: 1
• mozilla-central: 40

Platform breakdown:

• windows10-64-ccov: 40
• windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-02-04&endday=2019-02-10&tree=all

Comment 15

[Bogdan Tara[:bogdan_tara | bogdant]]

Recently, this looks like a perma failure:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&fromchange=6d8e6f960446fe8145ccb028d6644db879f2caa6&searchStr=windows%2C10%2Cx64%2Cccov%2Cdebug%2Cmochitests%2Cwith%2Ce10s%2Ctest-windows10-64-ccov%2Fdebug-mochitest-clipboard-e10s%2Cm-e10s%28cl%29&tochange=b9187fa10f13a7b84f21c973d33d2fdb0f37bbb0

Comment 16

[Intermittent Failures Robot]

30 failures in 533 pushes (0.056 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 30

Platform breakdown:

• windows10-64-ccov: 30

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-02-11&endday=2019-02-11&tree=all

Comment 17

[Intermittent Failures Robot]

57 failures in 2832 pushes (0.02 failures/push) were associated with this bug in the last 7 days.

This is the #26 most frequent failure this week.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• mozilla-central: 57

Platform breakdown:

• windows10-64-ccov: 57

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-02-11&endday=2019-02-17&tree=all

Comment 18

[Dorel Luca [:dluca]]

This bug failed 45 times in the last 7 days. Occurs only on windows10-64-ccov debug.

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=229329578&repo=mozilla-central&lineNumber=18080

Matt:
Any updates on this bug?

Comment 19

[Intermittent Failures Robot]

17 failures in 501 pushes (0.034 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 17

Platform breakdown:

• windows10-64-ccov: 17

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-02-19&endday=2019-02-19&tree=all

Comment 20

[Matt Woodrow (:mattwoodrow)]

I don't think the last run test is a useful metric for determining the cause of a leak unfortunately. This got put into Web Painting, but it doesn't look like there's any correlation.

Trying String as that's what's leaking, though I realize that that might not be any closer to the true cause of the bug :(

Comment 21

[Intermittent Failures Robot]

45 failures in 2602 pushes (0.017 failures/push) were associated with this bug in the last 7 days.

This is the #29 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 45

Platform breakdown:

• windows10-64-ccov: 45

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-02-18&endday=2019-02-24&tree=all

Comment 22

[Andreea Pavel [:apavel]]

There are 43 total failures in the last 7 days on win7 and win10 debug.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=230861225&repo=autoland&lineNumber=121792

22:42:40 INFO - TEST-START | /xhr/xmlhttprequest-unsent.htm
22:42:40 INFO - Closing window 700079669249
22:42:40 INFO - PID 1188 | [Parent 5120, Gecko_IOThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
22:42:40 INFO - PID 1188 | [Child 5280, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
22:42:40 INFO - PID 1188 | [Child 5280, Chrome_ChildThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
22:42:40 INFO - PID 1188 | [Child 5280, Main Thread] WARNING: A runnable was posted to a worker that is already shutting down!: file z:/build/build/src/dom/workers/WorkerPrivate.cpp, line 1357
22:42:40 INFO - PID 1188 | [Child 5280, Main Thread] WARNING: Failed to dispatch offline status change event!: file z:/build/build/src/dom/workers/WorkerPrivate.cpp, line 1809
22:42:40 INFO - PID 1188 | [Parent 5120, Gecko_IOThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
22:42:40 INFO - PID 1188 | [Parent 5120, Main Thread] WARNING: NS_ENSURE_TRUE(aCsp) failed: file z:/build/build/src/dom/base/nsOpenURIInFrameParams.cpp, line 84
22:42:40 INFO - PID 1188 | [Parent 5120, Gecko_IOThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
22:42:40 INFO - PID 1188 | ++DOCSHELL 00A67400 == 1 [pid = 2844] [id = {e43de823-8a7f-41df-8b56-8f8f0d930b63}]
22:42:40 INFO - PID 1188 | [Child 2844, Main Thread] WARNING: 'NS_FAILED(rv)', file z:/build/build/src/widget/windows/nsLookAndFeel.cpp, line 850
22:42:40 INFO - PID 1188 | ++DOMWINDOW == 1 (00AFB2E0) [pid = 2844] [serial = 1] [outer = 00000000]
22:42:40 INFO - PID 1188 | [Parent 5120, Gecko_IOThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 341
22:42:40 INFO - PID 1188 | ++DOMWINDOW == 2 (00A69000) [pid = 2844] [serial = 2] [outer = 00AFB2E0]
22:42:40 INFO - PID 1188 | [Child 2844, Main Thread] WARNING: 'NS_FAILED(GetAccentColor(unused))', file z:/build/build/src/widget/windows/nsLookAndFeel.cpp, line 480
22:42:40 INFO - PID 1188 | --DOCSHELL 01067400 == 0 [pid = 5280] [id = {42ca9d2e-ea88-420d-803f-a2a7fb6ca899}]
22:42:40 INFO - PID 1188 | --DOMWINDOW == 3 (010FC2E0) [pid = 5280] [serial = 1] [outer = 00000000] [url = http://web-platform.test:8000/xhr/xmlhttprequest-timeout-worker-twice.html?load%20fires%20normally%20with%20same%20timeout%20set%20twice]
22:42:40 INFO - PID 1188 | --DOMWINDOW == 2 (01069400) [pid = 5280] [serial = 2] [outer = 00000000] [url = about:blank]
22:42:40 INFO - PID 1188 | --DOMWINDOW == 1 (10C96800) [pid = 5280] [serial = 3] [outer = 00000000] [url = about:blank]
22:42:40 INFO - PID 1188 | --DOMWINDOW == 0 (14784800) [pid = 5280] [serial = 4] [outer = 00000000] [url = http://web-platform.test:8000/xhr/xmlhttprequest-timeout-worker-twice.html?load%20fires%20normally%20with%20same%20timeout%20set%20twice]
22:42:40 INFO - PID 1188 | nsStringStats
22:42:40 INFO - PID 1188 | => mAllocCount: 7839
22:42:40 INFO - PID 1188 | => mReallocCount: 0
22:42:40 INFO - PID 1188 | => mFreeCount: 7839
22:42:40 INFO - PID 1188 | => mShareCount: 6486
22:42:40 INFO - PID 1188 | => mAdoptCount: 469
22:42:40 INFO - PID 1188 | => mAdoptFreeCount: 477
22:42:40 INFO - PID 1188 | => Process ID: 5280, Thread ID: 2072
22:42:40 INFO - PID 1188 | ++DOMWINDOW == 3 (07B96800) [pid = 2844] [serial = 3] [outer = 00AFB2E0]
22:42:40 INFO - PID 1188 | ### XPCOM_MEM_BLOAT_LOG defined -- logging bloat/leaks to c:\users\task_1551303150\appdata\local\temp\tmpt8fnha.mozrunner\runtests_leaks_2668_tab_pid3244.log
22:42:40 INFO - PID 1188 | [Child 2844, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005: file z:/build/build/src/xpcom/base/nsSystemInfo.cpp, line 111
22:42:40 INFO - PID 1188 | [Child 2844, Main Thread] WARNING: site security information will not be persisted: file z:/build/build/src/security/manager/ssl/nsSiteSecurityService.cpp, line 506
22:42:40 INFO - PID 1188 | [Parent 5120, Main Thread] WARNING: 'NS_FAILED(rv)', file z:/build/build/src/netwerk/url-classifier/UrlClassifierCommon.cpp, line 122
22:42:40 INFO - PID 1188 | [Parent 5120, Main Thread] WARNING: '!parent', file z:/build/build/src/netwerk/ipc/NeckoParent.cpp, line 970
22:42:40 INFO - PID 1188 | ++DOMWINDOW == 4 (0B680400) [pid = 2844] [serial = 4] [outer = 00AFB2E0]
22:42:40 INFO - PID 1188 | [Child 3244, Main Thread] WARNING: Failed to acquire a DXGI adapter for enumerating outputs.: file z:/build/build/src/gfx/thebes/DeviceManagerDx.cpp, line 143
22:42:40 INFO - .
22:42:40 INFO - TEST-OK | /xhr/xmlhttprequest-unsent.htm | took 551ms

22:42:42 INFO - PROCESS LEAKS c:\users\task_1551303150\appdata\local\temp\tmpt8fnha.mozrunner\runtests_leaks_2668.log
22:42:42 INFO - leakcheck | Processing log file c:\users\task_1551303150\appdata\local\temp\tmpt8fnha.mozrunner\runtests_leaks_2668.log for scope /xhr
22:42:42 INFO - TEST-INFO | leakcheck | default process: leak threshold set at 0 bytes
22:42:42 INFO - TEST-INFO | leakcheck | plugin process: leak threshold set at 0 bytes
22:42:42 INFO - TEST-INFO | leakcheck | tab process: leak threshold set at 10000 bytes
22:42:42 INFO - TEST-INFO | leakcheck | gmplugin process: leak threshold set at 20000 bytes
22:42:42 INFO - TEST-INFO | leakcheck | gpu process: leak threshold set at 0 bytes
22:42:42 INFO - TEST-INFO | leakcheck | rdd process: leak threshold set at 400 bytes
22:42:42 INFO - TEST-INFO | leakcheck | vr process: leak threshold set at 0 bytes
22:42:42 INFO - TEST-INFO | leakcheck | socket process: leak threshold set at 0 bytes
22:42:42 INFO - leakcheck | Processing leak log file c:\users\task_1551303150\appdata\local\temp\tmpt8fnha.mozrunner\runtests_leaks_2668.log
22:42:42 INFO -
22:42:42 INFO - == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, default process 5120
22:42:42 INFO -
22:42:42 INFO - |<----------------Class--------------->|<-----Bytes------>|<----Objects---->|
22:42:42 INFO - | | Per-Inst Leaked| Total Rem|
22:42:42 INFO - 0 |TOTAL | 25 12|310252686 1|
22:42:42 INFO - 1891 |nsStringBuffer | 12 12| 1160096 1|
22:42:42 INFO -
22:42:42 INFO - nsTraceRefcnt::DumpStatistics: 2067 entries
22:42:42 INFO - TEST-INFO | leakcheck | default leaked 1 nsStringBuffer
22:42:42 INFO - TEST-UNEXPECTED-FAIL | leakcheck | default 12 bytes leaked (nsStringBuffer)
22:42:42 INFO -
22:42:42 INFO - leakcheck | Processing leak log file c:\users\task_1551303150\appdata\local\temp\tmpt8fnha.mozrunner\runtests_leaks_2668_tab_pid1060.log

Nathan, can you assign someone to take a look at this?

Comment 23

[Nathan Froyd [:froydnj]]

The windows-64-ccov failures are all in mochitest-clipboard, which I think makes this a DOM issue or possibly an issue for the ccov folks to look at? Not completely sure. Andrew?

The wpt failures are all on 32-bit Windows, which is a separate thing, and I think we should focus this bug on the mochitest-clipboard failures.

Comment 24

[Andrew Overholt [:overholt]]

If it is clipboard-related, we should ask Neil or maybe Nika. But maybe mccr8 has ideas?

Comment 25

[Andrew McCreight [:mccr8]]

I've hacked up a patch before to dump out the contents of a leaked string buffer. That might help. Besides that or refcount logging I have no particularly great strategies for tiny leaks like this.

Comment 26

[Andrew McCreight [:mccr8]]

I'm not sure how much headway I can make, but I'll try to get more information about what is leaking.

Comment 27

[Intermittent Failures Robot]

39 failures in 2981 pushes (0.013 failures/push) were associated with this bug in the last 7 days.

This is the #37 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 31
• mozilla-inbound: 3
• mozilla-beta: 1
• autoland: 4

Platform breakdown:

• windows10-64-ccov: 31
• windows7-32: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-02-25&endday=2019-03-03&tree=all

Comment 28

[Andrew McCreight [:mccr8]]

According to my logging, the leaking string is: 潬慣桬獯t

The allocation stack for the string is:

#00: nsTSubstring<char>::StartBulkWriteImpl(unsigned int,unsigned int,bool,unsigned int,unsigned int,unsigned int) [xpcom/string/nsTSubstring.cpp:203]
#01: nsTSubstring<char>::Assign(char const *,unsigned int,std::nothrow_t const &) [xpcom/string/nsTSubstring.cpp:409]
#02: nsTSubstring<char>::Assign(nsTSubstring<char> const &,std::nothrow_t const &) [xpcom/string/nsTSubstring.cpp:499]
#03: nsTSubstring<char>::Assign(nsTSubstring<char> const &) [xpcom/string/nsTSubstring.cpp:455]
#04: mozilla::net::nsHttpConnectionInfo::Init(nsTSubstring<char> const &,int,nsTSubstring<char> const &,nsTSubstring<char> const &,mozilla::net::nsProxyInfo *,mozilla::OriginAttributes const &,bool) [netwerk/protocol/http/nsHttpConnectionInfo.cpp:101]
#05: mozilla::net::nsHttpConnectionInfo::nsHttpConnectionInfo(nsTSubstring<char> const &,int,nsTSubstring<char> const &,nsTSubstring<char> const &,mozilla::net::nsProxyInfo *,mozilla::OriginAttributes const &,bool) [netwerk/protocol/http/nsHttpConnectionInfo.cpp:51]
#06: mozilla::net::nsHttpHandler::SpeculativeConnectInternal(nsIURI *,nsIPrincipal *,nsIInterfaceRequestor *,bool) [netwerk/protocol/http/nsHttpHandler.cpp:2482]
#07: mozilla::net::nsHttpHandler::SpeculativeConnect2(nsIURI *,nsIPrincipal *,nsIInterfaceRequestor *) [netwerk/protocol/http/nsHttpHandler.cpp:2490]
#08: mozilla::net::IOServiceProxyCallback::OnProxyAvailable(nsICancelable *,nsIChannel *,nsIProxyInfo *,nsresult) [netwerk/base/nsIOService.cpp:1762]
#09: mozilla::net::nsAsyncResolveRequest::DoCallback() [netwerk/base/nsProtocolProxyService.cpp:396]
#10: nsresult mozilla::net::nsAsyncResolveRequest::ProcessLocally::<unnamed-tag>::operator()(class mozilla::net::nsAsyncResolveRequest *, class nsIProxyInfo *, bool) [netwerk/base/nsProtocolProxyService.cpp:244]
#11: mozilla::net::nsAsyncResolveRequest::AsyncApplyFilters::Finish() [netwerk/base/nsProtocolProxyService.cpp:590]
#12: mozilla::net::nsAsyncResolveRequest::AsyncApplyFilters::ProcessNextFilter() [netwerk/base/nsProtocolProxyService.cpp:500]
#13: mozilla::net::nsAsyncResolveRequest::AsyncApplyFilters::AsyncProcess(mozilla::net::nsAsyncResolveRequest *) [netwerk/base/nsProtocolProxyService.cpp:477]
#14: mozilla::net::nsAsyncResolveRequest::ProcessLocally(mozilla::net::nsProtocolInfo &,nsIProxyInfo *,bool) [netwerk/base/nsProtocolProxyService.cpp:253]
#15: mozilla::net::nsProtocolProxyService::AsyncResolveInternal(nsIChannel *,unsigned int,nsIProtocolProxyCallback *,nsICancelable * *,bool,nsIEventTarget *) [netwerk/base/nsProtocolProxyService.cpp:1546]
#16: mozilla::net::nsProtocolProxyService::AsyncResolve2(nsIChannel *,unsigned int,nsIProtocolProxyCallback *,nsIEventTarget *,nsICancelable * *) [netwerk/base/nsProtocolProxyService.cpp:1566]
#17: mozilla::net::nsIOService::SpeculativeConnectInternal(nsIURI *,nsIPrincipal *,nsIInterfaceRequestor *,bool) [netwerk/base/nsIOService.cpp:1828]
#18: mozilla::net::nsIOService::SpeculativeConnect2(nsIURI *,nsIPrincipal *,nsIInterfaceRequestor *) [netwerk/base/nsIOService.cpp:1838]
#19: XPTC__InvokebyIndex // This just means it was called from JS.

Line 101 of nsHttpConnectionInfo.cpp is a call to nsHttpConnectionInfo::SetOriginServer(). There's a lot of string stuff that is called in there, so I'm not sure exactly where it is being created.

I'm doing another try push to get the JS stack to figure out how this code is being called.

Comment 29

[Andrew McCreight [:mccr8]]

None of the Chinese characters in that string appear in any test, so I wonder if the string is just being decoded incorrectly in my logging code. It worked when I tried it on some simple ASCII values.

Comment 30

[Andrew McCreight [:mccr8]]

Yeah, if I cast the string u"潬慣桬獯t" to a char* and print it out, I get the string "localhost" which makes more sense for something to appear in networking code. :)

Comment 31

[Andrew Overholt [:overholt]]

(In reply to Andrew McCreight [:mccr8] from comment #30)

Yeah, if I cast the string u"潬慣桬獯t" to a char* and print it out, I get the string "localhost" which makes more sense for something to appear in networking code. :)

But it was more fun to try to search for sub-strings of the Chinese characters in searchfox ;)

Thank you for tracking this down, Andrew!

Comment 32

[Andrew McCreight [:mccr8]]

I tried to get a JS stack for this, but I couldn't get it to work with nsStringBuffer. Hopefully a networking person can figure out what is going on here. If not, I could try refcount logging or rr and see if that sheds any light on this.

Comment 33

[Neha Kochar [:neha]]

Valentin, can you look into this leak which seems to be from networking code (see comment 28)?

Comment 34

[Intermittent Failures Robot]

24 failures in 2818 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 24

Platform breakdown:

• windows10-64-ccov: 24

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-03-04&endday=2019-03-10&tree=all

Comment 35

[Stefan Hindli [:stefan_hindli]]

Over the last 7 days there are 32 failures on this bug. These happen on: windows10-64, windows10-64-ccov, windows7-32.

Here is the most recent log failure: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=233819125&repo=autoland&lineNumber=11518

06:18:00 INFO - zombiecheck | Checking for orphan process with PID: 5252
06:18:00 INFO - zombiecheck | Checking for orphan process with PID: 6044
06:18:00 INFO - Stopping web server
06:18:00 INFO - Stopping web socket server
06:18:00 INFO - Stopping ssltunnel
06:18:00 INFO - leakcheck | Processing log file c:\users\task_1552543054\appdata\local\temp\tmpbr9tdm.mozrunner\runtests_leaks.log
06:18:00 INFO - TEST-INFO | leakcheck | default process: leak threshold set at 0 bytes
06:18:00 INFO - TEST-INFO | leakcheck | plugin process: leak threshold set at 0 bytes
06:18:00 INFO - TEST-INFO | leakcheck | tab process: leak threshold set at 1000 bytes
06:18:00 INFO - TEST-INFO | leakcheck | gmplugin process: leak threshold set at 20000 bytes
06:18:00 INFO - TEST-INFO | leakcheck | gpu process: leak threshold set at 0 bytes
06:18:00 INFO - TEST-INFO | leakcheck | rdd process: leak threshold set at 400 bytes
06:18:00 INFO - TEST-INFO | leakcheck | vr process: leak threshold set at 0 bytes
06:18:00 INFO - TEST-INFO | leakcheck | socket process: leak threshold set at 0 bytes
06:18:00 INFO - leakcheck | Processing leak log file c:\users\task_1552543054\appdata\local\temp\tmpbr9tdm.mozrunner\runtests_leaks.log
06:18:00 INFO -
06:18:00 INFO - == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, default process 2904
06:18:00 INFO -
06:18:00 INFO - |<----------------Class--------------->|<-----Bytes------>|<----Objects---->|
06:18:00 INFO - | | Per-Inst Leaked| Total Rem|
06:18:00 INFO - 0 |TOTAL | 23 12|151739448 1|
06:18:00 INFO - 1860 |nsStringBuffer | 12 12| 670777 1|
06:18:00 INFO -
06:18:00 INFO - nsTraceRefcnt::DumpStatistics: 2046 entries
06:18:00 INFO - TEST-INFO | leakcheck | default leaked 1 nsStringBuffer
06:18:00 INFO - TEST-UNEXPECTED-FAIL | leakcheck | default 12 bytes leaked (nsStringBuffer)
06:18:00 INFO -
06:18:00 INFO - leakcheck | Processing leak log file c:\users\task_1552543054\appdata\local\temp\tmpbr9tdm.mozrunner\runtests_leaks_tab_pid196.log
06:18:00 INFO -
06:18:00 INFO - == BloatView: ALL (cumulative) LEAK AND BLOAT STATISTICS, tab process 196
06:18:00 INFO -
06:18:00 INFO - |<----------------Class--------------->|<-----Bytes------>|<----Objects---->|
06:18:00 INFO - | | Per-Inst Leaked| Total Rem|
06:18:00 INFO - 0 |TOTAL | 28 0| 35196 0|
06:18:00 INFO -

Comment 36

[Valentin Gosu [:valentin]]

I think the problem is in nsHttpHandler::SpeculativeConnectInternal.

auto *ci = new nsHttpConnectionInfo(...);
return SpeculativeConnect(ci, aCallbacks); // calls into nsHttpConnectionMgr::SpeculativeConnect

If nsHttpConnectionMgr::SpeculativeConnect returns early, we'd leak the nsHttpConnectionInfo.

Comment 37

[Valentin Gosu [:valentin]]

Attachment: Bug 1512472 - Hold nsHttpConnectionInfo in a RefPtr to avoid leak when returning early r=dragana!

nsHttpConnectionMgr::SpeculativeConnect may return early, without assigning ci to a RefPtr. In that case, we'd leak the nsHttpConnectionInfo. We need to have it in a RefPtr from the start to make sure we can't leak it.

Comment 38

[Intermittent Failures Robot]

26 failures in 2771 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 18
• mozilla-inbound: 6
• autoland: 2

Platform breakdown:

• windows10-64-ccov: 14
• windows7-32: 6
• windows10-64: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-03-11&endday=2019-03-17&tree=all

Comment 39

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e94a55919dea
Hold nsHttpConnectionInfo in a RefPtr to avoid leak when returning early r=dragana

Comment 40

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/e94a55919dea

Comment 41

[Intermittent Failures Robot]

3 failures in 2561 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3

Platform breakdown:

• windows10-64-ccov: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1512472&startday=2019-03-18&endday=2019-03-24&tree=all

Comment 42

[Ryan VanderMeulen [:RyanVM]]

Please nominate this for Beta approval when you get a chance.

Comment 43

[Valentin Gosu [:valentin]]

Comment on attachment 9050971 [details]
Bug 1512472 - Hold nsHttpConnectionInfo in a RefPtr to avoid leak when returning early r=dragana!

Beta/Release Uplift Approval Request

• Feature/Bug causing the regression: None
• User impact if declined: Memory leak when attempting to speculatively connect to a local IP address.
• Is this code covered by automated tests?: No
• Has the fix been verified in Nightly?: Yes
• Needs manual test from QE?: No
• If yes, steps to reproduce:
• List of other uplifts needed: None
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): It just makes sure the variable is immediately assigned into a RefPtr so there are no chances of leaking it.
• String changes made/needed:

Comment 44

[Pascal Chevrel:pascalc]

Comment on attachment 9050971 [details]
Bug 1512472 - Hold nsHttpConnectionInfo in a RefPtr to avoid leak when returning early r=dragana!

Fix for a memory leak on Nightly for +10 days, uplift approved for 67 beta 7, thanks.

Comment 45

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/releases/mozilla-beta/rev/3b2fad2fe236