1326305 - Intermittent leakcheck | default process: 636891 bytes leaked (AsyncLatencyLogger, BackstagePass, CSSStyleSheet, CSSStyleSheetInner, CacheObserver, ...)

Status: RESOLVED FIXED

(Toolkit :: General, defect)

Description

[Treeherder Bug Filer]

Filed by: wkocher [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=65324510&repo=autoland

https://queue.taskcluster.net/v1/task/YGp8m3hSTneGAC_K5XjcDA/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Ryan VanderMeulen [:RyanVM]]

Looks like we're leaking the world here. Kats, is this the leak you were fixing on the graphics branch?

Comment 2

[Kartikaya Gupta (email:[e-mail])]

No, the leak on the graphics branch was specific to the webrender code, which is only on the graphics branch.

Comment 3

[Ryan VanderMeulen [:RyanVM]]

It's always happening in the toolkit/modules/tests/browser directory, and it started on the push for bug 1325464 (confirmed by a large number of retriggers before and after).

Conveniently, that push included changes in that test directory via rev cd10db6087dd. Or maybe one of the JSM changes in toolkit/modules?
https://hg.mozilla.org/integration/autoland/rev/cd10db6087ddd3a8e872c0de6ef35a886250173e

Any idea what might be going on here, Jared?

Comment 4

[Jared Wein [:jaws] (please needinfo? me)]

I'm looking in to this now.

Comment 5

[Jared Wein [:jaws] (please needinfo? me)]

While working on bug 1325464 I encountered a test failure that was due to the usage of object-shorthand for constructors. This was commented on at https://hg.mozilla.org/mozilla-central/annotate/15ea2e33fcd4/browser/base/content/browser-fullScreenAndPointerLock.js#l34

I've pushed to tryserver a patch that reverts the changes from bug 1325464 in /toolkit/modules with respect to constructors: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a29979633ad9577c97c1375d98e2c0276d432c8d

Comment 6

[Intermittent Failures Robot]

58 failures in 70 pushes (0.829 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* autoland: 37
* mozilla-inbound: 12
* mozilla-central: 7
* try: 2

Platform breakdown:
* linux64: 39
* linux32: 19

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1326305&startday=2016-12-30&endday=2016-12-30&tree=all

Comment 7

[Xidorn Quan [:xidorn] UTC+11]

(In reply to Jared Wein [:jaws] (please needinfo? me) from comment #5)
> While working on bug 1325464 I encountered a test failure that was due to
> the usage of object-shorthand for constructors. This was commented on at
> https://hg.mozilla.org/mozilla-central/annotate/15ea2e33fcd4/browser/base/
> content/browser-fullScreenAndPointerLock.js#l34

That one should be rewritten to a class (actually my initial impl was a class, but es6-class was not enabled at that moment).

Comment 8

[Jared Wein [:jaws] (please needinfo? me)]

The above trypush didn't fix the leak. This trypush reverts all of the changes under /toolkit/modules, https://treeherder.mozilla.org/#/jobs?repo=try&revision=5c3d4d4674d4727b2da9d3403e9292fec24415b5.

It fails the eslint job but that can be ignored because this is just seeing if the leak goes away.

(In reply to Xidorn Quan [:xidorn] (UTC+10) from comment #7)
> (In reply to Jared Wein [:jaws] (please needinfo? me) from comment #5)
> > While working on bug 1325464 I encountered a test failure that was due to
> > the usage of object-shorthand for constructors. This was commented on at
> > https://hg.mozilla.org/mozilla-central/annotate/15ea2e33fcd4/browser/base/
> > content/browser-fullScreenAndPointerLock.js#l34
> 
> That one should be rewritten to a class (actually my initial impl was a
> class, but es6-class was not enabled at that moment).

Can you please file a bug for this? That could be good-first-bug material :)

Comment 9

[Xidorn Quan [:xidorn] UTC+11]

(In reply to Jared Wein [:jaws] (please needinfo? me) from comment #8)
> > That one should be rewritten to a class (actually my initial impl was a
> > class, but es6-class was not enabled at that moment).
> 
> Can you please file a bug for this? That could be good-first-bug material :)

Filed bug 1326581.

Comment 10

[Intermittent Failures Robot]

79 failures in 305 pushes (0.259 failures/push) were associated with this bug in the last 7 days. 

This is the #2 most frequent failure this week. 

** This failure happened more than 50 times this week! Resolving this bug is a high priority. **

Repository breakdown:
* autoland: 47
* mozilla-inbound: 12
* try: 11
* mozilla-central: 9

Platform breakdown:
* linux64: 54
* linux32: 25

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1326305&startday=2016-12-26&endday=2017-01-01&tree=all

Comment 11

[Jared Wein [:jaws] (please needinfo? me)]

The following try pushes reverse all the changes under second level directories:

browser/base: https://treeherder.mozilla.org/#/jobs?repo=try&revision=9324465b85578cbafcc59f967306218b54dfd77f
browser/components: https://treeherder.mozilla.org/#/jobs?repo=try&revision=15750c6100d2b7b5b4813e06fd37a1555958bb07
browser/experiments: https://treeherder.mozilla.org/#/jobs?repo=try&revision=0ef915d41a67ecc73396953f53147e3d787c2fb7
browser/extensions: https://treeherder.mozilla.org/#/jobs?repo=try&revision=dd8ddd85baf376e2e6d877cc20ca991671c33ca0
browser/modules: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f12b9a4819e55d68c5dcd145f5502c0fa56a3fd2
browser/tools: https://treeherder.mozilla.org/#/jobs?repo=try&revision=dba43d79118800b62fb04d74e43e62450a3da593
storage/test: https://treeherder.mozilla.org/#/jobs?repo=try&revision=5dd00c6c0174e0871d0d64a7fe2767bd4c386cd5
toolkit/components: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f4f67d511dd2172d9742eeea0ed095cf73e15a56
toolkit/content: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7ca55707a5a9bf35e8b888abcc278ba3441a3449
toolkit/crashreporter: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ab4e21cfdeeabb663c1561abc8e7671460eb884b
toolkit/forgetaboutsite: https://treeherder.mozilla.org/#/jobs?repo=try&revision=96d0507b4851269188716704de279067f9c92daf
toolkit/identity: https://treeherder.mozilla.org/#/jobs?repo=try&revision=bd84e41f9802d11b3a08096d2f5b22fc952635f4
toolkit/modules: https://treeherder.mozilla.org/#/jobs?repo=try&revision=5213738d2650e30df6c99e089c61c725f3adbaaa
toolkit/mozapps: https://treeherder.mozilla.org/#/jobs?repo=try&revision=25f2856ab37560c5b7363b0b2f0b64b1f06c46b3

Comment 12

[Jared Wein [:jaws] (please needinfo? me)]

Of all the trypushes, https://treeherder.mozilla.org/#/jobs?repo=try&revision=25f2856ab37560c5b7363b0b2f0b64b1f06c46b3 is the only one that doesn't reproduce the leak.

I've now split that try push in to 8 separate try pushes and should have results soon.

Chunk 1: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a5b256966573b5a66f37dde636794442b208016b
Chunk 2: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c0e5dde3f9a33538b20279a476f6679404ee5dce
Chunk 3: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8c7668f38d9ec1d3c94a6993072ad075832f8353
Chunk 4: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ba36a97d99dd518a3b2a45197d268f2d0bb9b9a0
Chunk 5: https://treeherder.mozilla.org/#/jobs?repo=try&revision=3af50cad473bbeb7840129cfc690354afcf2be1b
Chunk 6: https://treeherder.mozilla.org/#/jobs?repo=try&revision=4a5176d4c305f88d950fb16a6a94ca7a5d68c94d
Chunk 7: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c7b89b3e21128bcaadf5a5195daca4b67c7342cc
Chunk 8: https://treeherder.mozilla.org/#/jobs?repo=try&revision=48654af7bcdab674c7f58a8da031e255d8553a53

Comment 13

[Jared Wein [:jaws] (please needinfo? me)]

Of the previous trypushes, https://treeherder.mozilla.org/#/jobs?repo=try&revision=c0e5dde3f9a33538b20279a476f6679404ee5dce is the only one that doesn't reproduce the leak.

I scanned through the list of files that changed in that push and ProductAddonChecker.jsm stands out as the only file that changed that required a manual merge. It looks like I screwed up the merge and this is the cause for the leaks.

I've pushed a patch to tryserver that should fix this and will request review once I have confirmation that it is fixed: https://treeherder.mozilla.org/#/jobs?repo=try&revision=46b06b6b9ec382eb66e1474545448a3d9a61d704

Comment 14

[Jared Wein [:jaws] (please needinfo? me)]

Attachment: Bug 1326305 - Revert bad manual merge from bug 1325464 that lead to memory leaks.

Review commit: https://reviewboard.mozilla.org/r/102278/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/102278/

Comment 15

[Robert Helmer [:rhelmer]]

Comment on attachment 8823765 [details]
Bug 1326305 - Revert bad manual merge from bug 1325464 that lead to memory leaks.

https://reviewboard.mozilla.org/r/102278/#review102996

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0be68b87d9db
Revert bad manual merge from bug 1325464 that lead to memory leaks. r=rhelmer

Comment 17

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/0be68b87d9db