1340425 - Intermittent leakcheck | default process: 4734311 bytes leaked (AbstractThread, AbstractWatcher, Animation, AnimationEffectReadOnly, AnimationEffectTimingReadOnly, ...)

Status: RESOLVED WORKSFORME

(Core :: General, defect)

Description

[Treeherder Bug Filer]

Filed by: philringnalda [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=78035070&repo=mozilla-central

https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-central-win32-debug/1487266159/mozilla-central_win7_vm-debug_test-mochitest-browser-chrome-7-bm129-tests1-windows-build131.txt.gz

Comment 1

[Intermittent Failures Robot]

13 failures in 812 pushes (0.016 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-inbound: 4
* autoland: 4
* try: 3
* mozilla-central: 1
* mozilla-aurora: 1

Platform breakdown:
* windows7-32-vm: 11
* windows8-64: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-02-20&endday=2017-02-26&tree=all

Comment 2

[Intermittent Failures Robot]

13 failures in 867 pushes (0.015 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 6
* mozilla-inbound: 4
* try: 1
* oak: 1
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 13

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-04-03&endday=2017-04-09&tree=all

Comment 3

[Intermittent Failures Robot]

6 failures in 817 pushes (0.007 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 4
* mozilla-inbound: 1
* mozilla-central: 1

Platform breakdown:
* windows7-32-vm: 4
* windows8-64: 1
* windows7-32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-04-17&endday=2017-04-23&tree=all

Comment 4

[Intermittent Failures Robot]

5 failures in 770 pushes (0.006 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-central: 2
* autoland: 2
* mozilla-inbound: 1

Platform breakdown:
* linux32: 3
* linux64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-05-01&endday=2017-05-07&tree=all

Comment 5

[Intermittent Failures Robot]

15 failures in 111 pushes (0.135 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-inbound: 15

Platform breakdown:
* osx-10-10: 15

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-05-15&endday=2017-05-15&tree=all

Comment 6

[Geoff Brown [:gbrown]]

osx leaks on May 15 caused by bug 1364911, which was backed out.

Comment 7

[Intermittent Failures Robot]

44 failures in 77 pushes (0.571 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 29
* mozilla-inbound: 14
* oak: 1

Platform breakdown:
* osx-10-10: 44

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-05-16&endday=2017-05-16&tree=all

Comment 8

[Intermittent Failures Robot]

71 failures in 777 pushes (0.091 failures/push) were associated with this bug in the last 7 days. 

This is the #13 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 32
* mozilla-inbound: 31
* try: 3
* mozilla-beta: 3
* oak: 1
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 68
* windows8-64: 2
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-05-15&endday=2017-05-21&tree=all

Comment 9

[Intermittent Failures Robot]

6 failures in 892 pushes (0.007 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 2
* autoland: 2
* pine: 1
* mozilla-central: 1

Platform breakdown:
* linux32: 5
* linux64-ccov: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-06-19&endday=2017-06-25&tree=all

Comment 10

[Intermittent Failures Robot]

23 failures in 718 pushes (0.032 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 13
* autoland: 7
* try: 2
* mozilla-central: 1

Platform breakdown:
* linux32: 18
* osx-10-10: 2
* linux64-qr: 2
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-06-26&endday=2017-07-02&tree=all

Comment 11

[Intermittent Failures Robot]

36 failures in 656 pushes (0.055 failures/push) were associated with this bug in the last 7 days. 

This is the #50 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 17
* mozilla-inbound: 12
* mozilla-central: 4
* try: 2
* pine: 1

Platform breakdown:
* linux32: 26
* osx-10-10: 5
* osx-cross: 3
* linux64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-07-03&endday=2017-07-09&tree=all

Comment 12

[Geoff Brown [:gbrown]]

This bug previously tracked a leak in browser-chrome tests; that was resolved around May 16.


Currently, this intermittent leak is observed in mochitest-a11y, and OF suggests it started around June 21.

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=mochitest-a11y%20debug%20linux&tochange=7bdf1aa4e51482e51c0104845dc249cdc027a809&fromchange=8e1de9b8675384f6307830d381fcd195a432ea83 strongly suggests this leak started with https://hg.mozilla.org/integration/mozilla-inbound/rev/e2efdd8dcd82d74128f47410014d491e76276263, bug 833143.  I note https://bugzilla.mozilla.org/show_bug.cgi?id=833143#c41: "this may cause new intermittent leaks".

:mccr8 -- Just checking with you in case this leak is concerning or you can offer insight into resolving it.

Comment 13

[Andrew McCreight (out of office until 8/21) [:mccr8]]

That's bad. I'll look into it. (Unfortunately, large leaks like this end up all looking the same as far as Treeherder classification goes.)

Comment 14

[Andrew McCreight (out of office until 8/21) [:mccr8]]

Thanks for tracking down what caused this.

Comment 15

[Intermittent Failures Robot]

22 failures in 720 pushes (0.031 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* try: 11
* autoland: 6
* pine: 2
* oak: 1
* mozilla-inbound: 1
* mozilla-central: 1

Platform breakdown:
* linux32: 17
* osx-cross: 3
* linux64-stylo: 1
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-07-10&endday=2017-07-16&tree=all

Comment 16

[Intermittent Failures Robot]

23 failures in 822 pushes (0.028 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 10
* try: 6
* pine: 4
* mozilla-inbound: 3

Platform breakdown:
* linux32: 13
* osx-10-10: 8
* linux64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-07-17&endday=2017-07-23&tree=all

Comment 17

[Intermittent Failures Robot]

11 failures in 1008 pushes (0.011 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 5
* autoland: 3
* mozilla-central: 2
* try: 1

Platform breakdown:
* linux32: 7
* windows7-32: 1
* osx-10-10: 1
* linux64-qr: 1
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-07-24&endday=2017-07-30&tree=all

Comment 18

[Intermittent Failures Robot]

17 failures in 888 pushes (0.019 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 7
* mozilla-inbound: 6
* mozilla-central: 3
* try: 1

Platform breakdown:
* linux32: 9
* linux64: 6
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-07-31&endday=2017-08-06&tree=all

Comment 19

[Intermittent Failures Robot]

15 failures in 901 pushes (0.017 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 6
* mozilla-central: 5
* mozilla-inbound: 4

Platform breakdown:
* linux32: 8
* linux64: 3
* osx-10-10: 2
* windows10-64: 1
* linux64-qr: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-08-07&endday=2017-08-13&tree=all

Comment 20

[Intermittent Failures Robot]

18 failures in 949 pushes (0.019 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 9
* mozilla-central: 5
* mozilla-inbound: 3
* try: 1

Platform breakdown:
* linux32: 8
* osx-10-10: 3
* windows8-64: 2
* linux64: 2
* windows7-32-stylo: 1
* macosx64-stylo: 1
* linux64-ccov: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-08-14&endday=2017-08-20&tree=all

Comment 21

[Joel Maher ( :jmaher ) (UTC -8)]

I see this has a steady stream of intermittents- not a high failure rate, but steady- :mccr8, I see you assigned yourself to this, is there a chance you had some WIP code to finish up?

Comment 22

[Andrew McCreight (out of office until 8/21) [:mccr8]]

Oops, I forgot about this bug. I should just add back in the extra shutdown GC for debug builds and hope that fixes it.

Comment 23

[Andrew McCreight (out of office until 8/21) [:mccr8]]

Attachment: Bug 1340425 - Add back the magic GC in DoShutdown() to avoid intermittent leaks.

Review commit: https://reviewboard.mozilla.org/r/171778/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/171778/

Comment 24

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 8900428 [details]
Bug 1340425 - Add back the magic GC in DoShutdown() to avoid intermittent leaks.

https://reviewboard.mozilla.org/r/171778/#review177018

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b4f88e70e596
Add back the magic GC in DoShutdown() to avoid intermittent leaks. r=smaug

Comment 26

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/b4f88e70e596

Comment 27

[Andrew McCreight (out of office until 8/21) [:mccr8]]

No occurrences of this failure since I landed my patch.

Comment 28

[Andrew McCreight (out of office until 8/21) [:mccr8]]

Comment on attachment 8900428 [details]
Bug 1340425 - Add back the magic GC in DoShutdown() to avoid intermittent leaks.

Approval Request Comment
[Feature/Bug causing the regression]: bug 833143
[User impact if declined]: None. This just fixes an intermittent failure on Treeherder.
[Is this code covered by automated tests?]: Yes.
[Has the fix been verified in Nightly?]: Yes.
[Needs manual test from QE? If yes, steps to reproduce]: No.
[List of other uplifts needed for the feature/fix]: None.
[Is the change risky?]: No.
[Why is the change risky/not risky?]: This only affects debug builds.
[String changes made/needed]: None.

Comment 29

[Intermittent Failures Robot]

13 failures in 908 pushes (0.014 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 6
* mozilla-inbound: 4
* try: 2
* mozilla-central: 1

Platform breakdown:
* linux32: 7
* osx-10-10: 3
* linux64: 2
* linux64-ccov: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-08-21&endday=2017-08-27&tree=all

Comment 30

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8900428 [details]
Bug 1340425 - Add back the magic GC in DoShutdown() to avoid intermittent leaks.

Fix for failing tests on debug builds, avoids a leak, please uplift for beta 7.

Comment 31

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/cb236dfdd7bb

Comment 32

[Andrei Vaida [:avaida]]

(In reply to Andrew McCreight [:mccr8] from comment #28)
> [Is this code covered by automated tests?]: Yes.
> [Has the fix been verified in Nightly?]: Yes.
> [Needs manual test from QE? If yes, steps to reproduce]: No.

Setting qe-verify- based on Andrew's assessment on manual testing needs and the fact that this fix has automated coverage.

Comment 33

[Intermittent Failures Robot]

1 failures in 924 pushes (0.001 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-beta: 1

Platform breakdown:
* windows8-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-09-04&endday=2017-09-10&tree=all

Comment 34

[Intermittent Failures Robot]

3 failures in 1032 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 3

Platform breakdown:
* windows7-32: 1
* linux64-stylo-disabled: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-09-11&endday=2017-09-17&tree=all

Comment 35

[Intermittent Failures Robot]

1 failures in 824 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-10-02&endday=2017-10-08&tree=all

Comment 36

[Intermittent Failures Robot]

1 failures in 947 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-10-09&endday=2017-10-15&tree=all

Comment 37

[Intermittent Failures Robot]

1 failures in 864 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-10-16&endday=2017-10-22&tree=all

Comment 38

[Intermittent Failures Robot]

1 failures in 857 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 1

Platform breakdown:
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-10-30&endday=2017-11-05&tree=all

Comment 39

[Intermittent Failures Robot]

1 failures in 744 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-release: 1

Platform breakdown:
* linux32-stylo-disabled: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-11-20&endday=2017-11-26&tree=all

Comment 40

[Intermittent Failures Robot]

1 failures in 792 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-release: 1

Platform breakdown:
* linux64-stylo-disabled: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-11-27&endday=2017-12-03&tree=all

Comment 41

[Intermittent Failures Robot]

1 failures in 423 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-release: 1

Platform breakdown:
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-12-11&endday=2017-12-17&tree=all

Comment 42

[Intermittent Failures Robot]

2 failures in 147 pushes (0.014 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-release: 2

Platform breakdown:
* linux64-stylo-disabled: 1
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2017-12-25&endday=2017-12-31&tree=all

Comment 43

[Intermittent Failures Robot]

2 failures in 788 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-release: 2

Platform breakdown:
* linux32-stylo-disabled: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1340425&startday=2018-01-08&endday=2018-01-14&tree=all

Comment 44

[BugBot [:suhaib / :marco/ :calixte]]

The leave-open keyword is there and there is no activity for 6 months.
:mccr8, maybe it's time to close this bug?