1436810 - browser/components/originattributes/test/browser/browser_cache.js perma leaks windows on a try push when only arbitrarily changed

Status: RESOLVED DUPLICATE of bug 1330046

(Core :: DOM: Security, defect)

Description

[Honza Bambas (:mayhemer)]

STR:
- make a trivial change to that test like [1]
- push to try (empty) and trigger debug tier 2[TV] test, like [2]

Result:
the test result log claims on every retrigger there is a domwindow leak caused by the browser_cache.js test ; it seems platform independent

[1] https://hg.mozilla.org/try/rev/84979b843d45519868f702a854ee5f03d13c4449
[2] https://treeherder.mozilla.org/#/jobs?repo=try&revision=99cd8c49a350e8b27f560a16f7e8242aaed4f60c&filter-searchStr=tv

Comment 1

[Ed Morley [:emorley]]

This isn't related to Treeherder (it doesn't run the jobs themselves, determine the result, nor is it involved with the test harnesses). Guessing at the correct component :-)

Comment 2

[Ryan VanderMeulen [:RyanVM]]

Test-verify is a job that explicitly stress-tests the changed tests in a given push. Read the post below:
https://groups.google.com/d/msg/mozilla.dev.platform/sf3SjDAKL60/Cs0ZJPBECAAJ

If you have any more questions on how it works, Geoff can probably help more than I can.

Comment 3

[Honza Bambas (:mayhemer)]

Thanks!  Could it then be that bug 1330046 is a perma failure that shows only when we either clobber or touch the file?

Comment 4

[Geoff Brown [:gbrown]]

Yes! test-verify will only verify a test when it is modified.

Comment 5

[Geoff Brown [:gbrown]]

I notice that this seems to fail on "step 2" of verification:

[task 2018-02-08T19:25:37.935Z] 19:25:37     INFO - :::
[task 2018-02-08T19:25:37.936Z] 19:25:37     INFO - ::: Test verification summary for:
[task 2018-02-08T19:25:37.936Z] 19:25:37     INFO - :::
[task 2018-02-08T19:25:37.937Z] 19:25:37     INFO - ::: browser/components/originattributes/test/browser/browser_cache.js
[task 2018-02-08T19:25:37.938Z] 19:25:37     INFO - :::
[task 2018-02-08T19:25:37.939Z] 19:25:37     INFO - ::: 1. Run each test 10 times in one browser. : Pass
[task 2018-02-08T19:25:37.940Z] 19:25:37     INFO - ::: 2. Run each test 5 times in a new browser each time. : FAIL
[task 2018-02-08T19:25:37.940Z] 19:25:37     INFO - ::: 3. Run each test 10 times in one browser, in chaos mode. : not run / incomplete
[task 2018-02-08T19:25:37.941Z] 19:25:37     INFO - ::: 4. Run each test 5 times in a new browser each time, in chaos mode. : not run / incomplete
[task 2018-02-08T19:25:37.941Z] 19:25:37     INFO - :::
[task 2018-02-08T19:25:37.942Z] 19:25:37     INFO - ::: Test verification FAILED!
[task 2018-02-08T19:25:37.942Z] 19:25:37     INFO - :::

That is roughly equivalent to:

for i = 1 to 5:
    mach mochitest browser/components/originattributes/test/browser/browser_cache.js

In a case like this where a leak is seen in test-verify but not in "normal" test runs in continuous integration, usually the important difference is that the test is run by itself, rather than as part of a larger test set. Maybe a later test is cleaning up something for browser_cache.js, or maybe something just needs time to run after browser_cache.js but before the browser is shutdown.

At any rate, remember that this is a tier 2 test, and the fault may have nothing to do with changes you have made. It would be great if you followed up and got this passing in test-verify, but it's certainly not required.

Comment 6

[Honza Bambas (:mayhemer)]

(In reply to Geoff Brown [:gbrown] from comment #5)

thanks for confirming this.

> It would be great if you followed
> up and got this passing in test-verify, but it's certainly not required.

I did spend some time on this (maybe too much) w/o any results..


Anyway, what should we do with this bug?  WONTFIX?  INVALID?

Let you know that (sorry for my possible ignorance here) this really took some of my time until I found this was actually only intermittent.  Same trap for Natalia in bug 1152332 again.

Comment 7

[Daniel Veditz [:dveditz]]

looks like bug 1330046 is still open and covers this test causing leaks.