1258897 - Intermittent test_ext_sendmessage_reply2.html | Test timed out. | Extension left running at test shutdown

Status: RESOLVED FIXED

(WebExtensions :: General, defect, P2)

Description

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://treeherder.mozilla.org/logviewer.html#?job_id=24358392&repo=mozilla-inbound

Comment 2

[BMO Automation]

Bulk assigning P3 to all open intermittent bugs without a priority set in Firefox components per bug 1298978.

Comment 3

[Andy McKay]

This intermittent has not occurred for over 180 days, so closing.

https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2016-11-20&endday=2017-05-19&tree=all

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://treeherder.mozilla.org/logviewer.html#?job_id=119257849&repo=autoland

Comment 5

[Intermittent Failures Robot]

10 failures in 1008 pushes (0.01 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* autoland: 7
* mozilla-inbound: 2
* mozilla-central: 1

Platform breakdown:
* windows7-32: 3
* osx-10-10: 3
* linux64: 3
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2017-07-24&endday=2017-07-30&tree=all

Comment 6

[Intermittent Failures Robot]

41 failures in 888 pushes (0.046 failures/push) were associated with this bug in the last 7 days. 

This is the #42 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 23
* mozilla-inbound: 14
* mozilla-beta: 2
* try: 1
* mozilla-central: 1

Platform breakdown:
* windows7-32: 20
* osx-10-10: 9
* windows8-64: 4
* linux64: 3
* linux32: 3
* windows7-32-nightly: 1
* macosx64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2017-07-31&endday=2017-08-06&tree=all

Comment 7

[Intermittent Failures Robot]

16 failures in 182 pushes (0.088 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 5
* mozilla-beta: 4
* mozilla-inbound: 3
* try: 2
* mozilla-central: 2

Platform breakdown:
* windows7-32: 4
* windows8-64: 3
* linux32: 3
* windows7-32-nightly: 2
* macosx64-nightly: 2
* osx-10-10: 1
* linux32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2017-08-08&endday=2017-08-08&tree=all

Comment 8

[Intermittent Failures Robot]

18 failures in 179 pushes (0.101 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 14
* mozilla-inbound: 2
* try: 1
* mozilla-beta: 1

Platform breakdown:
* windows7-32: 4
* windows7-32-stylo: 3
* linux64-stylo-sequential: 3
* windows10-64-stylo: 2
* osx-10-10: 2
* linux32: 2
* windows10-64: 1
* macosx64-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2017-08-10&endday=2017-08-10&tree=all

Comment 9

[Intermittent Failures Robot]

73 failures in 901 pushes (0.081 failures/push) were associated with this bug in the last 7 days. 

This is the #14 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 36
* mozilla-inbound: 12
* mozilla-central: 10
* mozilla-beta: 8
* try: 7

Platform breakdown:
* windows7-32: 21
* linux32: 11
* windows7-32-stylo: 7
* windows10-64: 5
* osx-10-10: 5
* linux64-stylo-sequential: 5
* windows8-64: 4
* windows10-64-stylo: 4
* macosx64-nightly: 4
* windows7-32-nightly: 3
* linux64: 2
* windows7-32-devedition: 1
* linux32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2017-08-07&endday=2017-08-13&tree=all

Comment 10

[Geoff Brown [:gbrown]]

This long-dormant failure re-started around July 28 and has been reported frequently since then. It seems like all platforms are affected, but Windows 7 is worst.

:andym - Can you find someone to investigate this frequent intermittent test failure?

Comment 11

[Intermittent Failures Robot]

23 failures in 149 pushes (0.154 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 13
* mozilla-beta: 7
* mozilla-inbound: 3

Platform breakdown:
* windows7-32-stylo: 4
* windows7-32: 3
* osx-10-10: 3
* macosx64-nightly: 3
* linux64-stylo-sequential: 2
* linux64: 2
* windows7-32-nightly: 1
* windows10-64: 1
* macosx64-devedition: 1
* linux64-stylo: 1
* linux32-devedition: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2017-08-14&endday=2017-08-14&tree=all

Comment 12

[Intermittent Failures Robot]

17 failures in 188 pushes (0.09 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 7
* mozilla-central: 4
* mozilla-inbound: 3
* mozilla-beta: 2
* try: 1

Platform breakdown:
* osx-10-10: 3
* windows7-32-stylo: 2
* windows10-64: 2
* macosx64-nightly: 2
* macosx64-devedition: 2
* linux64-stylo: 2
* windows10-64-stylo: 1
* macosx64-stylo: 1
* linux64: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2017-08-16&endday=2017-08-16&tree=all

Comment 13

[Intermittent Failures Robot]

19 failures in 194 pushes (0.098 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 5
* try: 4
* mozilla-central: 4
* mozilla-inbound: 3
* mozilla-beta: 3

Platform breakdown:
* osx-10-10: 4
* linux64-stylo: 4
* windows7-32: 2
* macosx64-stylo: 2
* windows10-64-stylo: 1
* windows10-64-devedition: 1
* macosx64-nightly: 1
* macosx64-devedition: 1
* linux64-stylo-sequential: 1
* linux64: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2017-08-17&endday=2017-08-17&tree=all

Comment 14

[Intermittent Failures Robot]

95 failures in 949 pushes (0.1 failures/push) were associated with this bug in the last 7 days. 

This is the #14 most frequent failure this week. 

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 44
* mozilla-inbound: 20
* mozilla-beta: 12
* mozilla-central: 11
* try: 8

Platform breakdown:
* osx-10-10: 20
* windows10-64: 12
* windows7-32: 10
* linux64-stylo: 10
* linux64-stylo-sequential: 7
* linux64: 7
* windows7-32-stylo: 6
* macosx64-nightly: 6
* macosx64-devedition: 4
* linux32: 4
* windows10-64-stylo: 3
* macosx64-stylo: 3
* windows7-32-nightly: 1
* windows10-64-devedition: 1
* linux32-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2017-08-14&endday=2017-08-20&tree=all

Comment 15

[Intermittent Failures Robot]

15 failures in 129 pushes (0.116 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 8
* mozilla-inbound: 3
* mozilla-central: 2
* try: 1
* mozilla-beta: 1

Platform breakdown:
* osx-10-10: 6
* windows7-32: 2
* linux64-stylo-sequential: 2
* windows10-64-stylo: 1
* macosx64-nightly: 1
* linux64-stylo: 1
* linux64: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2017-08-21&endday=2017-08-21&tree=all

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/f28adccedcd2
disable test_ext_sendmessage_reply2.html. r=me a=testonly

Comment 17

[Joel Maher ( :jmaher ) (UTC -8)]

leaving this open, please consider looking into the failure and fixing this in the near future :)

Comment 18

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/0ff7ea250787

Comment 19

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/f28adccedcd2

Comment 20

[Intermittent Failures Robot]

30 failures in 908 pushes (0.033 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 17
* try: 4
* mozilla-inbound: 3
* mozilla-central: 3
* mozilla-beta: 3

Platform breakdown:
* osx-10-10: 7
* linux64-stylo-sequential: 5
* windows7-32-stylo: 3
* windows7-32: 3
* linux64-stylo: 3
* linux64: 2
* linux32: 2
* windows7-32-devedition: 1
* windows10-64-stylo: 1
* windows10-64: 1
* macosx64-nightly: 1
* linux32-nightly: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1258897&startday=2017-08-21&endday=2017-08-27&tree=all

Comment 21

[Bob Silverberg [:bsilverberg]]

This is skipped on all platforms, so should be treated as a high priority to re-enable. Marking as P2.

Comment 22

[Luca Greco [:rpl] [:luca] [:lgreco]]

Attachment: Bug 1258897 - Fix test_ext_sendmessage_reply2.html intermittency and re-enable. r?mixedpuppy!

This patch should fix the test_ext_sendmessage_reply2.html intermittency which, based on the
timeout failure that I've been able to intermittently reproduce when running this test
locally using --verify, seems to be likely due to the following race:

the tabs.create API (used to create the extension tabs used as part of the tests) was called
right before subscribing the extension message listeners, and so from time to time the
extension tabs were not able to successfully send their messages to the background page listeners
(because the background page didn't subscribe the listeners yet) and as a result the test was getting
stuck waiting for extension messages that were never going to be received.

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e5fafbd5103b
Fix test_ext_sendmessage_reply2.html intermittency and re-enable. r=mixedpuppy

Comment 24

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/e5fafbd5103b

Comment 25

[Luca Greco [:rpl] [:luca] [:lgreco]]

Attachment: Bug 1258897 - Ensure that both test_ext_sendmessage_reply2 tab scripts are ready to exchange messages. r?mixedpuppy!

This test seems to be still failing intermittently, and its intermittency rate looks high enough.

Looking to the logs collected on the recent intermittent failures, it seems that the issue
is still related to the extension tabs opened in the test:

The two extension tabs (each one part of one of the two test extension) are going to exchange messages
between each other, and currently there is a chance (and apparently a very good chance) that one of the
two extension tabs is not yet ready to listen to the messages sent from the other extension tab.

This patch applies to the test the additional changes needed to ensure that both the extension tabs
are ready to exchange messages, before we let them exchange these messages over the extension messaging
API.

Comment 26

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c127e67367b9
Ensure that both test_ext_sendmessage_reply2 tab scripts are ready to exchange messages. r=mixedpuppy

Comment 27

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/c127e67367b9

Comment 28

[Intermittent Failures Robot]

17 failures in 517 pushes (0.033 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 6
• autoland: 6
• mozilla-central: 1
• mozilla-inbound: 4

Platform breakdown:

• windows7-32: 1
• linux64: 4
• linux64-qr: 5
• windows10-64: 2
• windows10-64-qr: 2
• osx-10-10: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1258897&startday=2019-01-31&endday=2019-01-31&tree=all

Comment 29

[Bogdan Tara[:bogdan_tara | bogdant]]

This is still intermittently failing: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=linux%2Cdebug%2Cmochitests%2Cwith%2Ce10s%2Ctest-linux32%2Fdebug-mochitest-e10s-15%2Cm-e10s%2815%29&fromchange=59b627e138589a04d58c93185f41ea4c65316770&tochange=37103bfa3dca5149f414cc6265d24fd752a4385f&selectedJob=224966411

The failure on the landed patch: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=224966411&repo=autoland&lineNumber=12820

Comment 30

[Andreea Pavel [:apavel]]

There are 19 total failures in the last 7 days.

Luca can you please take another look at this?
Most recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=225342016&repo=mozilla-inbound&lineNumber=28138

Comment 31

[Intermittent Failures Robot]

26 failures in 2628 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 12
• mozilla-central: 1
• try: 7
• mozilla-inbound: 6

Platform breakdown:

• windows10-64: 2
• linux32: 1
• osx-10-10: 9
• windows10-64-qr: 2
• linux64: 5
• linux64-qr: 5
• windows7-32: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1258897&startday=2019-01-28&endday=2019-02-03&tree=all

Comment 32

[Luca Greco [:rpl] [:luca] [:lgreco]]

(In reply to Andreea Pavel [:apavel] from comment #30)

There are 19 total failures in the last 7 days.

Luca can you please take another look at this?
Most recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=225342016&repo=mozilla-inbound&lineNumber=28138

Hi Andreea,
I was already keeping an eye on this test after we landed the first patch that has re-enabled it (linked in comment 24) and, as I mentioned in comment 25, I noticed that the test was still failing intermittently and its rate was clearly increasing and so I took care of it right away.

The patch attached in comment 25 has been already landed (comment 27) and looking at the failure we got since then:

• https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-01-27&endday=2019-02-03&tree=trunk&bug=1258897
• https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1258897&startday=2019-01-28&endday=2019-02-03&tree=all

the last patch seems to have been able to prevent further failures from being triggered (and the last hg revisions with a failure, the one linked in comment 30 and the last failure happened on try on Feb 2, do not include the changes from the last patch).

Comment 33

[Andreea Pavel [:apavel]]

Luca, thanks for clarifying. I saw you landed the patch, however as Bogdan mentioned, there still was a failure on the push with your patch, thus I wanted to clarify this with you.

Looking in orange factor now, i see that there are no recent failures https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-01-29&endday=2019-02-05&tree=trunk&bug=1258897 so indeed this seems to be fixed.

Comment 34

[Luca Greco [:rpl] [:luca] [:lgreco]]

Closing as fixed (as mentioned in Comment 32 and Comment 33 there are no further failure from this intermittent starting from Feb 2 2019)