1611758 - OSX perma fail on Thunderbird - toolkit/components/telemetry/tests/unit/test_TelemetryController.js | xpcshell return code: 0

Status: RESOLVED WORKSFORME

(Thunderbird :: General, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: mkmelin+mozilla [at] iki.fi
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=286586188&repo=comm-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AFQzIstyTtCPHsFMZ9tpJQ/runs/0/artifacts/public/logs/live_backing.log

---

Comment 1

[Magnus Melin [:mkmelin]]

Khushil tells me it's reproducible locally.

Comment 2

[Geoff Lankow (:darktrojan)]

Could we have some action on this bug please?

Comment 3

[Intermittent Failures Robot]

33 failures in 2190 pushes (0.015 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 33

Platform breakdown:

• macosx1014-64: 28
• macosx1014-64-shippable: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-01-27&endday=2020-02-02&tree=all

Comment 4

[Intermittent Failures Robot]

31 failures in 3364 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 31

Platform breakdown:

• macosx1014-64: 27
• macosx1014-64-shippable: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-02-03&endday=2020-02-09&tree=all

Comment 5

[Ben Campbell]

Picking through the raw logs, I can see it's failing at this point:
https://searchfox.org/mozilla-central/source/toolkit/components/telemetry/tests/unit/test_TelemetryController.js#286

I think it's picking up a ping left over from the block of code directly above it, where it simulates a failed connection.

The TelemetryController.testReset() call a few lines earlier looks like it should just clobber any leftover pings and set us up in a pristine test state again... but still we receive a leftover deletion-request ping rather than the test-ping-type ping we were expecting.
I'm kind of stumped - I can't figure out why this is happening only on OSX, and why it doesn't seem to be an issue in m-c...

I think the thing to do is to start peppering the codebase with trace statements to pinpoint exactly what's happening (and maybe compare to Linux/windows). If all else fails I can try this, but without a mac I'd be resorting to submitting runs to the try server, which is a pretty excruciating way to debug code...

Comment 6

[Intermittent Failures Robot]

19 failures in 4690 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 19

Platform breakdown:

• macosx1014-64-shippable: 6
• macosx1014-64: 13

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-02-10&endday=2020-02-16&tree=all

Comment 7

[Magnus Melin [:mkmelin]]

Sample log: https://firefoxci.taskcluster-artifacts.net/H-g7IBzIRiexEXmuUWTGrQ/0/public/logs/live_backing.log

task 2020-02-18T12:04:26.134Z] 12:04:26 WARNING - TEST-UNEXPECTED-FAIL | toolkit/components/telemetry/tests/unit/test_TelemetryController.js | test_disableDataUpload - [test_disableDataUpload : 83] The ping must have the correct type. - "deletion-request" == "test-ping-type"

Up until here it's ok: https://searchfox.org/mozilla-central/rev/0439db3a39faddb90197a87cc184c57dcbf0a770/toolkit/components/telemetry/tests/unit/test_TelemetryController.js#263

... but the deletion-ping left behind there is then received here
https://searchfox.org/mozilla-central/rev/0439db3a39faddb90197a87cc184c57dcbf0a770/toolkit/components/telemetry/tests/unit/test_TelemetryController.js#286

Should the Preferences.set be called before the PingServer.start() is called?
Where is the deletion-request ping supposed to go. Is it supposed to be cleared by the PingServer.start()?

---

Chris, any idea why this test isn't working on Thunderbird mac?

Comment 8

[Chris H-C :chutten]

Both pings should be sent, but the order appears to be backwards. As part of the send task we'll first send whatever pings have been submitted (in this case TEST_PING_TYPE), then any on the pending pings list (in this case "deletion-request"). This behaviour can be found here

I'm not sure what would make this platform-dependent and product-dependent. Let's look at the logs:

12:04:26.082 - test-ping-type is saved to disk because it couldn't be sent
12:04:26.085 - removes all pending pings (should include test-ping-type)
12:04:26.090 - deletion-request ping is saved to disk because it couldn't be sent
12:04:26.106 - feletion-request pending ping loaded after restart
12:04:26.126 - upload is re-enabled
12:04:26.126 - test-ping-type submitted
12:04:26.128 - pings to send: 1 current and 1 pending
12:04:26.130 - test-ping-type can't reach the PingServer <---- WAIT, What?
12:04:26.132 - deletion-request ping is successfully sent, failing the test

Well there's the problem: test-ping-type failed to be received, so it doesn't get you past line 286. So when the later deletion-request ping is received it fails the checks.

I'm not sure why it wasn't received. Maybe PingServer isn't started? Maybe the port is still closed? Do you have other theories?

Comment 9

[Magnus Melin [:mkmelin]]

Thanks! Not too many theories atm.

Khushil, please try adding Assert.ok(PingServer.started); here:
https://searchfox.org/mozilla-central/rev/0439db3a39faddb90197a87cc184c57dcbf0a770/toolkit/components/telemetry/tests/unit/test_TelemetryController.js#280

Comment 10

[Khushil Mistry [:khushil324]]

It is passing i.e PingServer.started is true when we reach there.

Comment 11

[Intermittent Failures Robot]

24 failures in 4045 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

This is the #45 most frequent failure this week.

Repository breakdown:

• comm-central: 24

Platform breakdown:

• macosx1014-64-shippable: 6
• macosx1014-64: 18

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-02-17&endday=2020-02-23&tree=all

Comment 12

[Intermittent Failures Robot]

20 failures in 5233 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 20

Platform breakdown:

• macosx1014-64: 17
• macosx1014-64-shippable: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-02-24&endday=2020-03-01&tree=all

Comment 13

[Intermittent Failures Robot]

34 failures in 5483 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

This is the #35 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 34

Platform breakdown:

• macosx1014-64: 28
• macosx1014-64-shippable: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-03-02&endday=2020-03-08&tree=all

Comment 14

[Intermittent Failures Robot]

37 failures in 5336 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

This is the #45 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 37

Platform breakdown:

• macosx1014-64: 32
• macosx1014-64-shippable: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-03-09&endday=2020-03-15&tree=all

Comment 15

[Intermittent Failures Robot]

38 failures in 4955 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

This is the #41 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 38

Platform breakdown:

• macosx1014-64: 32
• macosx1014-64-shippable: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-03-16&endday=2020-03-22&tree=all

Comment 16

[Intermittent Failures Robot]

45 failures in 4773 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #22 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 44
• try-comm-central: 1

Platform breakdown:

• macosx1014-64-shippable: 5
• macosx1014-64: 40

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-03-23&endday=2020-03-29&tree=all

Comment 17

[Intermittent Failures Robot]

40 failures in 5015 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

This is the #21 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• try-comm-central: 1
• comm-central: 38
• comm-beta: 1

Platform breakdown:

• macosx1014-64: 33
• macosx1014-64-shippable: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-03-30&endday=2020-04-05&tree=all

Comment 18

[Intermittent Failures Robot]

32 failures in 4823 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

This is the #41 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 32

Platform breakdown:

• macosx1014-64: 28
• macosx1014-64-shippable: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-04-06&endday=2020-04-12&tree=all

Comment 19

[Intermittent Failures Robot]

31 failures in 4528 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 31

Platform breakdown:

• macosx1014-64: 24
• macosx1014-64-shippable: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-04-13&endday=2020-04-19&tree=all

Comment 20

[Intermittent Failures Robot]

10 failures in 5582 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 10

Platform breakdown:

• macosx1014-64: 8
• macosx1014-64-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-04-20&endday=2020-04-26&tree=all

Comment 21

[Intermittent Failures Robot]

1 failures in 5276 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-05-04&endday=2020-05-10&tree=all

Comment 22

[Intermittent Failures Robot]

13 failures in 5458 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 13

Platform breakdown:

• macosx1014-64: 8
• macosx1014-64-shippable: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-05-25&endday=2020-05-31&tree=all

Comment 23

[Intermittent Failures Robot]

27 failures in 4949 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 25
• try-comm-central: 2

Platform breakdown:

• macosx1014-64: 15
• macosx1014-64-shippable: 12

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-06-01&endday=2020-06-07&tree=all

Comment 24

[Intermittent Failures Robot]

51 failures in 5211 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This is the #25 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 51

Platform breakdown:

• macosx1014-64: 30
• macosx1014-64-shippable: 21

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-06-08&endday=2020-06-14&tree=all

Comment 25

[Intermittent Failures Robot]

17 failures in 872 pushes (0.019 failures/push) were associated with this bug yesterday.

Repository breakdown:

• comm-central: 17

Platform breakdown:

• macosx1014-64: 11
• macosx1014-64-shippable: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-06-18&endday=2020-06-18&tree=all

Comment 26

[Intermittent Failures Robot]

43 failures in 4035 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #33 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 43

Platform breakdown:

• macosx1014-64: 26
• macosx1014-64-shippable: 17

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-06-15&endday=2020-06-21&tree=all

Comment 27

[Intermittent Failures Robot]

30 failures in 5252 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 29
• try-comm-central: 1

Platform breakdown:

• macosx1014-64: 22
• macosx1014-64-shippable: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-06-22&endday=2020-06-28&tree=all

Comment 28

[Intermittent Failures Robot]

14 failures in 4516 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 14

Platform breakdown:

• macosx1014-64: 14

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-06-29&endday=2020-07-05&tree=all

Comment 29

[Intermittent Failures Robot]

10 failures in 5265 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 10

Platform breakdown:

• macosx1014-64: 9
• macosx1014-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-07-06&endday=2020-07-12&tree=all

Comment 30

[Intermittent Failures Robot]

38 failures in 4014 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #28 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 38

Platform breakdown:

• macosx1014-64-shippable: 14
• macosx1014-64: 24

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-07-13&endday=2020-07-19&tree=all

Comment 31

[Intermittent Failures Robot]

28 failures in 4310 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

This is the #48 most frequent failure this week.

Repository breakdown:

• comm-central: 25
• autoland: 2
• try-comm-central: 1

Platform breakdown:

• macosx1014-64-shippable: 14
• macosx1014-64: 14

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-07-20&endday=2020-07-26&tree=all

Comment 32

[Intermittent Failures Robot]

37 failures in 4289 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #32 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• comm-central: 37

Platform breakdown:

• macosx1014-64: 17
• macosx1014-64-shippable: 20

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-07-27&endday=2020-08-02&tree=all

Comment 33

[Intermittent Failures Robot]

20 failures in 4618 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 20

Platform breakdown:

• macosx1014-64: 10
• macosx1014-64-shippable: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-08-03&endday=2020-08-09&tree=all

Comment 34

[Intermittent Failures Robot]

12 failures in 3601 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• comm-central: 12

Platform breakdown:

• macosx1014-64: 6
• macosx1014-64-shippable: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1611758&startday=2020-08-10&endday=2020-08-16&tree=all

Comment 35

[Magnus Melin [:mkmelin]]

This disappeared.

Comment 36

[Rob Lemley [:rjl]]

It's showing up on c-esr78 on every release now: https://treeherder.mozilla.org/jobs?repo=comm-esr78&selectedTaskRun=EgmiY4CHQE2Z498hz3n4bw.0

Apparently something needs uplifting?

Comment 37

[Magnus Melin [:mkmelin]]

Seems OSX only, like it use to be.
We couldn't fix it earlier, so unlikely to be able to fix it now. But it should be just a test failure, so not too bad either. I guess we could disable that test for esr 78 if you don't want to see the yellow. r=mkmelin for that