1236248 - Intermittent apps/sms/test/marionette/new_activity_test.js | Messages as "new" activity target Send new message Activity with thread "number"

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::SMS, defect)

Description

[Phil Ringnalda (:philor)]

https://treeherder.mozilla.org/logviewer.html#?job_id=19189635&repo=mozilla-inbound

Comment 1

[Intermittent Failures Robot]

7 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 7

Platform breakdown:
* b2g-linux64: 5
* mulet-linux64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1236248&startday=2016-01-04&endday=2016-01-10&tree=all

Comment 2

[GitHub Autolander Bot]

Attachment: [gaia] azasypkin:bug-1236248-fix-intermittent-new-activity-tests > mozilla-b2g:master

Comment 3

[Oleg Zasypkin [:azasypkin]]

Since bug 1175116 is resolved, let's see what is our real stack trace here [1]

[1] https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=95808d45f2bb03074d39fb34ed9239bdadcf556d

Comment 7

[Oleg Zasypkin [:azasypkin]]

So here is breakdown of 60 runs of Messages integration tests from the link in comment 3:

* successfully completed runs - 50

* tests/integration/sms/contacts_interaction_test failures - 5 failures;

* apps/sms/test/marionette/new_activity_test - 5 failures;

> new_activity_test:

* 4 out 5 failures look like the same as we see in bug 1233552.
* the last one is that app was too slow to even start as activity;

> contacts_interaction_test:

* 3 out of 5 failures are related to the fact that it took too long for the app to become ready (add "js-app-ready" css class), we seen similar thing in drafts tests when reading from asyncstorage was too slow - here is just guess, needs to be investigated.

* the rest are likely related to very slow test contact save functionality.

So I'll try to focus on the bug 1233552 to dig deeper.

Comment 8

[Oleg Zasypkin [:azasypkin]]

Comment on attachment 8709030 [details] [review]
[gaia] azasypkin:bug-1236248-fix-intermittent-new-activity-tests > mozilla-b2g:master

Hey Julien,

Here is the trick I've mentioned in bug 1233552 comment 24. Looks like Treeherder is quite happy (only two unrelated failures in 50+ runs):

https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=c57f6a6c35e7cc381d40dba76b94e9696c0c109d

What do you think?

Comment 9

[Julien Wajsberg [:julienw]]

Comment on attachment 8709030 [details] [review]
[gaia] azasypkin:bug-1236248-fix-intermittent-new-activity-tests > mozilla-b2g:master

I restarted the job some more.
r=me if none of new and share activity tests are failing.

Comment 10

[Oleg Zasypkin [:azasypkin]]

So here is breakdown of 134 runs with 18 failed tries (1 failed run includes from 1 to 3 failed tests):

* contacts interaction tests - 6 failed runs - we have separate bug for it;

* drafts tests - 2 failed runs with something that looks like intermittent failure + 5 failed runs with "Cu is not defined", "Error polling recv() socket!" and alike - doesn't look like something we should care about;

* new activity test - 1 failed run, marionette just wasn't able to run app at all for some reason - can be ignored;

* notification test - 1 failed run - looks like really rare intermittent and nobody complained so far.

* share activity test - 6 failed runs and none are related to the issues we are fixing here, all failures are in the same test and in the same place, so I'll just mark this single test ("Should return to Conversation panel if in Participants panel") as skipped and investigate further.

So I'm removing my temporal changes in taskcluster configs and pushing to Treeherder with 50+ runs for Gij 16 and Gij 17 - will see how it looks like.

Comment 11

[Oleg Zasypkin [:azasypkin]]

new_activity test failed only once out of 83 Gij17 runs (Marionette was not able to run the app in time), so doesn't look like something we can fix [1].

Landed in master: https://github.com/mozilla-b2g/gaia/commit/4c80dd0309b63098106ed37a5efcf2d0a8ee65e8

Let's see if it sticks well.


[1] https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=84100fc54b611182c93700bd9cb06eafc8fd0808

Comment 13

[GitHub Autolander Bot]

Attachment: [gaia] albertopq:gij11-18 > mozilla-b2g:master

Comment 14

[Oleg Zasypkin [:azasypkin]]

Hey Alberto, could you please clarify why you're going to revert this PR? :)

Thanks!

Comment 15

[Alberto Pastor [:albertopq]]

Hey! Not planning to do it! Just testing why Gij11 and Gij18 are permantly red, and this is the first commit that failed (I think). I don't think is related to the issue, but wanted to try it out.

Comment 16

[Oleg Zasypkin [:azasypkin]]

(In reply to Alberto Pastor [:albertopq] from comment #15)
> Hey! Not planning to do it! Just testing why Gij11 and Gij18 are permantly
> red, and this is the first commit that failed (I think). I don't think is
> related to the issue, but wanted to try it out.

Ah, okay, thanks for clarifying :)

Comment 17

[Julien Wajsberg [:julienw]]

That said, gij18 was very flaky for a long time !