1175110 - Disable SMS Gip tests that have been migrated to Gij

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Johan Lorenzo [:jlorenzo]]

Once every test is migrated in Gij, we can delete them from Gip.

Comment 1

[Johan Lorenzo [:jlorenzo]]

I meant disable them from Gip.  We still need them to run on device.

Comment 2

[Johan Lorenzo [:jlorenzo]]

Attachment: Gaia PR

Like mentioned in the spreadsheet [1], test_mms_add_subject.py and test_sms_auto_save_draft.py have equivalents, respectively [2] and [3].

Also, test_add_to_new_contact_from_messages.py and test_add_to_an_existing_contact_from_messages.py are not running on b2gdesktop because they send actual SMS (which is not needed for the test).

Apart from that, each test tracked in one of the bug-dependencies of this current bug, gets deactivated.

[1] https://docs.google.com/spreadsheets/d/19Cl5OaLSiynwW5NItjPuOQYpKbEKG_2eneiuwqm3Gew/edit#gid=0
[1] https://github.com/mozilla-b2g/gaia/blob/master/apps/sms/test/marionette/composer_test.js#L239
[2] https://github.com/mozilla-b2g/gaia/blob/master/apps/sms/test/marionette/drafts_test.js#L32

Comment 3

[Johan Lorenzo [:jlorenzo]]

Side note: I'll land this patch once bug 1175108 gets merged.

Comment 4

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

Comment on attachment 8623037 [details] [review]
Gaia PR

LGTM. 

As mentioned to John, btw, I'm fine with disabling these from Gip (and prefer we do) as soon as a porting task is put on backlog or rejected by the destination team. Only disabling once actually ported particularly won't work with the case where the new team doesn't want to take on the maintenance.

The coverage we potentially lose temporarily on the integration side isn't terribly valuable since any python failures there inevitably just get kicked to us to triage out of band anyway.

Comment 5

[Martijn Wargers (dead)]

So I guess that means that the  skip-if = device == "desktop" is now also used for tests that actually perfectly fine to run on desktop, but are not run for other reasons.
Also, the nice thing with b2g desktop is that they are run on taskcluster, so issues with the tests or anything they catch will be catched sooner than from the Jenkins runs.

Comment 6

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

(In reply to Martijn Wargers [:mwargers] (QA) from comment #5)
> So I guess that means that the  skip-if = device == "desktop" is now also
> used for tests that actually perfectly fine to run on desktop, but are not
> run for other reasons.
> Also, the nice thing with b2g desktop is that they are run on taskcluster,
> so issues with the tests or anything they catch will be catched sooner than
> from the Jenkins runs.

Rolling up from vidyo convo, yeah, let's use skip-if for this. I'd prefer a solid flag, but in the absence of one this is what we've got.

Re: running on taskcluster, the issue is that things that are found then aren't generally actionable at the time they're found. That makes the test results obstructive for an integration process without adding much real value to balance the hassle.

Comment 7

[Johan Lorenzo [:jlorenzo]]

Based on comment 4, landed in master at: https://github.com/mozilla-b2g/gaia/commit/eefa5fdaa92f5f83ec3a59f77a0e587406396959