Disable SMS Gip tests that have been migrated to Gij

RESOLVED FIXED

Status

defect
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: jlorenzo, Assigned: jlorenzo)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

46 bytes, text/x-github-pull-request
gmealer
: review+
martijn.martijn
: review+
Details | Review
Assignee

Description

4 years ago
Once every test is migrated in Gij, we can delete them from Gip.
Assignee

Comment 1

4 years ago
I meant disable them from Gip.  We still need them to run on device.
Summary: Remove SMS Gip tests that have been migrated to Gij → Disable SMS Gip tests that have been migrated to Gij
Assignee

Updated

4 years ago
QA Whiteboard: [fxosqa-auto-s19]
Assignee

Comment 2

4 years ago
Posted file Gaia PR
Like mentioned in the spreadsheet [1], test_mms_add_subject.py and test_sms_auto_save_draft.py have equivalents, respectively [2] and [3].

Also, test_add_to_new_contact_from_messages.py and test_add_to_an_existing_contact_from_messages.py are not running on b2gdesktop because they send actual SMS (which is not needed for the test).

Apart from that, each test tracked in one of the bug-dependencies of this current bug, gets deactivated.

[1] https://docs.google.com/spreadsheets/d/19Cl5OaLSiynwW5NItjPuOQYpKbEKG_2eneiuwqm3Gew/edit#gid=0
[1] https://github.com/mozilla-b2g/gaia/blob/master/apps/sms/test/marionette/composer_test.js#L239
[2] https://github.com/mozilla-b2g/gaia/blob/master/apps/sms/test/marionette/drafts_test.js#L32
Attachment #8623037 - Flags: review?(martijn.martijn)
Attachment #8623037 - Flags: review?(gmealer)
Assignee

Comment 3

4 years ago
Side note: I'll land this patch once bug 1175108 gets merged.
Blocks: 1146997
Comment on attachment 8623037 [details] [review]
Gaia PR

LGTM. 

As mentioned to John, btw, I'm fine with disabling these from Gip (and prefer we do) as soon as a porting task is put on backlog or rejected by the destination team. Only disabling once actually ported particularly won't work with the case where the new team doesn't want to take on the maintenance.

The coverage we potentially lose temporarily on the integration side isn't terribly valuable since any python failures there inevitably just get kicked to us to triage out of band anyway.
Attachment #8623037 - Flags: review?(gmealer) → review+
Attachment #8623037 - Flags: review?(martijn.martijn) → review+
So I guess that means that the  skip-if = device == "desktop" is now also used for tests that actually perfectly fine to run on desktop, but are not run for other reasons.
Also, the nice thing with b2g desktop is that they are run on taskcluster, so issues with the tests or anything they catch will be catched sooner than from the Jenkins runs.
(In reply to Martijn Wargers [:mwargers] (QA) from comment #5)
> So I guess that means that the  skip-if = device == "desktop" is now also
> used for tests that actually perfectly fine to run on desktop, but are not
> run for other reasons.
> Also, the nice thing with b2g desktop is that they are run on taskcluster,
> so issues with the tests or anything they catch will be catched sooner than
> from the Jenkins runs.

Rolling up from vidyo convo, yeah, let's use skip-if for this. I'd prefer a solid flag, but in the absence of one this is what we've got.

Re: running on taskcluster, the issue is that things that are found then aren't generally actionable at the time they're found. That makes the test results obstructive for an integration process without adding much real value to balance the hassle.
Assignee

Updated

4 years ago
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.