927126 - Expand automated coverage around editing of gallery photos

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Jason Smith [:jsmith]]

Editing of gallery photos was a big hot spot for regressions in the 1.2 timeframe. We should explore resulting regressions that happened to figure out what automated coverage we should expand on. Should include running this test across multiple photos? Should we explore using more edit options? These are such questions we need to answer here. Then, implement the solution that will cover us against regressions we saw in the 1.2 timeframe.

Comment 1

[Zac C (:zac)]

This is very similar to bug 927126.

Let's seed the gmail account in the testvars file with one or two contacts.


Test case as follows:
1. Start b2g
2. Tap 'Settings'
3. Tap 'Import Contacts'
4. Tap 'Gmail'
5. Sign in to Gmail
6. After import process, verify list of contacts is shown.
7. Select the contact and tap 'Import'
8. Verify that the imported contact(s) are listed on the Contacts app main screen.

Comment 2

[Zac C (:zac)]

Ignore the comment #1, wrong bug.

Comment 3

[Zac C (:zac)]

We have a fair bit of good coverage here and have caught lots of bugs here before. 
I'll try and come up with some more test cases in this area but nothing immediately comes to mind!

Comment 4

[Jason Smith [:jsmith]]

FWIW - gallery regression bugs motivating the filing of this bug are:

- bug 915876 (caught by automation)
- bug 917444 (not caught by automation)
- bug 899257 (not caught by automation)
- bug 908449 (not caught by automation)

For issues not caught by automation, trends I see are:

1. When entering crop mode in a gallery automated test, we need to verify that the photo is present and not blank
2. We need to verify that the color of the photo isn't blown away on an edit
3. Editing effects should affect the whole image, not part of it

[2] and [3] would require an ability to be able to analyze image results. [1] just requires verifying the present of the image on the crop page.

Comment 5

[Jason Smith [:jsmith]]

Talking stephend, sounds like doing [2] & [3] won't be possible in gaia ui tests, but [1] might be.

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Jason Smith [:jsmith] from comment #5)
> Talking stephend, sounds like doing [2] & [3] won't be possible in gaia ui
> tests, but [1] might be.

We should be able to do all of these in automation. We can use Marionette to return a screenshot of the element and analyse it or compare it to a reference image. We don't have any such tests yet, and no good way to represent such failures, but that can be worked out if this is an area we need coverage in.

Comment 7

[Zac C (:zac)]

It sounds a bit edge-case and probably high maintenance, neither qualities that go well with automation.

Comment 8

[Jason Smith [:jsmith]]

(In reply to Zac C (:zac) from comment #7)
> It sounds a bit edge-case and probably high maintenance, neither qualities
> that go well with automation.

I agree on the maintenance concern on the screenshot analysis approach. However, I don't think that causes [1] to be out of the picture - [1] indicates that we should be checking for presence of the picture being cropped, not analyze the picture itself.

Comment 9

[Zac C (:zac)]

Yes that might be possible. Let's treat this as an investigation task.

The image element's size may have a before/after states that represent whether it has been resized properly or not.

If we can do that we'll break out test_gallery_edit_photo.py into two tests:
1. test_gallery_crop_photo.py - Crops and asserts the size has changed
2. test_gallery_edit_photo.py - Goes through the options as now and looks for a graphics crash

Comment 10

[Trif Andrei-Alin[:AlinT]]

:jsmith :zac
I've investigated this issue today and it seems everything in the edit image frame is done in canvas( which we cannot query) so the only thing we could check would be the size of image before and after the croping.

Comment 11

[Zac C (:zac)]

To clarify an IRC conversation, by size I was talking about the image's x/y size rather than its kb size.

Comment 12

[Trif Andrei-Alin[:AlinT]]

Attachment: Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13182#attch-to-bugzilla

Pointer to Github pull-request

Comment 13

[Zac C (:zac)]

https://github.com/mozilla-b2g/gaia/commit/903fb4005ce7ac3047dc03363505082f739d4fe4

Comment 14

[Bob Silverberg [:bsilverberg]]

Attachment: Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13329

Pointer to Github pull-request

Comment 15

[Bob Silverberg [:bsilverberg]]

Comment on attachment 826447 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13329

Uplifting to v1.2

Comment 16

[[:AndreiH]]

Comment on attachment 826447 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/13329

LGTM

Comment 17

[[:AndreiH]]

https://github.com/mozilla-b2g/gaia/commit/e977d9999e00388a5f059005871206e095253ac0