1523973 - Review and update the raptor-tp6m-1 recordings

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P1)

Description

[Florin Strugariu [:Bebe]]

I ran all the tests from raptor-tp6m-1 today and saw that the websites are not displaying correctly

Here is a link to the screenshot report:

https://screenshots.firefox.com/ArU9X91TVwcWc8mO/null

And here is a link with some screenshots made to the same websites with the ghecko view app

https://photos.app.goo.gl/15dDPBGACjgh1qdY9

As you can see there are big differences between the two.

Do we have issues with the recordings?

Tested on Pixel 2 and gecko_example 20190129134346

Comment 1

[Robert Wood [:rwood]]

Attachment: raptor tp6m-1 gp2 screenshots

When I run Raptor tp6m-1 on my Google Pixel 2 locally (OSX) it looks fine - see attached screenshots from the local test run - however note I am running quite an old geckoview_example app (20181205191909).

Maybe something has changed in the app that makes the recordings not render properly - I'll try upgrading my geckoview_example app and see if that makes a difference.

Comment 2

[Florin Strugariu [:Bebe]]

here are new recordings

https://drive.google.com/drive/folders/1GgbvSiXDC3F3INwQg5RWyUoEdSNkFSO8?usp=sharing

:rwood davehunt can you download and test these on your devices?

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Florin Strugariu [:Bebe] from comment #2)

here are new recordings

https://drive.google.com/drive/folders/1GgbvSiXDC3F3INwQg5RWyUoEdSNkFSO8?usp=sharing

:rwood davehunt can you download and test these on your devices?

I see issues with these recordings. I think we should block on bug 1525280.

Comment 4

[Florin Strugariu [:Bebe]]

Updated the recordings for amazon google and YouTube

See the recodings in the gdrive https://drive.google.com/drive/folders/1GgbvSiXDC3F3INwQg5RWyUoEdSNkFSO8?usp=sharing

For Facebook we have a empty test account. Do we want data in it?

Comment 5

[Florin Strugariu [:Bebe]]

Updated and ready for review/upload to tooltool

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

These all look great to me. I downloaded them and tested locally. I had to stop the files downloaded from tooltool overriding the new recordings, but I just renamed the files so the downloaded ones were ignored.

So all we need to do is upload the new recordings to tooltool? Could you take care of that Rob?

Comment 7

[Robert Wood [:rwood]]

Uploaded to tooltool, emailed the manifests.

Comment 8

[Florin Strugariu [:Bebe]]

Attachment: Bug 1523973 Review and update the raptor-tp6m-1 recordings

Comment 9

[Florin Strugariu [:Bebe]]

Try build with screenshots
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5096c75fe17f63d83fec2b13298d0f0375a4bd71

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3924d1ec16af
Review and update the raptor-tp6m-1 recordings r=davehunt

Comment 11

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/3924d1ec16af