1558764 - Log timestamp from page load recordings

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P2)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

We'd like to increase the visibility of when page recordings are updated. A first step would be to extract and log the timestamp from the recording. For mitmproxy, the recordings have many matches for 'timestamp' so we should just find an appropriate and consistently available match to convert to date/time for logging.

Comment 1

[Florin Strugariu [:Bebe]]

This will require some changes in different places:

1. Raptor studio:

Update raptor studio record plugin to save the current date in the json file it creates at record time.
This is added in pull:
https://github.com/mozilla/raptor-studio/pull/16
See:
https://github.com/mozilla/raptor-studio/pull/16/files#diff-61ec0cd7c664286456ae66ff26aa7e9b

the format of the json should be:

{
  "record_date": "",
  "protocol": {
    "detectportal.firefox.com": "HTTP/1.1"
    }
}

2. Update Alternate server replay to read the new json format
3. In Raptor before starting the proxy check if the Json file is available and output the date the recording was made in the logs.

This will pave the way for accessing other information about the recordings

Comment 2

[Arnold Iakab]

Attachment: Bug 1558764 Log timestamp from page load recordings

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/405087ccd676
Log timestamp from page load recordings r=perftest-reviewers,Bebe

Comment 4

[Alexandru Michis [:malexandru]]

Attachment: Bug 1558764 - Fix linting failure. r=Bebe

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b9243653712d
Fix linting failure. r=Bebe CLOSED TREE

Comment 6

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/405087ccd676
https://hg.mozilla.org/mozilla-central/rev/b9243653712d

Comment 7

[Mike Trinkala [:trink]]

The only message I have seen in the logs doing a quick manual inspection is:
16:41:04 INFO - raptor-main Info: Playback recording information not available.

Comment 8

[Arnold Iakab]

Hi! Did you inspect a test that runs a recently recorded site? Old recordings don't have a json file from where the recording information is being taken.

Comment 9

[Mike Trinkala [:trink]]

No, I was just looking at the test runs associated with this patch as it is unclear to me what other runs should include this data. Is there a specific test in there I should be looking at with a recently recorded site?... In any case I will setup some automation to look at them all.

Comment 10

[Arnold Iakab]

I think that there were no tests that had a json file, except the one for Netflix I guess. We started re-recording after this patch landed.

Here's an example from treeherder:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=253865308&repo=mozilla-central&lineNumber=631