1239293 - [tc-gip] Turn on video recording

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

With bug 1144103 and bug 1237069 being done, we can get a video of each test case that failed.

Comment 1

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

Attachment: MozReview Request: Bug 1239293 - [tc-gip] Turn on video recording

Review commit: https://reviewboard.mozilla.org/r/30681/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/30681/

Comment 2

[Greg Arndt [:garndt]]

Comment on attachment 8707413 [details]
MozReview Request: Bug 1239293 - [tc-gip] Turn on video recording

https://reviewboard.mozilla.org/r/30681/#review27675

::: testing/taskcluster/tasks/phone_test.yml:48
(Diff revision 1)
> +      'private/b2g/videos':

Are these videos large? Are they something we should be keeping around for a year? Just keeping that in mind as we add this.

Comment 3

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

These videos are usually 5MB long for a test that failed after 90 seconds. Firefox OS doesn't record more than 3 minutes. So worst case scenario, we have 1 video of 10MB.

Right now we have around 4 failures per tasks. Hence, the average usage per task will be around 20MB.

There are 6 (currently) failing jobs for each commit and about 10 commits a day. Then we'd require about 5.8TB the first year. Then, after a year, old videos will be discarded and we should maintain around the same usage.

Do you think taskcluster can handle this amount of data?

If not, I'm fine with reducing the expiration date to 3 months. The main use case is just to find out the reason of an intermittent failure.

Comment 4

[Greg Arndt [:garndt]]

I don't think it's a crazy amount considering everything else that gets uploaded for a task.  Just wanted to make sure it wasn't like 1gb per task.  We can leave it like this if you think a year is sufficient.  Of course we would want to reduce that expiration down as much as we could to ensure we're not storing it for too long but also keep it around long enough to be useful for people.  Do you think people in 8 months will go back to look at these videos?

Comment 5

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

(In reply to Greg Arndt [:garndt] from comment #4)
> Do you think people in 8 months will go back to look at these videos?

Chatted with Martijn and NoJun. Martijn made a good point: if we want to keep a video, we can always attach it to a bug. Then 3 months will be enough. I'm updating the PR.

Comment 6

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

Comment on attachment 8707413 [details]
MozReview Request: Bug 1239293 - [tc-gip] Turn on video recording

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/30681/diff/1-2/

Comment 7

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

https://hg.mozilla.org/integration/b2g-inbound/rev/5097ab0e8aec9b85b05e5f60b41a18cf79917b01
Bug 1239293 - [tc-gip] Turn on video recording

Comment 8

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

There's an error in the configuration:
> [taskcluster] === Task Finished ===
> [taskcluster] Artifact "private/b2g/videos" not found at "/home/worker/upload/videos/"

Example: https://tools.taskcluster.net/task-inspector/#Tzmj4ollRm-onXB8sIijGA/0

Comment 9

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

https://hg.mozilla.org/integration/b2g-inbound/rev/9ec8cb9f2f4b532d3d77e1acc8182508a52fb0c6
Backed out 5097ab0e8aec9b85b05e5f60b41a18cf79917b01

Comment 10

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

Per [1], it seems like gaiatest doesn't support video yet.


[1] https://tools.taskcluster.net/task-inspector/#LRwcNhMIQUquP44EFCpeug/0

Comment 11

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

After running half a dozen of tasks, I realized the video is never pulled to the docker container. 

In [1], for instance, you can see this hint in the logs:
> TEST-START | test_login_bzlite.py TestBugzillaLite.test_login_BugzillaLite
> * daemon not running. starting it now on port 5037 *
> * daemon started successfully *
> error: device not found
> TEST-UNEXPECTED-ERROR | test_login_bzlite.py TestBugzillaLite.test_login_BugzillaLite | TimeoutException: TimeoutException: Timed out after 30.3 seconds

This is the output of this line[2]: as we don't use mozdevice per se, we don't pass the ADB host/port/serial to the command line. Then, we require to implement the screen recorder function in mozdevice. 


[1] https://tools.taskcluster.net/task-inspector/#S-1Kzh3uRGephT9n7q-NkQ/0
[2] https://github.com/mozilla-b2g/gaia/pull/33815/files#diff-6d544bbb7c759ffb68f344f348f7740eR949

Comment 12

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #11)
> Then, we require to implement the screen recorder function in mozdevice. 

Tracked under bug 1239964.

Comment 13

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

Comment on attachment 8707413 [details]
MozReview Request: Bug 1239293 - [tc-gip] Turn on video recording

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/30681/diff/2-3/

Comment 14

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

https://hg.mozilla.org/integration/b2g-inbound/rev/d83a10007bccd01700592fd1d92760c2e1dfca81
Bug 1239293 - [tc-gip] Turn on video recording

Comment 15

[Johan Lorenzo [:jlorenzo] - On PTO - Back on March 18th]

For the record: the patch bitrot due to bug 1225537 (phone_test.yml has been replaced by b2g_e2e_tests_base_definition.yml). Other than that, the changeset is exactly the same as the one reviewed.

Comment 16

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/d83a10007bcc