1633332 - Intermittent Failed to run visualmetrics.py error=b'' video_path=<random>

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P5)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://treeherder.mozilla.org/logviewer.html#?job_id=299445063&repo=fenix

[task 2020-04-26T01:36:57.391Z] Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/builds/worker/fetches/browsertime-results/amazon-cold/pages/www.amazon.com/data/video/14.mp4':
[task 2020-04-26T01:36:57.391Z] Metadata:
[task 2020-04-26T01:36:57.392Z] major_brand : isom
[task 2020-04-26T01:36:57.392Z] minor_version : 512
[task 2020-04-26T01:36:57.392Z] compatible_brands: isomiso2avc1mp41
[task 2020-04-26T01:36:57.393Z] encoder : Lavf57.83.100
[task 2020-04-26T01:36:57.393Z] Duration: 00:00:12.64, start: 0.000000, bitrate: 156 kb/s
[task 2020-04-26T01:36:57.393Z] Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1080x1536, 155 kb/s, 5.22 fps, 25 tbr, 12800 tbn, 50 tbc (default)
[task 2020-04-26T01:36:57.394Z] Metadata:
[task 2020-04-26T01:36:57.394Z] handler_name : VideoHandler
[task 2020-04-26T01:36:57.395Z] Stream mapping:
[task 2020-04-26T01:36:57.395Z] Stream #0:0 -> #0:0 (h264 (native) -> png (native))
[task 2020-04-26T01:36:57.395Z] Press [q] to stop, [?] for help
[task 2020-04-26T01:36:57.501Z] Output #0, image2, to '/tmp/vis-Kp0mbP/viewport.png':
[task 2020-04-26T01:36:57.501Z] Metadata:
[task 2020-04-26T01:36:57.501Z] major_brand : isom
[task 2020-04-26T01:36:57.502Z] minor_version : 512
[task 2020-04-26T01:36:57.503Z] compatible_brands: isomiso2avc1mp41
[task 2020-04-26T01:36:57.504Z] encoder : Lavf57.83.100
[task 2020-04-26T01:36:57.504Z] Stream #0:0(und): Video: png, rgb24, 1080x1536, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
[task 2020-04-26T01:36:57.505Z] Metadata:
[task 2020-04-26T01:36:57.506Z] handler_name : VideoHandler
[task 2020-04-26T01:36:57.507Z] encoder : Lavc57.107.100 png
[task 2020-04-26T01:36:57.567Z] frame= 1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A speed=0.234x
[task 2020-04-26T01:36:57.567Z] video:8kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[task 2020-04-26T01:36:57.779Z] 2020-04-26T01:36:57.777585Z Command succeeded result=b'{"SpeedIndex": 1250, "FirstVisualChange": 1160, "LastVisualChange": 4000, "VisualProgress": "0=0, 1160=85, 1600=99, 1720=99, 1760=99, 1840=99, 2600=99, 3360=99, 3400=99, 3960=99, 4000=100", "ContentfulSpeedIndex": 1266, "videoRecordingStart": 2080, "ContentfulSpeedIndexProgress": "0=0, 1160=91, 1600=91, 1720=91, 1760=91, 1840=91, 2600=93, 3360=94, 3400=94, 3960=99, 4000=100", "PerceptualSpeedIndex": 1286, "PerceptualSpeedIndexProgress": "0=0, 1160=90, 1600=98, 1720=98, 1760=98, 1840=98, 2600=99, 3360=99, 3400=99, 3960=99, 4000=100"}\n'
[task 2020-04-26T01:36:57.779Z] 2020-04-26T01:36:57.778159Z Running command cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '--video', '/builds/worker/fetches/browsertime-results/amazon-cold/pages/www.amazon.com/data/video/15.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport']
[task 2020-04-26T01:36:57.852Z] 01:36:57.852 - Input video file /builds/worker/fetches/browsertime-results/amazon-cold/pages/www.amazon.com/data/video/15.mp4 does not exist
[task 2020-04-26T01:36:57.852Z] 01:36:57.852 - No video frames found in /tmp/vis-fIIXp1
[task 2020-04-26T01:36:57.861Z] 2020-04-26T01:36:57.860718Z Command failed cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '--video', '/builds/worker/fetches/browsertime-results/amazon-cold/pages/www.amazon.com/data/video/15.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport'] output=b'' status=1
[task 2020-04-26T01:36:58.041Z] 2020-04-26T01:36:58.041614Z Command succeeded result=b'{"SpeedIndex": 1470, "FirstVisualChange": 1400, "LastVisualChange": 3440, "VisualProgress": "0=0, 1400=85, 1760=99, 2120=99, 2200=99, 2920=99, 3080=99, 3360=99, 3440=100", "ContentfulSpeedIndex": 1464, "videoRecordingStart": 2080, "ContentfulSpeedIndexProgress": "0=0, 1400=95, 1760=95, 2120=96, 2200=96, 2920=97, 3080=97, 3360=99, 3440=100", "PerceptualSpeedIndex": 1500, "PerceptualSpeedIndexProgress": "0=0, 1400=93, 1760=99, 2120=99, 2200=99, 2920=99, 3080=99, 3360=99, 3440=100"}\n'
[task 2020-04-26T01:37:01.372Z] 2020-04-26T01:37:01.372377Z Command succeeded result=b'{"SpeedIndex": 1558, "FirstVisualChange": 1480, "LastVisualChange": 3720, "VisualProgress": "0=0, 1480=85, 1880=99, 2080=99, 2320=99, 2440=99, 2640=99, 3040=99, 3280=99, 3680=99, 3720=100", "ContentfulSpeedIndex": 1548, "videoRecordingStart": 2040, "ContentfulSpeedIndexProgress": "0=0, 1480=95, 1880=95, 2080=95, 2320=95, 2440=96, 2640=96, 3040=96, 3280=97, 3680=100, 3720=100", "PerceptualSpeedIndex": 1590, "PerceptualSpeedIndexProgress": "0=0, 1480=93, 1880=99, 2080=99, 2320=99, 2440=99, 2640=99, 3040=99, 3280=99, 3680=99, 3720=100"}\n'
[task 2020-04-26T01:37:02.033Z] 2020-04-26T01:37:02.033716Z Command succeeded result=b'{"SpeedIndex": 2250, "FirstVisualChange": 1400, "LastVisualChange": 4680, "VisualProgress": "0=0, 1400=79, 1680=71, 2160=71, 2320=71, 2520=71, 2880=71, 3280=71, 3560=70, 3600=71, 3680=71, 3960=71, 4160=74, 4280=83, 4360=88, 4520=95, 4680=100", "ContentfulSpeedIndex": 2001, "videoRecordingStart": 2080, "ContentfulSpeedIndexProgress": "0=0, 1400=69, 1680=72, 2160=77, 2320=79, 2520=81, 2880=84, 3280=89, 3560=93, 3600=93, 3680=94, 3960=97, 4160=98, 4280=99, 4360=99, 4520=99, 4680=100", "PerceptualSpeedIndex": 2828, "PerceptualSpeedIndexProgress": "0=0, 1400=49, 1680=54, 2160=61, 2320=64, 2520=67, 2880=73, 3280=79, 3560=84, 3600=85, 3680=86, 3960=90, 4160=93, 4280=95, 4360=96, 4520=98, 4680=100"}\n'
[task 2020-04-26T01:37:02.034Z] 2020-04-26T01:37:02.034419Z Failed to run visualmetrics.py error=b'' video_path=PosixPath('/builds/worker/fetches/browsertime-results/amazon-cold/pages/www.amazon.com/data/video/15.mp4')

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I assume this is related to bug 1632868 comment 1?

Comment 2

[Greg Mierzwinski [:sparky]]

No, this is a visual metrics failure which happens after that failure.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

So I assume it's blocked by that then.

Comment 4

[Intermittent Failures Robot]

3 failures in 5423 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 3

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-04-27&endday=2020-05-03&tree=all

Comment 5

[Intermittent Failures Robot]

1 failures in 5276 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-05-04&endday=2020-05-10&tree=all

Comment 6

[Intermittent Failures Robot]

2 failures in 4949 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-06-01&endday=2020-06-07&tree=all

Comment 8

[Intermittent Failures Robot]

3 failures in 5211 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-shippable: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-06-08&endday=2020-06-14&tree=all

Comment 9

[Intermittent Failures Robot]

5 failures in 4035 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3
• autoland: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-shippable: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-06-15&endday=2020-06-21&tree=all

Comment 10

[Intermittent Failures Robot]

11 failures in 5252 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• fenix: 10

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-shippable: 1
• android-hw-g5-7-0-arm7-api-16: 5
• android-hw-p2-8-0-android-aarch64: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-06-22&endday=2020-06-28&tree=all

Comment 12

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This is a permanent failure for Fenix now. Latest run.

Comment 13

[Greg Mierzwinski [:sparky]]

Hmm, the parallelization in vismet is making debugging this issue from logs very hard. I can't tell which logs are coming from where. Looking at the videos from the task I don't see any issues or visual differences between them (there tends to be at least one video that works, but the others fail). I wonder if the video metadata is corrupted.

:barret, would you have any ideas for how we could improve the vismet task logging for debugging?

Comment 14

[Intermittent Failures Robot]

16 failures in 4516 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 10
• mozilla-central: 4
• autoland: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 5
• android-hw-g5-7-0-arm7-api-16: 8
• android-hw-g5-7-0-arm7-api-16-shippable: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-06-29&endday=2020-07-05&tree=all

Comment 15

[Intermittent Failures Robot]

6 failures in 5265 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• fenix: 4
• mozilla-central: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 2
• android-hw-g5-7-0-arm7-api-16: 2
• android-hw-g5-7-0-arm7-api-16-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-07-06&endday=2020-07-12&tree=all

Comment 16

[Intermittent Failures Robot]

3 failures in 4014 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-07-13&endday=2020-07-19&tree=all

Comment 17

[Intermittent Failures Robot]

3 failures in 4310 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-central: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-07-20&endday=2020-07-26&tree=all

Comment 19

[Intermittent Failures Robot]

8 failures in 4289 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 7
• autoland: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-07-27&endday=2020-08-02&tree=all

Comment 20

[Andreea Pavel [:apavel]]

Recent failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&resultStatus=testfailed%2Cbusted%2Cexception&fromchange=e39b0c4bf7c9c7c25cdc001ca7e43474c0d7f235&selectedTaskRun=HYUhesKUR0-m2QJOEO-PBA.0

Comment 21

[Intermittent Failures Robot]

2 failures in 4618 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 2

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-08-03&endday=2020-08-09&tree=all

Comment 22

[Intermittent Failures Robot]

5 failures in 3601 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 5

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-shippable: 1
• android-hw-g5-7-0-arm7-api-16-shippable: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-08-10&endday=2020-08-16&tree=all

Comment 23

[Intermittent Failures Robot]

2 failures in 4100 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 1
• android-hw-p2-8-0-android-aarch64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-08-17&endday=2020-08-23&tree=all

Comment 24

[Intermittent Failures Robot]

1 failures in 4046 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-08-24&endday=2020-08-30&tree=all

Comment 25

[Andreea Pavel [:apavel]]

Recent failure log: https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception&revision=c400c4122d0447080d05b5bf03bc9bcd108ba541&selectedTaskRun=AYLKKJMzS8arSkcmZ6jQ7g.0

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=314594524&repo=autoland&lineNumber=367

[task 2020-09-01T19:22:07.796Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55baab4f77c0] Format mov,mp4,m4a,3gp,3g2,mj2 detected only with low score of 1, misdetection possible!
[task 2020-09-01T19:22:07.796Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55baab4f77c0] moov atom not found
[task 2020-09-01T19:22:07.796Z] /builds/worker/fetches/browsertime-results/facebook-cristiano/pages/m.facebook.com/Cristiano/data/video/14.mp4: Invalid data found when processing input
[task 2020-09-01T19:22:07.916Z] 19:22:07.916 - Error extracting the video frames from /builds/worker/fetches/browsertime-results/facebook-cristiano/pages/m.facebook.com/Cristiano/data/video/15.mp4
[task 2020-09-01T19:22:07.917Z] 19:22:07.916 - No video frames found in /tmp/vis-VctpNq
[task 2020-09-01T19:22:07.925Z] 2020-09-01T19:22:07.924820Z Command failed cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '--video', '/builds/worker/fetches/browsertime-results/facebook-cristiano/pages/m.facebook.com/Cristiano/data/video/15.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport'] output=b'' status=1
[task 2020-09-01T19:22:07.932Z] 19:22:07.932 - Error extracting the video frames from /builds/worker/fetches/browsertime-results/facebook-cristiano/pages/m.facebook.com/Cristiano/data/video/13.mp4
[task 2020-09-01T19:22:07.932Z] 19:22:07.932 - No video frames found in /tmp/vis-nrHuN0
[task 2020-09-01T19:22:07.938Z] 2020-09-01T19:22:07.938791Z Command failed cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '--video', '/builds/worker/fetches/browsertime-results/facebook-cristiano/pages/m.facebook.com/Cristiano/data/video/13.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport'] output=b'' status=1
[task 2020-09-01T19:22:07.939Z] 2020-09-01T19:22:07.939416Z Failed to run visualmetrics.py error=b'' video_path=PosixPath('/builds/worker/fetches/browsertime-results/facebook-cristiano/pages/m.facebook.com/Cristiano/data/video/13.mp4')
[task 2020-09-01T19:22:07.967Z] 19:22:07.966 - Error extracting the video frames from /builds/worker/fetches/browsertime-results/facebook-cristiano/pages/m.facebook.com/Cristiano/data/video/14.mp4
[task 2020-09-01T19:22:07.967Z] 19:22:07.967 - No video frames found in /tmp/vis-OwO_CG
[task 2020-09-01T19:22:07.973Z] 2020-09-01T19:22:07.973189Z Command failed cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '--video', '/builds/worker/fetches/browsertime-results/facebook-cristiano/pages/m.facebook.com/Cristiano/data/video/14.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport'] output=b'' status=1
[task 2020-09-01T19:22:07.974Z] 2020-09-01T19:22:07.973907Z Failed to run visualmetrics.py error=b'' video_path=PosixPath('/builds/worker/fetches/browsertime-results/facebook-cristiano/pages/m.facebook.com/Cristiano/data/video/14.mp4')
[task 2020-09-01T19:22:07.974Z] 2020-09-01T19:22:07.974178Z Failed to run visualmetrics.py error=b'' video_path=PosixPath('/builds/worker/fetches/browsertime-results/facebook-cristiano/pages/m.facebook.com/Cristiano/data/video/15.mp4')
[task 2020-09-01T19:22:09.377Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3624f80] moov atom not found
[task 2020-09-01T19:22:09.378Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.378Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.378Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.379Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.379Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.379Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.379Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.380Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.380Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.380Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.380Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.381Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.381Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.381Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x3625600] moov atom not found
[task 2020-09-01T19:22:09.383Z] 2020-09-01T19:22:09.383415Z Found 15 new videos
[task 2020-09-01T19:22:09.384Z] 2020-09-01T19:22:09.384084Z Querying Active-data...
[task 2020-09-01T19:22:12.586Z] 2020-09-01T19:22:12.586184Z Status: {'200'}
[task 2020-09-01T19:22:12.658Z] 2020-09-01T19:22:12.658071Z Found 8 datums
[task 2020-09-01T19:22:12.658Z] 2020-09-01T19:22:12.658203Z Comparing videos to TASK_GROUP=EhFN7FUwTuKQKBK4WK_CJA, TASK_ID=Lu0Q8PQmTXii0OZiarNMww
[task 2020-09-01T19:22:12.658Z] 2020-09-01T19:22:12.658433Z Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Lu0Q8PQmTXii0OZiarNMww/artifacts/public/test_info/browsertime-results.tgz
[task 2020-09-01T19:22:14.152Z] 2020-09-01T19:22:14.152094Z Found 15 old videos
[task 2020-09-01T19:22:15.463Z] 2020-09-01T19:22:15.463425Z Failed to calculate similarity score
[task 2020-09-01T19:22:15.463Z] Traceback (most recent call last):
[task 2020-09-01T19:22:15.464Z] File "/builds/worker/bin/run-visual-metrics.py", line 308, in main
[task 2020-09-01T19:22:15.464Z] for name, value in calculate_similarity(jobs_json, fetch_dir, OUTPUT_DIR).items():
[task 2020-09-01T19:22:15.464Z] File "/builds/worker/bin/similarity.py", line 333, in calculate_similarity
[task 2020-09-01T19:22:15.464Z] old_btime_videos, new_btime_videos, output
[task 2020-09-01T19:22:15.464Z] File "/builds/worker/bin/similarity.py", line 251, in _get_similarity
[task 2020-09-01T19:22:15.464Z] histn2d, _, _ = plt.hist(datan[-1, :, :].flatten(), bins=255)
[task 2020-09-01T19:22:15.464Z] IndexError: too many indices for array
[task 2020-09-01T19:22:15.471Z] 2020-09-01T19:22:15.471814Z PERFHERDER_DATA: {"framework": {"name": "browsertime"}, "application": {"name": "geckoview"}, "type": "pageload", "suites": []}
[taskcluster 2020-09-01 19:22:15.914Z] === Task Finished ===
[taskcluster 2020-09-01 19:22:16.944Z] Unsuccessful task run with exit code: 15 completed in 38.809 seconds

Greg can you take a look?

Comment 26

[Greg Mierzwinski [:sparky]]

I've scheduled a true retrigger here: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=c400c4122d0447080d05b5bf03bc9bcd108ba541&searchStr=fb-cris-c&selectedTaskRun=AYLKKJMzS8arSkcmZ6jQ7g.0

This task needs to be retriggered in the following way so I don't think this is a perma-fail just yet: https://wiki.mozilla.org/TestEngineering/Performance/Raptor/Browsertime#Retriggering_Browsertime_Visual_Metrics_Tasks

Comment 27

[Greg Mierzwinski [:sparky]]

Retrigger passed, this is not a perma-fail (thankfully :) ).

Comment 28

[Intermittent Failures Robot]

5 failures in 2984 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4
• mozilla-beta: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-08-31&endday=2020-09-06&tree=all

Comment 29

[Intermittent Failures Robot]

1 failures in 3410 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-09-07&endday=2020-09-13&tree=all

Comment 30

[Natalia Csoregi [:nataliaCs]]

Update:
There have been 43 failures within the last 7 days:

• 2 failures on Android 8.0 Pixel2 AArch64 Shippable opt
• 41 failures on Android 7.0 MotoG5 Shippable opt

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=316191069&repo=mozilla-central&lineNumber=1366

[task 2020-09-20T00:19:48.933Z] 2020-09-20T00:19:48.932911Z Found 30 new videos
[task 2020-09-20T00:19:48.933Z] 2020-09-20T00:19:48.933456Z Querying Active-data...
[task 2020-09-20T00:19:53.614Z] 2020-09-20T00:19:53.614170Z Status: {'200'}
[task 2020-09-20T00:19:53.758Z] 2020-09-20T00:19:53.758587Z Found 17 datums
[task 2020-09-20T00:19:53.758Z] 2020-09-20T00:19:53.758704Z Comparing videos to TASK_GROUP=ZPvKf31RSKC1qs_FOVinNQ, TASK_ID=VvpBRAelR5CJoZr_kLdVCg
[task 2020-09-20T00:19:53.758Z] 2020-09-20T00:19:53.758875Z Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VvpBRAelR5CJoZr_kLdVCg/artifacts/public/test_info/browsertime-results.tgz
[task 2020-09-20T00:19:56.230Z] 2020-09-20T00:19:56.230452Z Found 30 old videos
[task 2020-09-20T00:28:39.443Z] 2020-09-20T00:28:39.443629Z Average 3D similarity: 0.98015
[task 2020-09-20T00:28:39.443Z] 2020-09-20T00:28:39.443716Z Average 2D similarity: 0.99948
[task 2020-09-20T00:28:39.445Z] 2020-09-20T00:28:39.445460Z Querying Active-data...
[task 2020-09-20T00:28:41.232Z] 2020-09-20T00:28:41.232001Z Status: {'200'}
[task 2020-09-20T00:28:41.232Z] 2020-09-20T00:28:41.232216Z Couldn't get activedata data
[task 2020-09-20T00:28:41.232Z] 2020-09-20T00:28:41.232256Z Failed to find a live site variant
[task 2020-09-20T00:28:41.235Z] 2020-09-20T00:28:41.235336Z PERFHERDER_DATA: {"framework": {"name": "browsertime"}, "application": {"name": "geckoview"}, "type": "pageload", "suites": [{"name": "imdb", "subtests": [{"name": "SpeedIndex", "replicates": [3389, 3214, 3468, 3099, 3361, 3190, 3538, 3143, 3310, 3451, 3490, 3316, 3363, 3111], "lowerIsBetter": true, "unit": "ms", "value": 3338.5}, {"name": "FirstVisualChange", "replicates": [1309, 1145, 1442, 1071, 1369, 1135, 1470, 1070, 1270, 2055, 2090, 1324, 1305, 1066], "lowerIsBetter": true, "unit": "ms", "value": 1307.0}, {"name": "LastVisualChange", "replicates": [11095, 10921, 10699, 10678, 10679, 10327, 10945, 10728, 11013, 10479, 10494, 10482, 10546, 10292], "lowerIsBetter": true, "unit": "ms", "value": 10678.5}, {"name": "ContentfulSpeedIndex", "replicates": [3609, 3485, 3802, 3385, 3688, 3503, 3768, 3386, 3586, 4124, 4201, 3656, 3563, 3416], "lowerIsBetter": true, "unit": "ms", "value": 3597.5}, {"name": "videoRecordingStart", "replicates": [1607, 724, 740, 762, 748, 746, 756, 753, 760, 743, 765, 739, 739, 763], "lowerIsBetter": true, "unit": "ms", "value": 750.5}, {"name": "PerceptualSpeedIndex", "replicates": [5739, 5568, 5588, 5418, 5483, 5333, 5862, 5475, 5670, 5507, 5538, 5463, 5472, 5244], "lowerIsBetter": true, "unit": "ms", "value": 5495.0}, {"name": "Similarity", "value": 0.98015, "replicates": [0.98015], "lowerIsBetter": false, "unit": "a.u."}, {"name": "Similarity2D", "value": 0.99948, "replicates": [0.99948], "lowerIsBetter": false, "unit": "a.u."}], "extraOptions": ["cold"]}, {"name": "imdb", "subtests": [{"name": "SpeedIndex", "replicates": [1787, 1694, 1711, 1671, 1744, 1705, 1727, 1743, 1769, 1766, 1789, 1785, 1702, 1796, 1755], "lowerIsBetter": true, "unit": "ms", "value": 1744}, {"name": "FirstVisualChange", "replicates": [417, 429, 423, 438, 437, 369, 434, 435, 437, 435, 434, 467, 423, 456, 451], "lowerIsBetter": true, "unit": "ms", "value": 435}, {"name": "LastVisualChange", "replicates": [10176, 9855, 9897, 10030, 9895, 9996, 9895, 9910, 9978, 9992, 10075, 10008, 9883, 10079, 9858], "lowerIsBetter": true, "unit": "ms", "value": 9978}, {"name": "ContentfulSpeedIndex", "replicates": [2263, 1889, 1934, 1479, 1938, 1941, 1940, 1907, 1931, 1986, 2003, 1951, 1900, 1963, 1983], "lowerIsBetter": true, "unit": "ms", "value": 1940}, {"name": "videoRecordingStart", "replicates": [805, 843, 821, 809, 829, 828, 826, 827, 840, 839, 854, 819, 828, 828, 829], "lowerIsBetter": true, "unit": "ms", "value": 828}, {"name": "PerceptualSpeedIndex", "replicates": [4826, 4611, 4791, 4796, 4692, 4761, 4616, 4714, 4709, 4781, 4772, 4805, 4613, 4793, 4631], "lowerIsBetter": true, "unit": "ms", "value": 4761}], "extraOptions": ["warm"]}]}
[taskcluster 2020-09-20 00:28:42.417Z] === Task Finished ===
[taskcluster 2020-09-20 00:28:43.585Z] Unsuccessful task run with exit code: 1 completed in 974.054 seconds

Comment 31

[Intermittent Failures Robot]

43 failures in 3760 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #23 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 38
• mozilla-central: 5

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 41
• android-hw-p2-8-0-android-aarch64-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-09-14&endday=2020-09-20&tree=all

Comment 32

[Greg Mierzwinski [:sparky]]

This increased since we've enabled chimera mode but it's not clear if it's the cause of this increase since it took a lot of autoland patches/tests before it was triggered.

The failure only seems to occur on geckoview and on the IMDB test page, generally, it fails with a single video. The performance data is still being reported though for the videos that pass.

I'll dig into this some more to see what's going on.

Comment 33

[Greg Mierzwinski [:sparky]]

When I download an archive with a video that is passing and attempt to run it through visualmetrics.py locally, everything passes consistently. It sounds like there's an issue on the linux machines. I've tried redirecting stderr output to stdout but it didn't show anything useful in try either.

Comment 34

[Greg Mierzwinski [:sparky]]

Ah so with the -v (for verbose) option, I am able to get some better error messages out of the IMDB vismet tasks:

[task 2020-09-22T17:23:16.900Z] 17:23:16.889 - list index out of range
[task 2020-09-22T17:23:16.900Z] Traceback (most recent call last):
[task 2020-09-22T17:23:16.901Z]   File "/builds/worker/fetches/visualmetrics.py", line 1910, in main
[task 2020-09-22T17:23:16.901Z]     options.herodata)
[task 2020-09-22T17:23:16.901Z]   File "/builds/worker/fetches/visualmetrics.py", line 1331, in calculate_visual_metrics
[task 2020-09-22T17:23:16.901Z]     value, value_progress = calculate_contentful_speed_index(progress, dirs)
[task 2020-09-22T17:23:16.901Z]   File "/builds/worker/fetches/visualmetrics.py", line 1494, in calculate_contentful_speed_index
[task 2020-09-22T17:23:16.901Z]     value  = int(output.split()[7].split(':')[0])
[task 2020-09-22T17:23:16.901Z] IndexError: list index out of range

Looks like it fails when it's trying to produce the contenful speedindex metric.

Comment 35

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1633332 - Update browsertime visualmetrics script used in CI. r?#perftest

This patch updates the version of visualmetrics.py used in our CI to the latest one.

Depends on D91042

Comment 36

[Greg Mierzwinski [:sparky]]

Looks like we haven't updated visualmetrics.py in a while. Try run with it: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c29636769d177402ec0e11fbff8918f4df7f2160

Comment 37

[Greg Mierzwinski [:sparky]]

I'm able to trigger the error locally with the old script, but the new script works fine so this update should fix this issue.

Comment 38

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c24a4a66f4bd
Update browsertime visualmetrics script used in CI. r=perftest-reviewers,Bebe

Comment 39

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/c24a4a66f4bd

Comment 40

[Intermittent Failures Robot]

19 failures in 4019 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 5
• autoland: 9
• mozilla-central: 5

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-shippable: 4
• android-hw-g5-7-0-arm7-api-16-shippable: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-09-21&endday=2020-09-27&tree=all

Comment 41

[Cosmin Sabou [:CosminS]]

Greg, this doesn't seem to be fixed, it's now failing on microsoft-support/pages/support.microsoft.com/en-us/query-6bc054f4/data/video/15.mp4'):

[task 2020-09-28T16:02:18.814Z] 2020-09-28T16:02:18.814681Z [TEST-UNEXPECTED FAIL] Command failed cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '-v', '--video', '/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/3.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport'] output=b'' status=1
[task 2020-09-28T16:02:18.816Z] 2020-09-28T16:02:18.815403Z Running command                cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '-v', '--video', '/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/5.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport']
[task 2020-09-28T16:02:18.816Z] 2020-09-28T16:02:18.815735Z [TEST-UNEXPECTED FAIL] Command failed cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '-v', '--video', '/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/2.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport'] output=b'' status=1
[task 2020-09-28T16:02:18.816Z] 2020-09-28T16:02:18.816461Z Running command                cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '-v', '--video', '/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/6.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport']
[task 2020-09-28T16:02:18.817Z] 2020-09-28T16:02:18.816481Z [TEST-UNEXPECTED FAIL] Command failed cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '-v', '--video', '/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/1.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport'] output=b'' status=1
[task 2020-09-28T16:02:18.817Z] 2020-09-28T16:02:18.817674Z Failed to run visualmetrics.py error=b'' video_path=PosixPath('/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/1.mp4')
[task 2020-09-28T16:02:18.817Z] 2020-09-28T16:02:18.817814Z Failed to run visualmetrics.py error=b'' video_path=PosixPath('/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/2.mp4')
[task 2020-09-28T16:02:18.817Z] 2020-09-28T16:02:18.817767Z [TEST-UNEXPECTED FAIL] Command failed cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '-v', '--video', '/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/4.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport'] output=b'' status=1
[task 2020-09-28T16:02:18.817Z] 2020-09-28T16:02:18.817858Z Failed to run visualmetrics.py error=b'' video_path=PosixPath('/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/3.mp4')
[task 2020-09-28T16:02:18.819Z] 2020-09-28T16:02:18.818325Z Running command                cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '-v', '--video', '/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/7.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport']
[task 2020-09-28T16:02:18.819Z] 2020-09-28T16:02:18.818469Z Failed to run visualmetrics.py error=b'' video_path=PosixPath('/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/4.mp4')
[task 2020-09-28T16:02:18.819Z] 2020-09-28T16:02:18.818560Z Running command                cmd=['/usr/bin/python', '/builds/worker/fetches/visualmetrics.py', '-v', '--video', '/builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/8.mp4', '--orange', '--perceptual', '--contentful', '--force', '--renderignore', '5', '--json', '--viewport']
[task 2020-09-28T16:02:18.978Z] ffmpeg version 4.1.4-1~deb10u1 Copyright (c) 2000-2019 the FFmpeg developers
[task 2020-09-28T16:02:18.978Z]   built with gcc 8 (Debian 8.3.0-6)
[task 2020-09-28T16:02:18.978Z]   configuration: --prefix=/usr --extra-version='1~deb10u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
[task 2020-09-28T16:02:18.978Z] ffmpeg version 4.1.4-1~deb10u1 Copyright (c) 2000-2019 the FFmpeg developers
[task 2020-09-28T16:02:18.978Z]   libavutil      56. 22.100 / 56. 22.100
[task 2020-09-28T16:02:18.978Z]   libavcodec     58. 35.100 / 58. 35.100
[task 2020-09-28T16:02:18.978Z]   libavformat    58. 20.100 / 58. 20.100
[task 2020-09-28T16:02:18.978Z]   libavdevice    58.  5.100 / 58.  5.100
[task 2020-09-28T16:02:18.978Z]   libavfilter     7. 40.101 /  7. 40.101
[task 2020-09-28T16:02:18.978Z]   libavresample   4.  0.  0 /  4.  0.  0
[task 2020-09-28T16:02:18.978Z]   libswscale      5.  3.100 /  5.  3.100
[task 2020-09-28T16:02:18.978Z]   libswresample   3.  3.100 /  3.  3.100
[task 2020-09-28T16:02:18.978Z]   libpostproc    55.  3.100 / 55.  3.100
[task 2020-09-28T16:02:18.978Z]   built with gcc 8 (Debian 8.3.0-6)
[task 2020-09-28T16:02:18.978Z]   configuration: --prefix=/usr --extra-version='1~deb10u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
[task 2020-09-28T16:02:18.978Z]   libavutil      56. 22.100 / 56. 22.100
[task 2020-09-28T16:02:18.978Z]   libavcodec     58. 35.100 / 58. 35.100
[task 2020-09-28T16:02:18.978Z]   libavformat    58. 20.100 / 58. 20.100
[task 2020-09-28T16:02:18.978Z]   libavdevice    58.  5.100 / 58.  5.100
[task 2020-09-28T16:02:18.978Z]   libavfilter     7. 40.101 /  7. 40.101
[task 2020-09-28T16:02:18.979Z]   libavresample   4.  0.  0 /  4.  0.  0
[task 2020-09-28T16:02:18.979Z]   libswscale      5.  3.100 /  5.  3.100
[task 2020-09-28T16:02:18.979Z]   libswresample   3.  3.100 /  3.  3.100
[task 2020-09-28T16:02:18.979Z]   libpostproc    55.  3.100 / 55.  3.100
[task 2020-09-28T16:02:18.979Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55bab1e227c0] Format mov,mp4,m4a,3gp,3g2,mj2 detected only with low score of 1, misdetection possible!
[task 2020-09-28T16:02:18.979Z] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x55bab1e227c0] moov atom not found
[task 2020-09-28T16:02:18.979Z] ffmpeg version 4.1.4-1~deb10u1 Copyright (c) 2000-2019 the FFmpeg developers
[task 2020-09-28T16:02:18.979Z]   built with gcc 8 (Debian 8.3.0-6)
[task 2020-09-28T16:02:18.979Z]   configuration: --prefix=/usr --extra-version='1~deb10u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
[task 2020-09-28T16:02:18.979Z] /builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/data/video/5.mp4: Invalid data found when processing input

Treeherder link: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&selectedTaskRun=WdvQ2P7VSnmpSztGO1QWtQ.0&searchStr=android%2C7.0%2Cmotog5%2Cshippable%2Copt%2Cbrowsertime%2Cperformance%2Ctests%2Con%2Cfirefox%2Cmicro&revision=2d3383ca5dd166ea00f3cc60a720240dfd10169d

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=316926346&repo=autoland&lineNumber=433
Could you have a look over this? Thank you.

Comment 42

[Greg Mierzwinski [:sparky]]

:cosmins, can you file a new bug for those new failures and re-assign these to that bug? They are different from the failure that was fixed.

[task 2020-09-28T16:02:20.447Z] /builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/query-6bc054f4/data/video/8.mp4: Invalid data found when processing input
[task 2020-09-28T16:02:20.637Z] 16:02:20.636 - Error extracting the video frames from /builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/query-6bc054f4/data/video/9.mp4
[task 2020-09-28T16:02:20.637Z] 16:02:20.636 - Error extracting the video frames from /builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/query-6bc054f4/data/video/6.mp4
[task 2020-09-28T16:02:20.637Z] 16:02:20.636 - Error extracting the video frames from /builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/query-6bc054f4/data/video/7.mp4
[task 2020-09-28T16:02:20.638Z] 16:02:20.636 - Error extracting the video frames from /builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/query-6bc054f4/data/video/8.mp4
[task 2020-09-28T16:02:20.638Z] 16:02:20.637 - No video frames found in /tmp/vis-KTcf3Y
[task 2020-09-28T16:02:20.638Z] 16:02:20.637 - No video frames found in /tmp/vis-ASthgK
[task 2020-09-28T16:02:20.638Z] 16:02:20.637 - No video frames found in /tmp/vis-dtj0Fq
[task 2020-09-28T16:02:20.638Z] 16:02:20.638 - No video frames found in /tmp/vis-28MvXF

Comment 43

[Razvan Maries]

(In reply to Greg Mierzwinski [:sparky] from comment #42)

:cosmins, can you file a new bug for those new failures and re-assign these to that bug? They are different from the failure that was fixed.

[task 2020-09-28T16:02:20.447Z] /builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/query-6bc054f4/data/video/8.mp4: Invalid data found when processing input
[task 2020-09-28T16:02:20.637Z] 16:02:20.636 - Error extracting the video frames from /builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/query-6bc054f4/data/video/9.mp4
[task 2020-09-28T16:02:20.637Z] 16:02:20.636 - Error extracting the video frames from /builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/query-6bc054f4/data/video/6.mp4
[task 2020-09-28T16:02:20.637Z] 16:02:20.636 - Error extracting the video frames from /builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/query-6bc054f4/data/video/7.mp4
[task 2020-09-28T16:02:20.638Z] 16:02:20.636 - Error extracting the video frames from /builds/worker/fetches/browsertime-results/microsoft-support/pages/support.microsoft.com/en-us/query-6bc054f4/data/video/8.mp4
[task 2020-09-28T16:02:20.638Z] 16:02:20.637 - No video frames found in /tmp/vis-KTcf3Y
[task 2020-09-28T16:02:20.638Z] 16:02:20.637 - No video frames found in /tmp/vis-ASthgK
[task 2020-09-28T16:02:20.638Z] 16:02:20.637 - No video frames found in /tmp/vis-dtj0Fq
[task 2020-09-28T16:02:20.638Z] 16:02:20.638 - No video frames found in /tmp/vis-28MvXF

I filled Bug 1668088.

Comment 44

[Greg Mierzwinski [:sparky]]

Thanks! I've filled bug 1668049 since it's not easy to see these errors right now.

Comment 45

[Cosmin Sabou [:CosminS]]

Thank you both for filing those bugs.

Comment 46

[Intermittent Failures Robot]

4 failures in 4195 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 2
• mozilla-central: 2

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16-shippable: 3
• android-hw-p2-8-0-android-aarch64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1633332&startday=2020-09-28&endday=2020-10-04&tree=all