1596237 - Browsertime visual metrics fail on Windows due to ffmpeg not having png support

Status: RESOLVED FIXED

(Testing :: Performance, defect, P2)

Description

[Andrew Creskey [:acreskey]]

Steps to reproduce

1. Run ./mach browsertime --setup and ensure that the visual metrics tests pass. ("OK")
2. Collect visual metrics for a given site. The Firefox windowRecorder is required on Windows.
   e.g.

./mach browsertime https://www.mozilla.com --browser firefox --firefox.binaryPath C:/mozilla-source/build/obj-release/dist/bin/firefox.exe --visualMetrics true --firefox.windowRecorder true

Results
The visual metrics processing will fail with an error like:

[2019-11-12 18:36:13] ERROR: Visual Metrics failed to analyse the video Error: Command failed: c:/mozilla-source/build/obj-release\_virtualenvs\init\Scripts\python.exe c:\mozilla-source\mozilla-central\tools\browsertime\node_modules\browsertime\vendor\visualmetrics.py --video c:\mozilla-source\mozilla-central\browsertime-results\facebook\preload2\pages\www.facebook.com\data\video\1.mp4 --orange --perceptual --contentful --force --renderignore 5 --json --viewport ...

In my case, browsertime's visualmetrics.py was using ./mozbuild/browsertime/ImageMagick-7.0.8/ffmpeg.exe which did does not support png export on Windows.

If I replace that ffmpeg.exe with the one from .mozbuild\browsertime\ffmpeg-4.1.1-win64-static\bin, then visual metrics succeed.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The priority flag is not set for this bug.
:davehunt, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 2

[Tarek Ziadé (:tarek)]

Attachment: Bug 1596237 - make sure ImageMagick's ffmpeg does not take precedence

Tweaked sys.path on windows to make sure we use the right ffmpeg binary

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/53a76d0bbaa2
make sure ImageMagick's ffmpeg does not take precedence r=acreskey

Comment 4

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/53a76d0bbaa2