1559727 - "mach browsertime --setup" fails with "Error checking ffmpeg filters for decimate"

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Florian Quèze [:florian]]

When I run mach browsertime --setup I get the following error message:

Error checking ffmpeg filters for decimate
ffmpeg: FAIL
convert: FAIL
compare: FAIL
Pillow: OK
SSIM: OK

Here is a pastebin of the full output: https://paste.mozilla.org/EXLzy6ER

If I had to make a random guess, I would say this could be due to OS X being in French on my machine, so the decimal point is "," instead of ".".

Comment 1

[Nick Alexander :nalexander [he/him]]

Thanks for the report. It looks like there are two problems. Can you see if manually doing https://bugzilla.mozilla.org/show_bug.cgi?id=1559168 fixes some of the issue for you? And can you suggest why the decimal point is likely? I don't see it in the logs; did you dig deeper into what decimate does, for example?

Comment 2

[Florian Quèze [:florian]]

(In reply to Nick Alexander :nalexander [he/him] from comment #1)

Can you see if manually doing https://bugzilla.mozilla.org/show_bug.cgi?id=1559168 fixes some of the issue for you?

ffmpeg indeed wasn't executable, running chmod +x on it fixed the ffmpeg error.

convert and compare are already executable files. Attempting to run them fails though, here's the error output:

$ ~/.mozbuild/browsertime/ImageMagick-7.0.8/bin/convert
dyld: Library not loaded: /ImageMagick-7.0.8/lib/libMagickCore-7.Q16HDRI.6.dylib
Referenced from: /Users/florian/.mozbuild/browsertime/ImageMagick-7.0.8/bin/convert
Reason: image not found
Abort trap: 6

And can you suggest why the decimal point is likely? I don't see it in the logs; did you dig deeper into what decimate does, for example?

Because "decimate" and "decimal" are similar sounding words, the decimal point is a frequent internationalization issue when something is broken on a French system, and I'm jetlagged ;-). Sorry.

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I'm seeing the same locally, so it's unlikely to be due to macOS running in French locale as I'm running en-GB. Were you able to resolve this Florian?

Comment 4

[Florian Quèze [:florian]]

(In reply to Dave Hunt [:davehunt] [he/him] ⌚BST from comment #3)

I'm seeing the same locally, so it's unlikely to be due to macOS running in French locale as I'm running en-GB. Were you able to resolve this Florian?

No. I just tested again, and I'm getting:

ffmpeg: OK
convert: FAIL
compare: FAIL
Pillow: OK
SSIM: OK

And I guess ffmpeg only works because I ran a chmod +x on it in comment 2.

Comment 5

[Greg Mierzwinski [:sparky]]

:florian, the convert and compare issues might be caused by missing libMagickWand*.6.dylib in /usr/local/lib. You can find the binary in the ImageMagick bin folder.

For me I had to run the following command to get it to work:

ln -s /Users/gregorymierzwinski/.mozbuild/browsertime/ImageMagick-7.0.8/lib/libMagickWand-7.Q16HDRI.6.dylib /usr/local/lib/libMagickWand-7.Q16HDRI.6.dylib

EDIT: I noticed that it's failing the checks when called through ./mach browsertime --setup but it's passing when calling the visual-metrics.py --check directly with:

/Users/gregorymierzwinski/mozilla-source/mozilla-central/obj-ff-dbg/_virtualenvs/init/bin/python /Users/gregorymierzwinski/mozilla-source/mozilla-central/tools/browsertime/node_modules/browsertime/vendor/visualmetrics.py --check

EDIT 2: Actually, it's possible you have the wrong ImageMagick version (it should be 18.7) and unpack_file is breaking. That's what seems to be happening for me.

Comment 6

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1559727 - Set executable bit after extraction, and correct ImageMagick path. r?#perftest

This patch fixes some issues with binaries not being executable after being extracted. It also fixes the ImageMagick expected extraction path to the correct location. To prevent that from occuring again, a check is done after extraction to be sure that the extraction completed successfully.

Comment 7

[Greg Mierzwinski [:sparky]]

:florian, can you confirm that this patch fixes your issues?

Comment 9

[Greg Mierzwinski [:sparky]]

Try run for this patch: https://treeherder.mozilla.org/#/jobs?repo=try&revision=d602eebf9063195284309863a9e9ea7f8e6a5282

(flake8 issues are fixed in the latest patch).

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e3f92a06d6ad
Set executable bit after extraction, and correct ImageMagick path. r=perftest-reviewers,stephendonner

Comment 11

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/e3f92a06d6ad

Comment 12

[Florian Quèze [:florian]]

(In reply to Greg Mierzwinski [:sparky] from comment #7)

:florian, can you confirm that this patch fixes your issues?

I tried again with the current mozilla-central. It still doesn't work.

I still get:
ffmpeg: OK
convert: FAIL
compare: FAIL
Pillow: OK
SSIM: OK

(In reply to Greg Mierzwinski [:sparky] from comment #5)

:florian, the convert and compare issues might be caused by missing libMagickWand*.6.dylib in /usr/local/lib. You can find the binary in the ImageMagick bin folder.

For me I had to run the following command to get it to work:

ln -s /Users/gregorymierzwinski/.mozbuild/browsertime/ImageMagick-7.0.8/lib/libMagickWand-7.Q16HDRI.6.dylib /usr/local/lib/libMagickWand-7.Q16HDRI.6.dylib

I tried this and it didn't help.

EDIT: I noticed that it's failing the checks when called through ./mach browsertime --setup but it's passing when calling the visual-metrics.py --check directly with:

/Users/gregorymierzwinski/mozilla-source/mozilla-central/obj-ff-dbg/_virtualenvs/init/bin/python /Users/gregorymierzwinski/mozilla-source/mozilla-central/tools/browsertime/node_modules/browsertime/vendor/visualmetrics.py --check

When I try running this directly without mach, I get:
ffmpeg: FAIL
convert: FAIL
compare: FAIL
Pillow: OK
SSIM: OK

EDIT 2: Actually, it's possible you have the wrong ImageMagick version (it should be 18.7) and unpack_file is breaking. That's what seems to be happening for me.

$ DYLD_LIBRARY_PATH="/Users/florian/.mozbuild/browsertime/ImageMagick-7.0.9/lib/" /Users/florian/.mozbuild/browsertime/ImageMagick-7.0.9/bin/convert --version
Version: ImageMagick 7.0.9-3 Q16 x86_64 2019-11-01 https://imagemagick.org

Comment 13

[Greg Mierzwinski [:sparky]]

:florian, can you see if it works after applying this patch? There was an issue with the ImageMagick version on mac: https://phabricator.services.mozilla.com/D52751

It looks like your error might be because of the version mismatch.

Comment 14

[Florian Quèze [:florian]]

(In reply to Greg Mierzwinski [:sparky] from comment #13)

:florian, can you see if it works after applying this patch? There was an issue with the ImageMagick version on mac: https://phabricator.services.mozilla.com/D52751

This doesn't seem to help. At the first run after applying the patch it downloaded another ImageMagick, but I still have the same failures: https://paste.mozilla.org/05woic8k

Comment 15

[Greg Mierzwinski [:sparky]]

Hmm, I can't reproduce this issue locally so this is hard for me to debug. I noticed from the log you posted that you are using ./mach browsertime --setup, can you try it with the following command: ./mach browsertime --setup --clobber

EDIT: Also, you can try deleting the /Users/name/.mozbuild/browsertime folder to see if that changes anything.

Comment 16

[Florian Quèze [:florian]]

Still the same result. Sorry :-|.

Comment 17

[Robert Wood [:rwood]]

I am also seeing this issue on OSX (10.13.6) with the latest central. With some debugging with :sparky on slack we see the 'convert' and 'compare' tools aren't found:

0:24.65 /Users/rwood/mozilla-central/obj-ff-dbg/_virtualenvs/init/bin/python /Users/rwood/mozilla-central/tools/browsertime/node_modules/browsertime/vendor/visualmetrics.py --check
ffmpeg: OK
convert: FAIL
compare: FAIL
Pillow: OK
SSIM: OK

Even though they do exist on my machine:

Roberts-MacBook-Pro-1927:bin rwood$ pwd
/Users/rwood/.mozbuild/browsertime/ImageMagick-7.0.8/bin
Roberts-MacBook-Pro-1927:bin rwood$ ls -al
total 48
drwxr-xr-x 18 rwood staff 576 4 Aug 2018 .
drwxr-xr-x 7 rwood staff 224 4 Aug 2018 ..
-rwxr-xr-x 1 rwood staff 1344 4 Aug 2018 Magick++-config
-rwxr-xr-x 1 rwood staff 1516 4 Aug 2018 MagickCore-config
-rwxr-xr-x 1 rwood staff 1323 4 Aug 2018 MagickWand-config
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 animate -> magick
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 compare -> magick
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 composite -> magick
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 conjure -> magick
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 convert -> magick
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 display -> magick
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 identify -> magick
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 import -> magick
-rwxr-xr-x 1 rwood staff 10648 4 Aug 2018 magick
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 magick-script -> magick
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 mogrify -> magick
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 montage -> magick
lrwxr-xr-x 1 rwood staff 6 20 Nov 16:26 stream -> magick

and are in the path:

RW append_env: {u'PYTHON': u'/Users/rwood/mozilla-central/obj-ff-dbg/_virtualenvs/init/bin/python', u'PATH': u'/Users/rwood/.mozbuild/node/bin:/Users/rwood/.mozbuild/browsertime/ffmpeg-4.1.1-macos64-static/bin:/Users/rwood/.mozbuild/browsertime/ImageMagick-7.0.8/bin', u'DYLD_LIBRARY_PATH': u'/Users/rwood/.mozbuild/browsertime/ImageMagick-7.0.8/lib', u'LD_LIBRARY_PATH': u'/Users/rwood/.mozbuild/browsertime/ImageMagick-7.0.8/lib', u'MAGICK_HOME': u'/Users/rwood/.mozbuild/browsertime/ImageMagick-7.0.8'}
0:05.14 /Users/rwood/mozilla-central/obj-ff-dbg/_virtualenvs/init/bin/python /Users/rwood/mozilla-central/tools/browsertime/node_modules/browsertime/vendor/visualmetrics.py --check