1607522 - ./mach browsertime --setup still shows some python packages are missing

Status: RESOLVED FIXED

(Testing :: Performance, defect, P1)

Description

[Sean Feng [:sefeng]]

➜  mozilla-central ./mach browsertime --setup --clobber
 0:00.31 Using artifact from local cache: /home/sefeng/.mozbuild/cache/browsertime/a560bb8c17bb72cc-ffmpeg-4.1.4-i686-static.tar.xz
 0:00.31 Unpacking temporary location /home/sefeng/.mozbuild/cache/browsertime/a560bb8c17bb72cc-ffmpeg-4.1.4-i686-static.tar.xz
 0:01.74 Installing browsertime node module from /home/sefeng/mozilla/mozilla-central/tools/browsertime/package.json
Clobbering /home/sefeng/mozilla/mozilla-central/tools/browsertime/node_modules...
Installing browsertime for mach using "/usr/bin/nodejs /home/sefeng/.mozbuild/node/bin/npm ci --loglevel=error"...
info sharp Using cached /home/sefeng/.npm/_libvips/libvips-8.7.4-linux-x64.tar.gz

browsertime installed successfully!

NOTE: Your local browsertime binary is at /home/sefeng/mozilla/mozilla-central/tools/browsertime/node_modules/.bin/browsertime

 0:13.22 /home/sefeng/mozilla/build/obj-sccache/_virtualenvs/init/bin/python /home/sefeng/mozilla/mozilla-central/tools/browsertime/node_modules/browsertime/vendor/visualmetrics.py --check
ffmpeg:   OK
convert:  OK
compare:  OK
Pillow:   FAIL
SSIM:     FAIL

Comment 1

[Tarek Ziadé (:tarek)]

I see that _activate_virtualenv() calls results are not checked.

Nick, any reason not to fail when _activate_virtualenv() returns False?

Comment 2

[Tarek Ziadé (:tarek)]

answered in a meeting : we should check that result and fail

Comment 3

[Tarek Ziadé (:tarek)]

Sean, can you try the following:

./mach bootstrap --no-system-changes
./mach browsertime --setup --clobber

works for me on Windows 10 and current mozilla-central

Comment 4

[Tarek Ziadé (:tarek)]

Attachment: Bug 1607522 - hard stop if Pillow or pyssim could not be installed

Let's stop if the packages could not be installed.
Also, let's skip the call if we find them.

Comment 5

[Sean Feng [:sefeng]]

Still the same error :(

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/08999096f418
hard stop if Pillow or pyssim could not be installed r=nalexander

Comment 7

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/08999096f418

Comment 8

[Sean Feng [:sefeng]]

So...this is still an issue for me. https://paste.mozilla.org/9LsA1f3K#L23

Adding import sys; raise Exception(sys.executable) before https://searchfox.org/mozilla-central/source/tools/browsertime/mach_commands.py#414 raises Exception: /usr/bin/python2.7 , which is the problem because it's not the python executable of the virtual env.

Comment 9

[Tarek Ziadé (:tarek)]

Attachment: Bug 1607522 - improve dependency detection

When a package is installed with "pip --user", the import
works right after the virtualenv activation.
Even if we force its install via mozbuild virtualenv
it will fail. This patch checks if the installed package is inside or
outside the virtualenv. If it's outside, it forces
its installation inside.

Notice that we might want to mvoe that logic in mozbuild/virtualenv.py

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d8b9fdbd8762
improve dependency detection r=sefeng,perftest-reviewers,sparky

Comment 11

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/d8b9fdbd8762

Comment 12

[BugBot [:suhaib / :marco/ :calixte]]

Change the status for beta to have the same as nightly and release.
For more information, please visit auto_nag documentation.