1695352 - perftest option browsertime-install-url does not appear to work

Status: RESOLVED WORKSFORME

(Testing :: mozperftest, defect, P3)

Description

[Andrew Creskey [:acreskey]]

It does not appear that the --browsertime-install-url option in mach perftest works.

e.g.

./mach perftest testing/performance/{some_test}.js  --browsertime-install-url https://github.com/sitespeedio/browsertime/tarball/cb1fa3aa2fcb5e6a916c9a6d0492cc872c40ef95

When running with --verbose I don't see any download or extract work.

Comment 1

[Andrew Creskey [:acreskey]]

Ahh, if I provide --browsertime-clobber then the install completes.

So maybe that's as intended?

Comment 2

[Andrew Creskey [:acreskey]]

(In reply to Andrew Creskey [:acreskey] [he/him] from comment #1)

Ahh, if I provide --browsertime-clobber then the install completes.

So maybe that's as intended?

This seems kind of clunky because the clobber/install workflow takes a long time.

I should be able to fix this.

Comment 3

[Andrew Creskey [:acreskey]]

(In reply to Andrew Creskey [:acreskey] [he/him] from comment #2)

(In reply to Andrew Creskey [:acreskey] [he/him] from comment #1)

Ahh, if I provide --browsertime-clobber then the install completes.

So maybe that's as intended?

This seems kind of clunky because the clobber/install workflow takes a long time.

I should be able to fix this.

I was wrong about the above.

Once you run with an install-url and a clobber, you can then run with just the install-url and it's quite performant.

I was thinking we should force-clobber when a custom URL is used, but that's quite slow.

Maybe I should just update the documentation to state that you should clobber when first changing the browsertime-url?
i.e.
https://searchfox.org/mozilla-central/rev/26330a08b1f9d06938faa0aa5e0f8c7a58064aa2/python/mozperftest/mozperftest/test/browsertime/runner.py#72

Comment 4

[Greg Mierzwinski [:sparky]]

Likely still an issue, and we should fix it. We'd like to move away from using ./mach browsertime for the install process. Something else to note is that since this bug was filed, we've made both mozperftest, and raptor use the same browsertime installation.

Comment 5

[Andrew Creskey [:acreskey]]

I'm personally happy to close this one as I don't think this is particularly useful anymore.

Comment 6

[Kash Shampur [:kshampur] ⌚EST]

since mozperftest and browsertime use the same browsertime installation, we can in theory just do the update with ./mach browsertime now. And we will eventually consolidate things into mozperftest (e.g. bug 1987461) there probably is little value in implementing this now.

so as per comment 4 and comment 6 i think it makes sense to close this, noting that this works with ./mach browsertime --update-upstream-url

Comment 7

[Kash Shampur [:kshampur] ⌚EST]

just to clarify ^ we will want to move away from mach browsertime as :sparky mentioned, but I think that is separate issue from this bug