1615686 - [Browsertime] Make sure --extension still works when migrating to upstream

Status: RESOLVED INVALID

(Testing :: Performance, enhancement, P3)

Description

[Sean Feng [:sefeng]]

Attachment: [e-mail]

Our browsertime uses a customized selenium driver which is not the same as the upstream. I found some inconsistencies between them when using --extension to test ublock for Fenix.

1. There's a bug in our selenium driver which makes it failed to install ublock for Fenix. The error message would be Could not find add-on ID. I wrote a patch to fix it https://gist.github.com/sefeng211/bb8c8ac9b58d19ee4ce520102136dd47. Once it is fixed, using --extension can successfully install ublock to Fenix (I attached the ublock xpi to this bug), and it would work as expected.

2. The above bug doesn't exist in the selenium driver that is being used in the upstream, however if you use the upstream to launch Fenix, the installed extension wouldn't work.

I think we need to figure out why it doesn't work in upstream and fix it.

To verify ublock is installed and is working
Launch browsertime with these 2 arguments --extension <path to the xpi> and --firefox.disableSafeBrowsing true, and visit https://ads-blocker.com/testing. You shouldn't see any ads on the page, and --firefox.disableSafeBrowsing true is required, otherwise the ads will be blocked by safe browsing.

Comment 1

[Greg Mierzwinski [:sparky]]

:sefeng, does this work on the latest browsertime version?

Comment 2

[Sean Feng [:sefeng]]

No, I still can't install extensions on Fenix by using --extension.

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I think this bug refers to a time when we were using a fork of browsertime. We archived our fork in February 2020, but the latest comment was after this time. @sefeng can you confirm if this is still an issue?

Comment 4

[Sean Feng [:sefeng]]

I think we can just close this bug given it's 3 years old.