1414404 - Prepare a system add-on to roll TP pref UI back to a single checkbox

Status: VERIFIED FIXED

(Firefox :: Protections UI, enhancement, P1)

Description

[François Marier [:francois]]

We are going to expose the full TP preferences in bug 1393627.

In order to have an easy way to undo this on release, we will use a system add-on which will be ready to go if we need it.

Comment 1

[François Marier [:francois]]

Attachment: tp-pref-rollback.xpi

I haven't been able to test this add-on on Beta yet due to problems with unsigned legacy add-ons not loading on unbranded builds. Maybe it needs to be signed before we can test it.

Comment 2

[Nicole Yee (:nicoleyee)]

Jorge,

Can you please advise how we should proceed with trying to test our add-on?

Comment 3

[François Marier [:francois]]

Mike, do you think it would be possible to get the add-on signed early so I can test it on a branded Beta?

Comment 4

[Osmose [:osmose, :mkelly]]

Redirecting to rdalal, since I'm not really working directly on system add-on stuff anymore.

You should be able to test on DevEdition anyway, which in most cases should be the same version as Beta nowadays.

Comment 5

[Jorge Villalobos [:jorgev] (he/him)]

If you just need it signed in some way, you can upload it to AMO as self-hosted. See https://developer.mozilla.org/en-US/Add-ons/Distribution

Keep in mind that AMO doesn't support version string duplication, so if you wanted to submit the same add-on to be hosted, you would need to use a new version string. Since this is a system add-on, I don't think you'll need this.

If the AMO signature doesn't work for your testing, I don't know other ways around it. The Beta+system+legacy+unsigned edge case combination is pretty rough.

Comment 6

[Rehan Dalal (Inactive)]

Jason Thomas (:jason) should be able to sign this for you if you need it signed for testing.

Comment 7

[Jorge Villalobos [:jorgev] (he/him)]

> If you just need it signed in some way, you can upload it to AMO as self-hosted.

Never mind about this, just realized it won't work since legacy add-ons will only load post-57 if they have the internal signature.

Comment 8

[François Marier [:francois]]

(In reply to Michael Kelly [:mkelly,:Osmose] from comment #4)
> You should be able to test on DevEdition anyway, which in most cases should
> be the same version as Beta nowadays.

Indeed and it works. Thanks Mike!

Comment 9

[François Marier [:francois]]

Jason, could we get the attached add-on signed using the system add-on key?

I'd like to test it properly on beta to make sure it works when installed permanently.

Comment 10

[Jason Thomas [:jason]]

Attachment: [e-mail] (signed)

Please see attached.

Comment 11

[François Marier [:francois]]

Thanks Jason!

It looks like having it signed is not enough to be able to install a bootstrap add-on on official Beta builds :(

Comment 12

[Andy McKay]

Is that the system add-on signer, or the AMO one, it says "Mozilla AMO Production Signing Service, CN=root-ca-production-amo" in the META-INF/mozilla.rsa.

Comment 13

[Hector Zhao [:hectorz]]

Hi :francois, I suspect the extension won't work as expected for funnelcake or partner builds (one of which is the Fx China repack distributed by Beijing office) due to bug 947838. The existence of distribution.ini within them will trigger a "reload-default-prefs" notification, and invalidates any change made to an existing preference on default branch before it.

Comment 14

[Mark Banner (:standard8)]

I've just left a couple of comments in issues:

https://github.com/mozilla/tp-pref-rollback/issues/1 - unnecessary version checks
https://github.com/mozilla/tp-pref-rollback/issues/2 - unnecessary actions on shutdown

Comment 15

[Mark Banner (:standard8)]

Francois: You might be able to get the add-on to work by dropping it into the browser/features/ directory with the application directory. That might get overridden by updates if the update cycle kicks in though.

Comment 16

[Robert Helmer [:rhelmer]]

(In reply to Mark Banner (:standard8) from comment #15)
> Francois: You might be able to get the add-on to work by dropping it into
> the browser/features/ directory with the application directory. That might
> get overridden by updates if the update cycle kicks in though.

This won't work anymore (on purpose, see bug 1348981)

I linked francois to https://gist.github.com/rhelmer/b392e97d0b6ff81c06c205c760050c2b which simulates the way updates work (still requires a signed add-on).

Comment 17

[François Marier [:francois]]

I successfully tested attachment 8925182 [details] (tp-pref-rollback 1.0.0) in the official 57RC1 that will be sent to the beta channel tomorrow: https://archive.mozilla.org/pub/firefox/candidates/57.0-candidates/build1/linux-x86_64/en-US/

I also confirmed that it fails to work in partner builds as pointed out in comment 13.

Here are the steps I followed:

1. Install official 57RC build.
2. Create a new profile.
3. Confirm that the tracking protection setting is tri-state in about:preferences#privacy
4. Go into about:config and set:

    extensions.logging.enabled = true
    devtools.chrome.enabled = true
    extensions.systemAddon.update.url = https://fmarier.org/gofaster/update.xml

5. Open the Browser Console (Tools | Web Developer | Browser Console).
6. Type the following in the command-line at the bottom and press enter:

    Cu.import("resource://gre/modules/AddonManager.jsm"); AddonManagerPrivate.backgroundUpdateCheck()
7. Reload about:preferences#privacy and confirm that the tracking protection setting is now just a checkbox.
8. Open about:support and check that the "Tracking Protection preference rollback 1.0.0" add-on is listed under "Firefox Features".
9. Restart browser
10. Open about:preferences#privacy and confirm that the tracking protection setting is still just a checkbox.
11. Open about:support and check that the "Tracking Protection preference rollback 1.0.0" add-on is still listed under "Firefox Features".
12. Switch to an official Firefox 58 build.
13. Start browser with the same profile.
14. Open about:preferences#privacy and confirm that the tracking protection setting is back to a tri-state one.
15. Open about:support and check that the "Tracking Protection preference rollback 1.0.0" add-on is gone from "Firefox Features".

Tomorrow, I will make a new version of the add-on which should make it work with partner builds too. However, we do have something that works (and could be verified by SoftVision) if we only care about rolling back the TP pref UI in official builds.

Comment 18

[François Marier [:francois]]

Attachment: [e-mail]

New version of the add-on which will hopefully work on distro/partner builds: https://github.com/mozilla/tp-pref-rollback/commit/231aaf182cf9a5e5e4a1a11eaba4511eb7d4b5d3

Jason, could you please sign this version so that I can test it on beta?

Comment 19

[Jason Thomas [:jason]]

:wezhou can you sign this add-on?

Comment 20

[François Marier [:francois]]

(In reply to Jason Thomas [:jason] from comment #19)
> :wezhou can you sign this add-on?

Version 1.1.0 (attachment 8926120 [details]) is the one that needs to be signed.

Comment 21

[:wezhou]

Attachment: [e-mail] (signed)

Signed file attached. Please test.

Comment 22

[François Marier [:francois]]

Attachment: [e-mail]

Version 1.1.0 worked fine in distro/partner builds but it didn't flip the pref at install time.

Here's a new version (1.1.1) which sets the default pref twice to handle both cases: https://github.com/mozilla/tp-pref-rollback/commit/d354864ad86fb96be236b96bd23e555492f35e3f

wezhou, could you please sign this one as well?

Comment 23

[:wezhou]

Attachment: [e-mail] (signed)

Sure! Signed file attached. Please test.

Comment 24

[François Marier [:francois]]

Attachment: [e-mail]

Forgot to bump the version number in 1.1.1 :(

Here's 1.1.2 with only that change: https://github.com/mozilla/tp-pref-rollback/commit/dbbc04695e042d3359082e49428f92b98ba59c8a

Comment 25

[:wezhou]

Attachment: [e-mail] (signed)

Signed file attached. Please test.

Comment 26

[François Marier [:francois]]

Comment on attachment 8926145 [details]
[e-mail] (signed)

This version works in both official and partner/distro builds.

It's ready to be verified using the steps from comment 17.

Comment 27

[Brindusa Tot, DTE]

Hi Francois,

I could not install the add-on  on RC 57.0 with the steps described in comment 17, I get following error in browser console:

1510147853688	addons.xpi	ERROR	Failed to download system add-on [e-mail]: Error: Failed downloading via XHR, status: 0, reason: error (resource://gre/modules/addons/ProductAddonChecker.jsm:318:16) JS Stack trace: [e-mail]:318:16  Log.jsm:752
1510147853689	addons.manager	WARN	Failed to update system addons: Error: Rejecting updated system add-on set that either could not be downloaded or contained unusable add-ons. (resource://gre/modules/addons/XPIProvider.jsm:2720:13) JS Stack trace: [e-mail]:2720:13

I tried to install the add-on manually. Downloaded the attached file [e-mail] (signed) from comment 26 and try to install with the options "Install Add-on From File..." from the about:addons page, but I get the error: "Tracking Protection preference rollback could not be installed because it is not compatible with Firefox 57.0."

Francois, could you please take a look on this?

Comment 28

[François Marier [:francois]]

(In reply to Brindusa Tot[:brindusat] from comment #27)
> 1510147853688	addons.xpi	ERROR	Failed to download system add-on
> [e-mail]: Error: Failed downloading via XHR, status: 0,
> reason: error (resource://gre/modules/addons/ProductAddonChecker.jsm:318:16)
> JS Stack trace: [e-mail]:318:16  Log.jsm:752
> 1510147853689	addons.manager	WARN	Failed to update system addons: Error:
> Rejecting updated system add-on set that either could not be downloaded or
> contained unusable add-ons.
> (resource://gre/modules/addons/XPIProvider.jsm:2720:13) JS Stack trace:
> [e-mail]:2720:13

It turns out I had forgotten to update the download URL inside https://fmarier.org/gofaster/update.xml to point to my server at https://fmarier.org and it was still pointing to my local webserver on localhost:

<addon id="[e-mail]" URL="http://localhost/francois/gofaster/[e-mail]" ... version="1.1.2"/>

That's why it worked for me but wouldn't work for anybody else. Sorry about that.

It's now fixed:

<addon id="[e-mail]" URL="https://fmarier.org/gofaster/[e-mail]" ... version="1.1.2"/>

and so the steps in comment 17 should work now.

> I tried to install the add-on manually. Downloaded the attached file
> [e-mail] (signed) from comment 26 and try to
> install with the options "Install Add-on From File..." from the about:addons
> page, but I get the error: "Tracking Protection preference rollback could
> not be installed because it is not compatible with Firefox 57.0."

That part is expected. It's only possible to install the add-on if it's shipped via the system add-on mechanism (steps 4 to 6 in comment 17). It can't be installed manually.

Comment 29

[Hani Yacoub, Desktop QA]

Verified as fixed on Firefox 57.0 on Windows 10 x 64, Windows 7 x32, Mac OS X 10.12 and Ubuntu 16.04 x64.

I still have a couple of questions that I need to clarify.

After I switched to an official Firefox 58 build with the same profile that I used on Firefox 57.0 and then I went back to Firefox 57.0 the tracking protection setting is back to a tri-state, is that the desirable behavior in this case?
If it's the desirable behavior if I want to write the command-line from step 6 back in the console log should the tracking protection setting be switched back to checkbox?

Thanks.

Comment 30

[François Marier [:francois]]

(In reply to Hani Yacoub from comment #29)
> Verified as fixed on Firefox 57.0 on Windows 10 x 64, Windows 7 x32, Mac OS
> X 10.12 and Ubuntu 16.04 x64.

Excellent, thank you!

> After I switched to an official Firefox 58 build with the same profile that
> I used on Firefox 57.0 and then I went back to Firefox 57.0 the tracking
> protection setting is back to a tri-state, is that the desirable behavior in
> this case?

That's the correct behaviour. When the profile to upgraded to 58, the add-on will automatically uninstall itself and the preferences will go back to the tri-state option. Going back to 57 does not reinstall the add-on automatically with this test plan.

This would be different if we shipped this add-on for real. In that case, the add-on would be resent automatically to you when you switch back to 57. I don't think we can easily test that.

> If it's the desirable behavior if I want to write the command-line from step
> 6 back in the console log should the tracking protection setting be switched
> back to checkbox?

Yes. To be clear, this is what I would expect following from the steps in comment 17:

16. Switch back to an official Firefox 57 build.
17. Start browser with the same profile.
18. Open about:preferences#privacy and confirm that the tracking protection setting is still a tri-state one.
19. Open about:support and check that the "Tracking Protection preference rollback 1.0.0" add-on is still missing from "Firefox Features".
20. Open the Browser Console (Tools | Web Developer | Browser Console).
21. Type the following in the command-line at the bottom and press enter:

    Cu.import("resource://gre/modules/AddonManager.jsm"); AddonManagerPrivate.backgroundUpdateCheck()

22. Reload about:preferences#privacy and confirm that the tracking protection setting is now just a checkbox.
23. Open about:support and check that the "Tracking Protection preference rollback 1.0.0" add-on is listed under "Firefox Features".

Comment 31

[François Marier [:francois]]

Rehan, do we want to upload this to Balrog so that it's ready to go if we need it or is leaving it on this bug fine?

Note: It should NOT be sent to anybody. It's only meant to be used in case of emergency.

Comment 32

[Rehan Dalal (Inactive)]

I will create a "release" for this in Balrog but will hold off on creating a rule. This will allow us to ship it quickly if necessary.

Comment 33

[François Marier [:francois]]

Great. I think we are done here then.

Comment 34

[Hani Yacoub, Desktop QA]

I verified the rest of the steps and I can confirm that the tracking protection setting is now just a checkbox, and "Tracking Protection preference rollback 1.0.0" add-on is listed under "Firefox Features" in about:support.

I also verified on Firefox partner-repacks (Bing) 57.0 on Windows 10 x 64 and Mac OS X 10.12.
Should I verify on other partner builds?

Comment 35

[François Marier [:francois]]

(In reply to Hani Yacoub from comment #34)
> I verified the rest of the steps and I can confirm that the tracking
> protection setting is now just a checkbox, and "Tracking Protection
> preference rollback 1.0.0" add-on is listed under "Firefox Features" in
> about:support.
> 
> I also verified on Firefox partner-repacks (Bing) 57.0 on Windows 10 x 64
> and Mac OS X 10.12.
> Should I verify on other partner builds?

That's great. I think that's all we need.

Thanks for your help!