Create an add-on hotfix to update the hotfix cert for Firefox 10-24

VERIFIED FIXED

Status

()

Firefox
General
VERIFIED FIXED
5 years ago
5 years ago

People

(Reporter: akeybl, Assigned: Felipe)

Tracking

unspecified
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox25 affected)

Details

(URL)

Attachments

(4 attachments, 3 obsolete attachments)

(Reporter)

Description

5 years ago
Assuming bug 870329 lands in FF24, we'll need to create an add-on hotfix to update the hotfix cert for Firefox 10-23 before October 2014.

Putting this on the tracking nom list because we don't have a longer term tracking process (yet).

Updated

5 years ago
Depends on: 803531
(Reporter)

Comment 1

5 years ago
(In reply to Alex Keybl [:akeybl] from comment #0)
> Assuming bug 870329 lands in FF24, we'll need to create an add-on hotfix to
> update the hotfix cert for Firefox 10-23 before October 2014.
> 
> Putting this on the tracking nom list because we don't have a longer term
> tracking process (yet).

I think I meant to say assuming bug 870329 *doesn't* land.
Adding needsinfo on :Mossop here to see if he can help with this request ?
Flags: needinfo?(dtownsend+bugmail)

Updated

5 years ago
status-firefox24: --- → affected
tracking-firefox24: ? → +
I don't have time to help to write this. It should be fairly simple though, just changing a pref. We've done it before, though I think it was mixed in with a bunch of other fixes too.
Flags: needinfo?(dtownsend+bugmail)
(Reporter)

Comment 4

5 years ago
Felipe, Gavin suggested you'd be able to help here.

bhearsum, when do we expect to have the new cert in hand to give to Felipe?
Assignee: nobody → felipc
Flags: needinfo?(bhearsum)
(Assignee)

Comment 5

5 years ago
Sure, I'll take it. There is one thing we need to decide: when we release this hotfix, users who have not yet received the previous hotfix will skip through it and just receive the most recent version, meaning that they will not get bug 812573.

We have two options here:
- if we believe that most users who were supposed to receive bug 812573 have already received it, we can call it not a problem and take the simpler approach to just ship the new hotfix to update the cert.
- otherwise, we can include in the new hotfix the code to do the two things at once for people who receive it: updating the cert for everyone and updating the update interval for fx10-16.

Code wise there's barely any difference for me as we can just reuse 99% the code from bug 812573, so the main difference would be in the release process. (mostly i'm thinking that the latter will require more testing, etc. ). So I'll wait for you to make the call
(In reply to Alex Keybl [:akeybl] from comment #4)
> Felipe, Gavin suggested you'd be able to help here.
> 
> bhearsum, when do we expect to have the new cert in hand to give to Felipe?

I'm not planning on renewing until September.
Flags: needinfo?(bhearsum)
:bhearsum, when does the current fingerprint expire ? We release Fx24 on Sep 17 and goto build with our release on Sep 9th with our final beta/release candidate.

I would ideally like to have the cert in Felipe's hand early enough in Sep so he has a few days in hand and can target Beta 9 which goes to build on Sep 5th . Does that sound feasibile ?
Flags: needinfo?(bhearsum)
(In reply to bhavana bajaj [:bajaj] from comment #7)
> :bhearsum, when does the current fingerprint expire ?

The current certificate expires on October 18th.

> I would ideally like to have the cert in Felipe's hand early enough in Sep
> so he has a few days in hand and can target Beta 9 which goes to build on
> Sep 5th . Does that sound feasibile ?

Why does this need to ship with the release? Even if it does, we need to release a hotfix with the updated fingerprints to make sure that users who don't update to 24 before October 18th receive the new fingerprints.

I don't think I can hit September 5th. I'm fully offline two days next week and as part of the renewal I'm planning to look at switching CAs, which I haven't done any research on yet.
Flags: needinfo?(bhearsum)
(Assignee)

Comment 9

5 years ago
Created attachment 795511 [details] [diff] [review]
Update cert fingerprint (w/ placeholder cert)

I created the hotfix with a placeholder cert so that we can already start the QA process whenever desired, and just swap in the string for the real cert when we get it.

It targets all platforms, 10.0 - 23.*

(I also updated the icons in the add-on with the new Firefox logo)


Regarding my previous question in comment 5, I think the last hotfix was released long enough that we don't need to merge the previous hotfix in here.


(In reply to Alex Keybl [:akeybl] from comment #0)
> Assuming bug 870329 lands in FF24, we'll need to create an add-on hotfix to
> update the hotfix cert for Firefox 10-23 before October 2014.

Just curious, since it seems we want to have the cert updated for FF24, did you mean before Oct `2013`?
Bhavana and I talked about this today to try and clear up confusion. She was particularly worried about our ability to ship a hotfix to users after the current cert expires (October 17th) but before Firefox 25 ships (October 29th). This is not an issue because we should be shipping out a hotfix with the updated fingerprints after I renew the cert (sometime mid-late september).

The reason I'm not pushing to do this renewal ahead of Firefox 24 is because if we do need to ship a hotfix between October 17th and October 29th, we would need to ship new fingerprints in a hotfix _anyways_, because our uptake rate on actual updates isn't fast enough to protect a large portion of our userbase.

So to be clear, these are my expectations:
* Mid/late September I will renew our Authenticode certificate and provide new fingerprints that Firefox should use to validate updates.
* Somebody needs to update the fingerprints in our repositories (central, aurora, beta, release, esr17, esr24).
* Somebody needs to finish creating the hotfix that updates the fingerprints for existing Firefox users.
* I will sign the hotfix that updates the fingerprints.
* Somebody needs to ship the hotfix.
* Any releases generated after the certificate has been renewed (mid/late cycle 25.0 betas, 25.0 release, any point releases that may happen between renewal and 25.0 being shipped) will be signed by the renewed certificate.

I hope this clears things up, and please let me know if you have any questions or concerns!
(Assignee)

Comment 11

5 years ago
Created attachment 796750 [details]
hotfix-v20130826.01.xpi

Uploading the built hotfix for testing
(Assignee)

Comment 12

5 years ago
Created attachment 796761 [details] [diff] [review]
Update cert fingerprint (w/ placeholder cert), 10.0 - 24.*

Just putting it on the radar. Bumped max version to 24.* given comment 10.
Attachment #795511 - Attachment is obsolete: true
Attachment #796761 - Flags: review?(mnoorenberghe+bmo)
(Assignee)

Comment 13

5 years ago
Created attachment 796765 [details]
hotfix-v20130826.01.xpi, 10.0 - 24.*

This xpi has maxVersion bumped to 24.*
Attachment #796750 - Attachment is obsolete: true

Updated

5 years ago
status-firefox24: affected → ---
status-firefox25: --- → affected
tracking-firefox24: + → ---
tracking-firefox25: --- → +
(Assignee)

Updated

5 years ago
Attachment #796761 - Flags: review?(bmcbride)
Comment on attachment 796761 [details] [diff] [review]
Update cert fingerprint (w/ placeholder cert), 10.0 - 24.*

Review of attachment 796761 [details] [diff] [review]:
-----------------------------------------------------------------

::: README
@@ +28,5 @@
>                 it in the future, it needs to have its version id (folder name) bumped.
>  v20130322.01 - Bug 812573 - Decrease the update interval for pre-17 users of beta, esr
>                 and release channels using mozilla.org servers in all platforms.
> +v20130826.01 - Bug 874513 - A hotfix to update the hotfix certificate fingerprint,
> +               rolled out to 10.0 - 23.*.

Version here needs updated.
Attachment #796761 - Flags: review?(mnoorenberghe+bmo)
Attachment #796761 - Flags: review?(bmcbride)
Attachment #796761 - Flags: review+
New fingerprints are: 91:53:98:0C:C1:86:DF:47:8F:35:22:9E:11:C9:A7:31:04:49:A1:AA

I'm updating the necessary repositories as part of bug 803531, ftr.
(Assignee)

Comment 16

5 years ago
Created attachment 807291 [details] [diff] [review]
Update cert fingerprint (w/ correct cert), 10.0 - 24.*

Patch pushed to the repo, using the fingerprint from comment 15 here (and bug 803531)
Attachment #807291 - Flags: review+
(Assignee)

Comment 17

5 years ago
Created attachment 807295 [details]
hotfix-v20130826.01.xpi, 10.0 - 24.*, unsigned

Unsigned hotfix. It can also be generated by running:

HOTFIX=v20130826.01 make package

in the root of the hotfixes repo
Attachment #796765 - Attachment is obsolete: true
(Assignee)

Comment 18

5 years ago
https://hg.mozilla.org/releases/firefox-hotfixes/rev/40a29960eaef
Status: NEW → RESOLVED
Last Resolved: 5 years ago
OS: Mac OS X → All
Hardware: x86 → All
Resolution: --- → FIXED
Summary: Create an add-on hotfix to update the hotfix cert for Firefox 10-23 → Create an add-on hotfix to update the hotfix cert for Firefox 10-24
Created attachment 807320 [details]
hotfix signed with old keys

Here's a copy of the hotfix signed with the _old_ keys. Who's responsible for shipping this out? Is this happening as part of this bug, or elsewhere?
Flags: needinfo?(felipc)
(Assignee)

Comment 20

5 years ago
I think the release can be handled in this bug (or a different one if someone prefers), but the QA we need to do first should happen on a separate bug (this was asked by QA the last time we did a hotfix). So I'll file a bug for that.

Jorge: can you upload this signed hotfix to the staging server, like you did in bug 865873 comment 3?
Flags: needinfo?(felipc)
I don't care if the QA is done here or not, I just need clear instructions with steps to follow and expected results.  Something along the lines of https://wiki.mozilla.org/QA/Desupport_Hotfix_Testing#Instructions_to_test_a_hotfix would be very helpful.  I'm not sure what this hotfix is doing, so can't say if that sort of plan is overkill or not.
The new version is up on staging now. It should take an hour or so for it to be fully available.
Please explain how to confirm this hotfix does what it is supposed to, then I'll verify against stage.
I used the instructions at https://wiki.mozilla.org/QA/Desupport_Hotfix_Testing#Installing_Hotfix to check for the hotfix against stage.

I assume I'm looking for extensions.hotfix.certs.1.sha1Fingerprint;CA:C4:7D:BF:63:4D:24:E9:DC:93:07:2F:E3:C8:EA:6D:C3:94:6E:89 to be updated to the new fingerprint, in addition to the hotfix add-on presence in add-ons manager.

No change and no hotfix add-on. In logs,I am getting:

[09:47:30.879] LOG addons.updates: Requesting https://addons-dev.allizom.org/update/VersionCheck.php?reqVersion=2&id=firefox-hotfix@mozilla.org&version=&maxAppVersion=%ITEM_MAXAPPVERSION%&status=userEnabled,incompatible&appID={ec8030f7-c20a-464f-9b0e-13a3a9e97384}&appVersion=24.0&appOS=Darwin&appABI=x86_64-gcc3&locale=en-US&currentAppVersion=24.0&updateType=%UPDATE_TYPE%&compatMode=%COMPATIBILITY_MODE%
[09:47:30.938] GET https://addons-dev.allizom.org/update/VersionCheck.php?reqVersion=2&id=firefox-hotfix@mozilla.org&version=&maxAppVersion=%ITEM_MAXAPPVERSION%&status=userEnabled,incompatible&appID={ec8030f7-c20a-464f-9b0e-13a3a9e97384}&appVersion=24.0&appOS=Darwin&appABI=x86_64-gcc3&locale=en-US&currentAppVersion=24.0&updateType=%UPDATE_TYPE%&compatMode=%COMPATIBILITY_MODE% [HTTP/1.1 301 Moved Permanently 196ms]
[09:47:30.892] LOG addons.repository: Requesting https://services.addons.mozilla.org/en-US/firefox/api/1.5/search/guid:%7B972ce4c6-7e08-4474-a285-3208198ce6fd%7D?src=firefox&appOS=Darwin&appVersion=24.0&tMain=22&tFirstPaint=730&tSessionRestored=755
[09:47:30.940] GET https://services.addons.mozilla.org/en-US/firefox/api/1.5/search/guid:%7B972ce4c6-7e08-4474-a285-3208198ce6fd%7D?src=firefox&appOS=Darwin&appVersion=24.0&tMain=22&tFirstPaint=730&tSessionRestored=755 [HTTP/1.1 200 OK 153ms]
[09:47:31.137] GET https://versioncheck.addons.mozilla.org/update/VersionCheck.php?reqVersion=2&id=firefox-hotfix@mozilla.org&version=&maxAppVersion=%ITEM_MAXAPPVERSION%&status=userEnabled,incompatible&appID={ec8030f7-c20a-464f-9b0e-13a3a9e97384}&appVersion=24.0&appOS=Darwin&appABI=x86_64-gcc3&locale=en-US&currentAppVersion=24.0&updateType=%UPDATE_TYPE%&compatMode=%COMPATIBILITY_MODE% [HTTP/1.1 200 OK 248ms]
[09:47:31.111] LOG addons.updates: Requesting https://addons-dev.allizom.org/update/VersionCheck.php?reqVersion=2&id={972ce4c6-7e08-4474-a285-3208198ce6fd}&version=24.0&maxAppVersion=24.0&status=userEnabled&appID={ec8030f7-c20a-464f-9b0e-13a3a9e97384}&appVersion=24.0&appOS=Darwin&appABI=x86_64-gcc3&locale=en-US&currentAppVersion=24.0&updateType=112&compatMode=normal
[09:47:31.138] GET https://addons-dev.allizom.org/update/VersionCheck.php?reqVersion=2&id={972ce4c6-7e08-4474-a285-3208198ce6fd}&version=24.0&maxAppVersion=24.0&status=userEnabled&appID={ec8030f7-c20a-464f-9b0e-13a3a9e97384}&appVersion=24.0&appOS=Darwin&appABI=x86_64-gcc3&locale=en-US&currentAppVersion=24.0&updateType=112&compatMode=normal [HTTP/1.1 301 Moved Permanently 65ms]
[09:47:31.235] GET https://versioncheck.addons.mozilla.org/update/VersionCheck.php?reqVersion=2&id={972ce4c6-7e08-4474-a285-3208198ce6fd}&version=24.0&maxAppVersion=24.0&status=userEnabled&appID={ec8030f7-c20a-464f-9b0e-13a3a9e97384}&appVersion=24.0&appOS=Darwin&appABI=x86_64-gcc3&locale=en-US&currentAppVersion=24.0&updateType=112&compatMode=normal [HTTP/1.1 200 OK 239ms]
[09:47:31.426] WARN addons.updates: Update manifest for {972ce4c6-7e08-4474-a285-3208198ce6fd} did not contain an updates property @ resource://gre/modules/AddonUpdateChecker.jsm:312
(Assignee)

Comment 25

5 years ago
It seems that the request for the hotfix update in the staging server is returning as no update available. I'm currently investigating if the request is not correct or if it's a cache problem in the staging server.

Meanwhile, you can test the add-on manually on different versions by downloading the signed xpi and drag&dropping it into Firefox. You should see a message that the add-on was installed, the sha1Fingerprint pref updated, and no traces of the hotfix afterwards as it will uninstall itself immediatelly.
(If you have the add-ons manager tab open while doing it it's possible that it won't disappear from the list until you close the tab and re-open it)

The manual testing won't update the extensions.hotfix.lastVersion pref
ok, manual check confirms the add-on updated the fingerprint to 91:53:98:0C:C1:86:DF:47:8F:35:22:9E:11:C9:A7:31:04:49:A1:AA.

let me know when you want me to try against stage again.
(Assignee)

Updated

5 years ago
Depends on: 919033
This looks good to go to production.
The new hotfix version is now live in prod.
Thanks everyone!
Was chasing a topcrash issue yesterday. A day late to verify here, but this looks good live with a few spot checks.
Status: RESOLVED → VERIFIED
(Assignee)

Comment 31

5 years ago
I created a page on MDN to document the hotfix process and testing for easier future reference, since the info was scattered through some bugs and wiki pages:
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/Hotfix

I also checked today a friend's computer who is running Firefox 24 release, and they have already received this hotfix.
(Reporter)

Updated

5 years ago
tracking-firefox25: + → ---
You need to log in before you can comment on or make changes to this bug.