803596 - 10/22: Need to update hotfix cert fingerprint for FF10-16.0.1 through a hotfix

Status: RESOLVED FIXED

(Firefox :: General, defect, P1)

Description

[Alex Keybl [:akeybl]]

Sadly, this will need to be combined with the 10.5 EOL hotfix (bug 774509) and the update bustage (bug 790096).

Comment 1

[Alex Keybl [:akeybl]]

new cert in bug 803583

Comment 2

[Alex Keybl [:akeybl]]

We should target 10/22 for pushing out this hotfix, so that as many users as possible will be able to get hotfixes in the future.

Assigning juanb as the QA contact (feel free to reassign as appropriate).

Comment 3

[[e-mail] (:bhearsum)]

Thanks for the CC. Just to avoid confusion: as far as I know I've provided everything necessary to make this happen. Other than signing the hotfix upon request, I don't know of anything else I'm need for. Let me know if that's not the case.

Comment 4

[Dave Townsend [:mossop]]

I'm spinning up a test build now to verify a patch for bug 803583 and that a hotfix can modify the expected cert after installation.

Comment 5

[Dave Townsend [:mossop]]

I've verified the new cert fingerprint and that we can update the expected cert in a hotfix. What range of Firefox versions do we want to push this out to? I'm tempted to say everything <17

Comment 6

[Alex Keybl [:akeybl]]

(In reply to Dave Townsend (:Mossop) from comment #5)
> I've verified the new cert fingerprint and that we can update the expected
> cert in a hotfix. What range of Firefox versions do we want to push this out
> to? I'm tempted to say everything <17

Sounds great to me, no issue there.

Comment 7

[Dave Townsend [:mossop]]

Attachment: patch rev 1

This copies the hotfix code from bug 774509 and adds code from the hotfix in bug 790096 into it as well as setting the new hotfix cert fingerprint.

It is complicated by the fact we target two different version and platform combinations for each case. Here is the matrix I coded against and have manualyl verifies for myself: https://etherpad.mozilla.org/GWJ08ni65B

Note I leave the hotfix installed for <FF16 when on unsupported OSX so if users update from lower versions they will start seeing the prompts.

Comment 8

[Dave Townsend [:mossop]]

Attachment: Built hotfix, unsigned

This is a built version of the hotfix, just the patch above plus a name change discussed in email.

Comment 9

[Matthew N. [:MattN]]

Comment on attachment 673458 [details] [diff] [review]
patch rev 1

Review of attachment 673458 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with the fixes.

::: v20120817.01/bootstrap.js
@@ +16,5 @@
>  
>  function install(data, reason) {
> +  // Always update the hotfix cert fingerprint
> +  Services.prefs.setCharPref("extensions.hotfix.certs.1.sha1Fingerprint",
> +                             "CA:C4:7D:BF:63:4D:24:E9:DC:93:07:2F:E3:C8:EA:6D:C3:94:6E:89");

I think we should |Services.prefs.savePrefFile(null);| here (like below) in case there is a crash before the next save.

@@ +22,2 @@
>    // If false was returned then the hotfix was uninstalled so don't enable.
> +  if (shouldUninstallHotfix()) {

Nit: outdated comment here

@@ +38,5 @@
>  }
>  
>  function startup(data, reason) {
>    // If false was returned then the hotfix was uninstalled.
> +  if (shouldUninstallHotfix()) {

…and here (my fault)

@@ +44,5 @@
>      return;
>    }
>  
> +  // Only attempt to fix background updates in Firefox 15
> +  if (Services.vc.compare(Services.appinfo.version, "15.0a1") >= 0 ||

I think you mean &&.

Doesn't this mean we're going to leave users stranded on 16 (beta) - 18 (nightly) who were affected by the update issue? Bug 790096 targeted 15.0a1 - 18.0a1.

@@ +62,5 @@
> +    uninstallHotfix(data);
> +    return;
> +  }
> +
> +  if (shouldShowBillboardWarning()) {

Can't the uninstall block above move to an else block below this condition? The condition of the if block above seems to be checking a subset of shouldShowBillboardWarning. It makes it more clear to me that either a billboard may be shown or the hotfix will be uninstalled (and not neither).

::: v20120817.01/install.rdf
@@ +22,1 @@
>          <em:maxVersion>16.*</em:maxVersion>

Note: As discussed on IRC, the hotfix won't install on Firefox 16.0.2 and higher because the hotfix will be signed by the old cert but 16.0.2 will require the new cert.

Comment 10

[Matthew N. [:MattN]]

Comment on attachment 673458 [details] [diff] [review]
patch rev 1

Review of attachment 673458 [details] [diff] [review]:
-----------------------------------------------------------------

::: v20120817.01/bootstrap.js
@@ +62,5 @@
> +    uninstallHotfix(data);
> +    return;
> +  }
> +
> +  if (shouldShowBillboardWarning()) {

Nevermind this comment, we already discussed this and you mentioned in comment 7 why you're doing this:

(Quoting Dave Townsend (:Mossop) from comment #7)
> Note I leave the hotfix installed for <FF16 when on unsupported OSX so if
> users update from lower versions they will start seeing the prompts.

Comment 12

[Dave Townsend [:mossop]]

(In reply to Matthew N. [:MattN] from comment #9)
> Comment on attachment 673458 [details] [diff] [review]
> patch rev 1
> 
> Review of attachment 673458 [details] [diff] [review]:
> -----------------------------------------------------------------
> @@ +44,5 @@
> >      return;
> >    }
> >  
> > +  // Only attempt to fix background updates in Firefox 15
> > +  if (Services.vc.compare(Services.appinfo.version, "15.0a1") >= 0 ||
> 
> I think you mean &&.
> 
> Doesn't this mean we're going to leave users stranded on 16 (beta) - 18
> (nightly) who were affected by the update issue? Bug 790096 targeted 15.0a1
> - 18.0a1.

Let's discuss this with the release-drivers by email

Comment 13

[Dave Townsend [:mossop]]

Attachment: patch rev 2

Updated patch, I can't land it right now for some reason

Comment 14

[Dave Townsend [:mossop]]

Attachment: unsigned xpi

Comment 15

[juan becerra [:juanb]]

After verifying the signed hotfix xpi I tested it on staging, but there seems to be a problem in that the hotfix doesn't get uninstalled immediately. This is unlike testing the hotfix by itself (not staged). This is the error message I get in the console:

Timestamp: 10/22/2012 12:29:21 PM
Error: ERROR addons.xpi: Failed to remove file C:\Documents and Settings\mozilla\Application Data\Mozilla\Firefox\Profiles\5ycd5csu.default\extensions\trash\[e-mail]: [Exception... "Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIFile.remove]"  nsresult: "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame :: resource:///modules/XPIProvider.jsm :: recursiveRemove :: line 1299"  data: no]
Source File: resource:///modules/XPIProvider.jsm
Line: 1299

Dave is investigating.

Comment 16

[Dave Townsend [:mossop]]

(In reply to juan becerra [:juanb] from comment #15)
> After verifying the signed hotfix xpi I tested it on staging, but there
> seems to be a problem in that the hotfix doesn't get uninstalled
> immediately. This is unlike testing the hotfix by itself (not staged). This
> is the error message I get in the console:
> 
> Timestamp: 10/22/2012 12:29:21 PM
> Error: ERROR addons.xpi: Failed to remove file C:\Documents and
> Settings\mozilla\Application
> Data\Mozilla\Firefox\Profiles\5ycd5csu.default\extensions\trash\firefox-
> [e-mail]: [Exception... "Component returned failure code:
> 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIFile.remove]"  nsresult:
> "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame ::
> resource:///modules/XPIProvider.jsm :: recursiveRemove :: line 1299"  data:
> no]
> Source File: resource:///modules/XPIProvider.jsm
> Line: 1299
> 
> Dave is investigating.

I can't reproduce this on my machine and juan confirmed that even though the hotfix doesn't go away immediately it does update the cert correctly and does then uninstall after restarting Firefox.

That's not ideal but I think it isn't bad enough to block us releasing today.

Comment 17

[Dave Townsend [:mossop]]

(In reply to Dave Townsend (:Mossop) from comment #16)
> (In reply to juan becerra [:juanb] from comment #15)
> > After verifying the signed hotfix xpi I tested it on staging, but there
> > seems to be a problem in that the hotfix doesn't get uninstalled
> > immediately. This is unlike testing the hotfix by itself (not staged). This
> > is the error message I get in the console:
> > 
> > Timestamp: 10/22/2012 12:29:21 PM
> > Error: ERROR addons.xpi: Failed to remove file C:\Documents and
> > Settings\mozilla\Application
> > Data\Mozilla\Firefox\Profiles\5ycd5csu.default\extensions\trash\firefox-
> > [e-mail]: [Exception... "Component returned failure code:
> > 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIFile.remove]"  nsresult:
> > "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame ::
> > resource:///modules/XPIProvider.jsm :: recursiveRemove :: line 1299"  data:
> > no]
> > Source File: resource:///modules/XPIProvider.jsm
> > Line: 1299
> > 
> > Dave is investigating.
> 
> I can't reproduce this on my machine and juan confirmed that even though the
> hotfix doesn't go away immediately it does update the cert correctly and
> does then uninstall after restarting Firefox.
> 
> That's not ideal but I think it isn't bad enough to block us releasing today.

I can reproduce it now, it should only be on windows and is only when you have the add-ons manager open at the same time as the hotfix installs. I'm pretty sure we saw this with a previous hotfix and decided it was trivial enough to ship anyway but I can't seem to find a record of that :(

Comment 18

[juan becerra [:juanb]]

(In reply to Dave Townsend (:Mossop) from comment #17)
> 
> I can reproduce it now, it should only be on windows and is only when you
> have the add-ons manager open at the same time as the hotfix installs. I'm
> pretty sure we saw this with a previous hotfix and decided it was trivial
> enough to ship anyway but I can't seem to find a record of that :(

Maybe it's what Matt referred to in https://bugzilla.mozilla.org/show_bug.cgi?id=790096#c20

Comment 19

[Dave Townsend [:mossop]]

(In reply to juan becerra [:juanb] from comment #18)
> (In reply to Dave Townsend (:Mossop) from comment #17)
> > 
> > I can reproduce it now, it should only be on windows and is only when you
> > have the add-ons manager open at the same time as the hotfix installs. I'm
> > pretty sure we saw this with a previous hotfix and decided it was trivial
> > enough to ship anyway but I can't seem to find a record of that :(
> 
> Maybe it's what Matt referred to in
> https://bugzilla.mozilla.org/show_bug.cgi?id=790096#c20

Yeah that's it I think

Comment 20

[juan becerra [:juanb]]

I've verified this on staging making sure the certificate fingerprint value is updated in the preferences when the hotfix is downloaded in builds 16.0.1 and below.

I've also made sure this hotfix still addresses bug 774509 by testing the behavior on Mac OS 10.5 where the hotfix gets updated and the billboard is presented.

In addition, I've made sure bug 790096 is also addressed by this, testing on Mac and Linux and checking to make sure the preferences are toggled.

Comment 21

[juan becerra [:juanb]]

This is in production and verified, including bug 774509 and bug 790096.

Comment 22

[bhavana bajaj [:bajaj]]

(In reply to Dave Townsend (:Mossop) from comment #17)
> (In reply to Dave Townsend (:Mossop) from comment #16)
> > (In reply to juan becerra [:juanb] from comment #15)
> > > After verifying the signed hotfix xpi I tested it on staging, but there
> > > seems to be a problem in that the hotfix doesn't get uninstalled
> > > immediately. This is unlike testing the hotfix by itself (not staged). This
> > > is the error message I get in the console:
> > > 
> > > Timestamp: 10/22/2012 12:29:21 PM
> > > Error: ERROR addons.xpi: Failed to remove file C:\Documents and
> > > Settings\mozilla\Application
> > > Data\Mozilla\Firefox\Profiles\5ycd5csu.default\extensions\trash\firefox-
> > > [e-mail]: [Exception... "Component returned failure code:
> > > 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIFile.remove]"  nsresult:
> > > "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame ::
> > > resource:///modules/XPIProvider.jsm :: recursiveRemove :: line 1299"  data:
> > > no]
> > > Source File: resource:///modules/XPIProvider.jsm
> > > Line: 1299
> > > 
> > > Dave is investigating.
> > 
> > I can't reproduce this on my machine and juan confirmed that even though the
> > hotfix doesn't go away immediately it does update the cert correctly and
> > does then uninstall after restarting Firefox.
> > 
> > That's not ideal but I think it isn't bad enough to block us releasing today.
> 
> I can reproduce it now, it should only be on windows and is only when you
> have the add-ons manager open at the same time as the hotfix installs. I'm
> pretty sure we saw this with a previous hotfix and decided it was trivial
> enough to ship anyway but I can't seem to find a record of that :(

Dave, can you please pass on the bug# which may be on file already as per our offline conversation causing this issue ?

Comment 23

[Dave Townsend [:mossop]]

(In reply to bhavana bajaj [:bajaj] from comment #22)
> (In reply to Dave Townsend (:Mossop) from comment #17)
> > (In reply to Dave Townsend (:Mossop) from comment #16)
> > > (In reply to juan becerra [:juanb] from comment #15)
> > > > After verifying the signed hotfix xpi I tested it on staging, but there
> > > > seems to be a problem in that the hotfix doesn't get uninstalled
> > > > immediately. This is unlike testing the hotfix by itself (not staged). This
> > > > is the error message I get in the console:
> > > > 
> > > > Timestamp: 10/22/2012 12:29:21 PM
> > > > Error: ERROR addons.xpi: Failed to remove file C:\Documents and
> > > > Settings\mozilla\Application
> > > > Data\Mozilla\Firefox\Profiles\5ycd5csu.default\extensions\trash\firefox-
> > > > [e-mail]: [Exception... "Component returned failure code:
> > > > 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIFile.remove]"  nsresult:
> > > > "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame ::
> > > > resource:///modules/XPIProvider.jsm :: recursiveRemove :: line 1299"  data:
> > > > no]
> > > > Source File: resource:///modules/XPIProvider.jsm
> > > > Line: 1299
> > > > 
> > > > Dave is investigating.
> > > 
> > > I can't reproduce this on my machine and juan confirmed that even though the
> > > hotfix doesn't go away immediately it does update the cert correctly and
> > > does then uninstall after restarting Firefox.
> > > 
> > > That's not ideal but I think it isn't bad enough to block us releasing today.
> > 
> > I can reproduce it now, it should only be on windows and is only when you
> > have the add-ons manager open at the same time as the hotfix installs. I'm
> > pretty sure we saw this with a previous hotfix and decided it was trivial
> > enough to ship anyway but I can't seem to find a record of that :(
> 
> Dave, can you please pass on the bug# which may be on file already as per
> our offline conversation causing this issue ?

Bug 727398 was what we came up with, but it was meant to be fixed for 14. If juan can still reproduce it we should probably file a new bug.