Closed Bug 1261333 Opened 9 years ago Closed 9 years ago

Error during blocklist pull from addons-dev.allizom.org

Categories

(Toolkit :: Blocklist Policy Requests, defect)

Product:
Toolkit
Component:
Blocklist Policy Requests
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED

People

(Reporter: eviljeff, Assigned: tarek)

References

Details

+++ This bug was initially created as a clone of Bug #1259458 +++ copied from bug 1259458#c9 I quickly went through a spot check using the latest versions of fx45.0.1, fx46.0, fx47.0 and fx48.0 and didn't run into any performance or timeout issues. The blocklist was pulled in pretty quickly without the 2min delay that I was noticing yesterday. However, I'm still seeing the following error in the browser console every time I pull in the blocklist from the -dev staging server. Receiving this error on the latest m-a, m-b and m-r but NOT under m-c: Blocklist::_handleCertItemNode: Error adding revoked cert by Issuer and Serial[Exception... "Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsICertBlocklist.revokeCertByIssuerAndSerial]" nsresult: "0x80070057 (NS_ERROR_ILLEGAL_VALUE)" location: "JS frame :: resource://gre/components/nsBlocklistService.js :: Blocklist.prototype._handleCertItemNode :: line 958" data: no]
Mossop, could this be a bug in the Add-ons Manager that was recently fixed?
Flags: needinfo?(dtownsend)
I'm not seeing anything on the blocklist service side, maybe this is something to do with the switch to kinto?
Flags: needinfo?(dtownsend) → needinfo?(mgoodwin)
(To our knowledge) the blocklist being pulled currently doesn't use kinto, if that helps. Its all still set and served via AMO.
I've added STR as they weren't added in bug # 1259458 either. It appears like m-c is the only channel that's not displaying the error message in the browser console. * latest m-c using BuildID: 20160404030231 --> NOT Reproducible * latest m-a using BuildID: 20160404004026 --> Reproducible * latest m-b using BuildID: 20160401021843 --> Reproducible * latest m-r using BuildID: 20160315153207 --> Reproducible STR: * install a vulnerable version of Java and launch fx (used the latest m-a in this case) * devtools.chrome.enabled;true * extensions.logging.enabled;true * extensions.blocklist.url;https://blocklist-dev.allizom.org/blocklist/3/%APP_ID%/%APP_VERSION%/%PRODUCT%/%BUILD_ID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/%PING_COUNT%/%TOTAL_PING_COUNT%/%DAYS_SINCE_LAST_PING%/ * launch the browser console * Components.classes["@mozilla.org/extensions/blocklist;1"].getService(Components.interfaces.nsITimerCallback).notify(null); Error in the browser console: Blocklist::_handleCertItemNode: Error adding revoked cert by Issuer and Serial[Exception... "Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsICertBlocklist.revokeCertByIssuerAndSerial]" nsresult: "0x80070057 (NS_ERROR_ILLEGAL_VALUE)" location: "JS frame :: resource://gre/components/nsBlocklistService.js :: Blocklist.prototype._handleCertItemNode :: line 959" data: no]
The issue is that there is a bad entry in the -dev blocklist. This entry does not exist in the production blocklist. The item in question is: <certItem issuerName="MGwxCzAJBgNVBAYTAlVTMRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxGTAXBgNVBAsTEHd3dy5kaWdpY2VydC5jb20xKzApBgNVBAMTIkRpZ2lDZXJ0IEhpZ2ggQXNzdXJhbmNlIEVWIFJvb3QgQ0E=^M "> <serialNumber>BOHnpNxc8vNtwCtCuF0Vnw==</serialNumber> </certItem> The error is from a LOG call from when the error is handled in nsBlocklistService.js - which means that other entries are unaffected. The issue does not appear in m-c because it's using the newer kinto-based blocklist client.
Flags: needinfo?(mgoodwin)
I removed the trailing newline from that block (looks like it was just a test anyway). Can you retry?
Flags: needinfo?(kjozwiak)
(In reply to Mark Goodwin [:mgoodwin] from comment #5) > The issue does not appear in m-c because it's using the newer kinto-based > blocklist client. How is the kinto-based blocklist used on m-c being populated? Its concerning if its not blocking the same items as the amo one.
I went through the same STR from comment # 4 and I'm not seeing the error under the browser console anymore. I used both Java SE 8u77 and Java SE 8u74 against the following builds: * latest m-c using changeset ae7413abfa4d --> NOT Reproducible * latest m-a using changeset 1287b9f362ee --> NOT Reproducible * latest m-b using changeset b007110e9005 --> NOT Reproducible * latest m-r using changeset e35da3da61cb --> NOT Reproducible Looks like this isn't an issue anymore. I'll let Andrew close this off as he's waiting for a response in comment #7.
Flags: needinfo?(kjozwiak)
Flags: needinfo?(mgoodwin)
(In reply to Andrew Williamson [:eviljeff] from comment #7) > (In reply to Mark Goodwin [:mgoodwin] from comment #5) > > The issue does not appear in m-c because it's using the newer kinto-based > > blocklist client. > > How is the kinto-based blocklist used on m-c being populated? Its > concerning if its not blocking the same items as the amo one. Services have a script which can pull entries from the blocklist.xml to populate a kinto instance. If the blocklist entries differ, this likely isn't being run. I'll follow this up with Tarek.
Flags: needinfo?(mgoodwin)
Yes that's the process, let me ping Bob so he can verify everything's fine there
Flags: needinfo?(bobm)
The process runs on this two jobs: - https://deploy.mozaws.net/job/kinto-writer-xml2kinto/ - https://deploy.mozaws.net/job/kinto-writer-xml2kinto-PROD/ But I don't think the cron updates the blocklist.xml but just take the one from the github repository. We should make sure that the blocklist file is updated in the cron. The cron runs every 30 minutes. A quick fix would be to commit the last version of the blocklists file in the github repository. In the meantime we should update the cron to make sure it updates the last blocklist file from AMO.
Assignee: nobody → tarek
(In reply to Tarek Ziadé (:tarek) from comment #10) > Yes that's the process, let me ping Bob so he can verify everything's fine > there Kicking this one over to :jp.
Flags: needinfo?(bobm) → needinfo?(jschneider)
just an update on this: I confirm the prod data is outdated. The cron job was failing with an error, and we are currently working on fixing it.
Flags: needinfo?(jschneider)
we've confirmed a fix on stage, and it will be shipped in the next prod push we are working on (stage was pushed). I'll link a bug here tomorrow once we have it.
Depends on: 1255776
A couple days ago while running m-c under debug mode, ran into the following exception. Is this related to this issue? ************************* A coding exception was thrown and uncaught in a Task. Full message: TypeError: versionInfo.data is undefined Full stack: this.checkVersions/<@resource://services-common/kinto-updater.js:48:14 TaskImpl_run@resource://gre/modules/Task.jsm:319:40 promise callback*TaskImpl_handleResultValue@resource://gre/modules/Task.jsm:395:7 TaskImpl_run@resource://gre/modules/Task.jsm:327:13 promise callback*TaskImpl_handleResultValue@resource://gre/modules/Task.jsm:395:7 TaskImpl_run@resource://gre/modules/Task.jsm:327:13 TaskImpl@resource://gre/modules/Task.jsm:280:3 createAsyncFunction/asyncFunction@resource://gre/modules/Task.jsm:254:14 Task_spawn@resource://gre/modules/Task.jsm:168:12 this.checkVersions@resource://services-common/kinto-updater.js:24:10 Blocklist.prototype.notify@file:///Users/kjozwiak/projects/mozilla-central/objdir-ff-release/dist/NightlyDebug.app/Contents/Resources/components/nsBlocklistService.js:637:7 TM_notify/<@file:///Users/kjozwiak/projects/mozilla-central/objdir-ff-release/dist/NightlyDebug.app/Contents/Resources/components/nsUpdateTimerManager.js:201:11 TM_notify@file:///Users/kjozwiak/projects/mozilla-central/objdir-ff-release/dist/NightlyDebug.app/Contents/Resources/components/nsUpdateTimerManager.js:249:7 *************************
Yes. However it should check the versionInfo data before reading it, I thought this was already changed. MArk?
Depends on: 1267949
Flags: needinfo?(mgoodwin)
(In reply to Tarek Ziadé (:tarek) from comment #16) > Yes. However it should check the versionInfo data before reading it, I > thought this was already changed. MArk? I thought that was addressed in bug 1257533? Kamil, when did you see this?
Flags: needinfo?(mgoodwin) → needinfo?(kjozwiak)
I was using an older build from 04/15 [1]. It looks like the patch from bug # 1257533 landed in m-c on 04/18 which explains why I was receiving the error message. I went through the same test case using a newer m-c build [2] and didn't receive the error message when m-c pinged the blocklist. [1] https://hg.mozilla.org/mozilla-central/rev/d62963756d9a9d19cbbb5d8f3dd3c7cfa8fdef88 [2] buildID: 20160427113439, changeset: ab0044bfa1df
Flags: needinfo?(kjozwiak)
Correction, I used the build from 04/09 when I ran into the exception, not 04/15.
This should be working properly now, thanks for the feedback everyone!
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Note that on the AMO side there's a duplicated entry for F7PAjw2k0dTX5escPnyVOBo= in the xml file. So right now we end up with 113 records when the AMO xml file has 114 entries with one duplicated. Just pinging Jorge on this in case it's relevant
Flags: needinfo?(jorge)
looks like c1107 is duplicated with c1115
I've deleted the latter.
Flags: needinfo?(jorge)
Do we currently have any systems in place for validating entries in the xml files? Should we be doing monthly audits/checks to make sure that incorrect entries or errors haven't made it into the blocklists which could cause issues down the road?
I am working on a script that will ensure that every day for different version of the softwares that are loading the blocklist file daily. (Thunderbird, Firefox, Fennec and Seamonkey)
You need to log in before you can comment on or make changes to this bug.