1702936 - 3m/day volume of network errors from Mate Translate extension (CSS/NS_ERROR_CORRUPTED_CONTENT)

Status: RESOLVED MOVED

(WebExtensions :: Developer Outreach, defect, P3)

Description

[Zibi Braniecki [:zbraniecki][:gandalf]]

Not sure where to report it (I'll try here and on the extensions' github), but while investingating network errors in Gecko we noticed that on release we see a very high volume of CSS errors from Meta Translate extension.

Namely:

• [e-mail]!/res/styles/ui_components/contextMenu.css
• [e-mail]!/res/styles/pages/common/main.css
• [e-mail]!/res/styles/pages/internal/content.css

with status NS_ERROR_CORRUPTED_CONTENT are reported at the volume of 1m/day events each topping the css category.

There are other errors from this extension but they're at the volume of 1/100th of those three, so I think there's something particular about those three files.

The reason why I'm filing it here is that the error is not NS_ERROR_FILE_NOT_FOUND which would indicate an error in the extension code, but rather corrupted content which may be indicative of some malformed zip being used or shipped to users.

I'm planning to filter out that error in our probe, as it's not very actionable for our research, but the Addons group may want to look into why 1m events a day on release are ending up in corrupted content and can we do something about it.

For example, if a file in a zip can be corrupted, should we disable such extension? Should we CRC it? Maybe it should be harder to end up with a running extension that is loading resources if the zip is corrupted?

Comment 1

[Zibi Braniecki [:zbraniecki][:gandalf]]

The errors out of this extension are almost uniquely CSS files, (there are 4 reports from manifest.json) at volume of 1m/day each of the three. There are other CSS files with the same error type but at very low volume (sub 100k/day/file).

Comment 2

[Zibi Braniecki [:zbraniecki][:gandalf]]

An example payload looks like this:

[{"f":[{"v":"sync"},{"v":"false"}]},{"f":[{"v":"file_name"},{"v":"[e-mail]!/res/styles/ui_components/contextMenu.css"}]},{"f":[{"v":"status"},{"v":"NS_ERROR_CORRUPTED_CONTENT"}]},{"f":[{"v":"cancelled"},{"v":"false"}]}]

Comment 3

[Zibi Braniecki [:zbraniecki][:gandalf]]

I analyzed random 10 events from March 31st and in them I saw MacOS, Linux and Windows, different locales and countries, and also both some experiments and no experiments running, so I don't suspect that one of those is a strong factor.

Comment 4

[Zibi Braniecki [:zbraniecki][:gandalf]]

Reported to extension authors as https://github.com/MetaMask/metamask-extension/issues/10821

Comment 5

[Kris Maglione [:kmag]]

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #0)

For example, if a file in a zip can be corrupted, should we disable such extension? Should we CRC it? Maybe it should be harder to end up with a running extension that is loading resources if the zip is corrupted?

We don't need to CRC it. Each file in every extension is individually signed, which gives much stronger guarantees than a CRC. The signatures are also periodically re-verified, and the extensions are disabled if they ever fail to validate.

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Networking' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 7

[Zibi Braniecki [:zbraniecki][:gandalf]]

We don't need to CRC it. Each file in every extension is individually signed, which gives much stronger guarantees than a CRC. The signatures are also periodically re-verified, and the extensions are disabled if they ever fail to validate.

Does any hypothesis come to your mind when you see that volume of network errors out of those three files in that extension? What may be going on there?

Comment 8

[Tomislav Jovanovic :zombie]

Hey Zibi, where/how are you seeing this error, and know how many there are in the wild?

Comment 9

[Zibi Braniecki [:zbraniecki][:gandalf]]

where/how are you seeing this error,

I'm seeing it coming out of the zero_byte_load Necko event - https://searchfox.org/mozilla-central/source/toolkit/components/telemetry/Events.yaml#2678

Here's a dashboard for Release that I'm using - https://sql.telemetry.mozilla.org/dashboard/ui-resource-network-errors-release-

and know how many there are in the wild?

I don't understand. Can you rephrase the question?

Comment 10

[Zibi Braniecki [:zbraniecki][:gandalf]]

In case your question was about number of clients - it seems like the volume of 3m events comes from ~30k clients a day.

Comment 11

[Rob Wu [:robwu]]

This add-on has several files in web_accessible_resources, which means that those with knowledge of the extension UUID (typically via the extension, but potentially any website) can load the resource.

moz-extension:-resources (inside xpi) files are typically loaded via a JAR channel (the !/ parts in the reported URLs also hint towards that). The reported errors are about .css files. There is a part in nsJARChannel::OnStartRequest that aborts the request with NS_ERROR_CORRUPTED_CONTENT when the resource is loaded in an incorrect context.

• Stylesheets should have the text/css MIME type, which is be the case for .css extensions (.css are text/css and not overridable)
• Another way to encounter the error is when this .css file is loaded as a script. I have occasionally seen developers loading a resource as a <script> and using onload/onerror to detect whether the load has succeeded. The extension doesn't appear to be using this (I searched for uses of the onerror event inside the extension package, but it's possible that a website used it).

Comment 12

[Rob Wu [:robwu]]

Seems like the referenced Github repository is unrelated to this add-on.

This is the relevant add-on: https://addons.mozilla.org/en-US/firefox/addon/instant-translate/

I note that the add-on contains files such as contextMenu.css and contextMenu.js, but there is no cleartext reference to to those strings inside the extension package. So the file is either loaded indirectly via a website or another extension (possibly using the script onload/onerror trick that I mentioned in comment 11), or the extension has obfuscated code.

Comment 13

[Shane Caraveo (:mixedpuppy)]

Running the extension I see some issues in the console:

Unchecked lastError value: Error: Script 'moz-extension://6d098a3d-e9de-6345-ab19-f6564bb8d402/src/ui_components/scrollbar/scrollbar.js' result is non-structured-clonable data opt.js:69
    attactContentFile moz-extension://6d098a3d-e9de-6345-ab19-f6564bb8d402/src/bg_events/opt.js:69
[object Object] ExtensionCommon.jsm:742
    normalizeError resource://gre/modules/ExtensionCommon.jsm:742
    recvAPICall resource://gre/modules/ExtensionParent.jsm:950
Unchecked lastError value: Error: An unexpected error occurred opt.js:69
    attactContentFile moz-extension://6d098a3d-e9de-6345-ab19-f6564bb8d402/src/bg_events/opt.js:69
Unchecked lastError value: Error: Script 'moz-extension://6d098a3d-e9de-6345-ab19-f6564bb8d402/src/ui_components/scrollbar/scrollbar.js' result is non-structured-clonable data opt.js:69
    attactContentFile moz-extension://6d098a3d-e9de-6345-ab19-f6564bb8d402/src/bg_events/opt.js:69
[object Object] 2 ExtensionCommon.jsm:742
    normalizeError resource://gre/modules/ExtensionCommon.jsm:742
    recvAPICall resource://gre/modules/ExtensionParent.jsm:950
Unchecked lastError value: Error: An unexpected error occurred 2 opt.js:69
    attactContentFile moz-extension://6d098a3d-e9de-6345-ab19-f6564bb8d402/src/bg_events/opt.js:69

When I add some dumps in opt.js I see:

        attactContentFile: function(data, sendResponse, sender) {
            dump(`******** attactContentFile ${data.src}\n`);
            chrome.tabs.executeScript(sender.tab.id, {
                file: data.src
            }, function() {
                dump(`******** attactContentFile lasterror ${browser.runtime.lastError}\n`);
                sendResponse();
            });
        },

******** attactContentFile moz-extension://6d098a3d-e9de-6345-ab19-f6564bb8d402/res/styles/pages/common/main.css
******** attactContentFile moz-extension://6d098a3d-e9de-6345-ab19-f6564bb8d402/res/styles/pages/internal/content.css
JavaScript error: moz-extension://6d098a3d-e9de-6345-ab19-f6564bb8d402/res/styles/pages/common/main.css, line 1: SyntaxError: illegal character U+0040
******** attactContentFile lasterror Error: illegal character U+0040
******** attactContentFile lasterror null
******** attactContentFile moz-extension://6d098a3d-e9de-6345-ab19-f6564bb8d402/res/styles/pages/common/main.css
******** attactContentFile moz-extension://6d098a3d-e9de-6345-ab19-f6564bb8d402/res/styles/pages/internal/content.css
******** attactContentFile lasterror Error: illegal character U+0040
******** attactContentFile lasterror null

as well as other errors in console:

[Exception... "Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIDOMWindowUtils.addSheet]"  nsresult: "0x80070057 (NS_ERROR_ILLEGAL_VALUE)"  location: "JS frame :: resource://gre/modules/ExtensionCommon.jsm :: runSafeSyncWithoutClone :: line 74"  data: no] ExtensionCommon.jsm:74:12
    runSafeSyncWithoutClone resource://gre/modules/ExtensionCommon.jsm:74
    cssPromise resource://gre/modules/ExtensionContent.jsm:524
    (Async: promise callback)
    inject resource://gre/modules/ExtensionContent.jsm:515
    injectInto resource://gre/modules/ExtensionContent.jsm:463
    AsyncFunctionNext self-hosted:695
    (Async: async)
    executeInWin resource://gre/modules/ExtensionContent.jsm:1172
    map self-hosted:224
    handleActorExecute resource://gre/modules/ExtensionContent.jsm:1176
    InterpretGeneratorResume self-hosted:1485
    AsyncFunctionNext self-hosted:695
    (Async: async)
    receiveMessage resource://gre/modules/ExtensionContent.jsm:1224

The css is using @-webkit-keyframes and @keyframes, the css error above for U+0040 is the @ symbol.

Comment 14

[Zibi Braniecki [:zbraniecki][:gandalf]]

Is there a way for us to get in touch with the author?

Comment 15

[Rob Wu [:robwu]]

This is not a bug with Firefox but with the extension. From comment 13 it is obvious that the extension uses chrome.tabs.executeScript when they should have been using browser.tabs.insertCSS.

Nothing to do here for Firefox engineers, the extension authors can fix this.

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #14)

Is there a way for us to get in touch with the author?

https://addons.mozilla.org/en-US/firefox/addon/instant-translate/ lists contact details under "Support E-mail": matetranslate dot com, support at
(add-on reviewers can also contact them through the review system if needed)

Comment 16

[Zibi Braniecki [:zbraniecki][:gandalf]]

Thanks! Sent an email with a link to this bug and CC'ed Rob.

Comment 17

[Abhishek Nagekar [:Abhi]]

The add-on's developer has been notified about this issue.