Closed Bug 1576508 Opened 5 months ago Closed 3 months ago

downloads.onCreated API event not fired when the download items include a referrer property

Categories

(WebExtensions :: General, defect, P1, major)

71 Branch
defect

Tracking

(firefox-esr60 unaffected, firefox-esr68 unaffected, firefox67 unaffected, firefox68 unaffected, firefox69 unaffected, firefox70 verified, firefox71 verified)

VERIFIED FIXED
mozilla71
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox67 --- unaffected
firefox68 --- unaffected
firefox69 --- unaffected
firefox70 --- verified
firefox71 --- verified

People

(Reporter: fogcitynative, Assigned: rpl)

References

(Regression)

Details

(Keywords: regression)

Attachments

(3 files)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0

Steps to reproduce:

I upgraded to the latest Apple OS X Public Beta 6. Public Beta 5 rendered all Firefox apps useless as they would not launch.

Beta 6 resolved this p[roblem. And created a new one.

Firefox approved Add-On called Download Manager S3 no longer works properly. It was working in previous betas.

And this must be a Firefox bug because the Chrome version of this add-on is functioning properly.

Actual results:

I went to a website (any website) and tried to download a file.
The file downloaded in the background using the default download capabilities of Firefox. THIS IS NOT WHAT SHOULD HAVE HAPPENED.

Expected results:

Upon starting the download, Firefox should have offloaded the task to Download Manager S3.

Download Manager would normals add the download to its history and manage the download.

In addition, Download Manager S3 can if preferenced display a Download Bar across the bottom of the screen to monitor the download.

My preference in Download Manager S3 upon completion of the download to switch to the parent folder of the just downloaded file, play a sound and display a notification.

None of those things happened. I believe they would have all happened if Firefox had properly sent the download to Download Manager S3 and then gotten out of the way.

Duplicate of this bug: 1580676
Product: Firefox → WebExtensions
Version: 70 Branch → Firefox 70

Hello,

We currently do not have any machines running the 10.15 Beta 6 version of Mac OS, as all are updated to the latest one, Beta 8.

However, I have nonetheless attempted to reproduce the issue you have reported on the systems we have available. The issue you are encountering does not occur on systems updated to Beta 8.
Firefox properly hands the download task to Download Manager S3. I have also set the add-on preferences as you did and they appear to work as expected.

In conclusion, I would suggest you update your machine to the latest Mac OS (Beta 8) and try again to see whether the issue persists and let me know of the results. After that, if it still occurs, we will try and further debug the problem. Thank you !

Flags: needinfo?(fogcitynative)
Component: Untriaged → General

It looks as if the final version of macOS Catalina is out - can you please retest with that version and confirm whether the problem still exists? Thanks in advance.

Hello,

I re-tested the issue against the latest MacOS Catalina 10.15 and Windows 10 Pro 64-bit on the latest Nightly (71.0a1/20191009213914), Beta (70.0b13/20191007220302) and Release (69.0.2/20191001234643) with the following results:

  • on Nightly and Beta, the browser does not properly hand down the download task to Download Manager S3, thus confirming the issue the reporter has.
  • on Release, the browser will correctly hand down the download task to the add-on.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(fogcitynative)

Thank you for confirming this bug. I am traveling and relying more on iOS so I apologize for not responding to your request.

Hope you can find what changes in Nightly and Beta combined with Catalina broke this add-on, which I really miss because without it I need to click on the downloads window to confirm the download has started.

The priority flag is not set for this bug.
:jimm, could you have a look please?

For more information, please visit auto_nag documentation.

Flags: needinfo?(jmathies)

Bugbug thinks this bug is a regression, but please revert this change in case of error.

Keywords: regression

Bugbug is right, this looks like a regression.
The download event is not being received by the extension because it is failing to be sent from the main parent process to the extension process, due to a DataCloneError.

I did run a mozregression to look for the exact regression range and it pointed out to this commit:

https://hg.mozilla.org/integration/autoland/rev/ee1b4fa1b54e063f8796af2bfd6807e89ba1f180

landed from Bug 1567940, which makes sense because in Bug 1567940 the downloadItem referrer getter defined in ext-downloads.js has been changed and it seems to be an nsIURI instead of a string now:

and so the referrer nsIURI property of the downloadItem seems to be what is triggering the DataCloneError and preventing the downloads API event to be sent to the listener waiting for it in the webextensions child process.

I'm marking it as a P1 because the regression has been introduced in Firefox 70 which is currently in beta and near to be released,
in my opinion we should track this for the 70 release.

I'm assigning this to myself, as Thomas seems to be PTO until Oct 15.

Assignee: nobody → lgreco
Status: NEW → ASSIGNED
Flags: needinfo?(jmathies)
Priority: -- → P1
Regressed by: 1567940

[Tracking Requested - why for this release]: as Luca explained in comment 8, this likely breaks most download addons since Firefox 70.

Pushed by luca.greco@alcacoop.it:
https://hg.mozilla.org/integration/autoland/rev/41d6b2c57641
Fix downloads.onCreated DataClone error regression due to nsIURI referrer property. r=zombie

Please request uplift to mozilla-release.
Alex, can you verify the fix in nightly after it lands?

Flags: qe-verify+
Flags: needinfo?(lgreco)
Flags: needinfo?(alexandru.cornestean)
OS: Unspecified → All
Hardware: Unspecified → All
Version: Firefox 70 → Firefox 71

Comment on attachment 9100533 [details]
Bug 1576508 - Fix downloads.onCreated DataClone error regression due to nsIURI referrer property. r?zombie!

Beta/Release Uplift Approval Request

  • User impact if declined: broken downloads on apple's latest and greatest mess.
  • Is this code covered by automated tests?: Yes
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: Yes
  • If yes, steps to reproduce: ask :luca (European time zone)
  • List of other uplifts needed: None
  • Risk to taking this patch: Medium
  • Why is the change risky/not risky? (and alternatives if risky): Unbtested changes at the last minute to a heavily used webextension api.
  • String changes made/needed: n/a
Attachment #9100533 - Flags: approval-mozilla-release?

(In reply to Jim Mathies [:jimm] from comment #13)

Beta/Release Uplift Approval Request

  • User impact if declined: broken downloads on apple's latest and greatest mess.

The issue doesn't affect only the latest MacOS version (the underlying issue isn't platform-specific, and so the same issue affects the downloads API also on Windows, Linux and any other MacOS version), I'm going to update the bugzilla issue summary to point that out.

  • If yes, steps to reproduce: ask :luca (European time zone)

STR:

  • install the extension mentioned in comment 0
  • open a webpage with a link to an external file (e.g. https://github.com/mozilla/web-ext/releases/tag/3.2.0)
  • right click on the link to the zip file and select "Save Link As"
  • click "Save" in the save file dialog
  • EXPECTED:
    • the extension successfully detects the new download (e.g. show the notification and the download is listed in the extension's browserAction popup window)
Flags: needinfo?(lgreco)
Summary: Firefox Approved Add-On Download Manager S3 fails under latest Catalina Mac beta. → downloads.onCreated API event not fired when the download items include a referrer property

OK wow, then, it's even more of a release blocker. I am going to plan an RC2 with this as the driver as it is getting late in the day. Likely tomorrow or Wednesday. Thanks Luca and Jim.

I have the sense that download related add-ons are quite popular, but I'll check on that as well to get a better idea of user impact.

Status: ASSIGNED → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla71

Hello,

Verified the fix using the latest Nightly (71.0a1/20191014213051) under Windows 10 Pro 64-bit and MacOS Catalina 10.15. I have also checked the behavior of the extension on the latest Release (69.0.3/20191009172106) for comparison.

The extension now properly detects the download and takes over the task from the browser however, several processes are not occurring as in the Release version:

  • the download bar at the bottom of the screen will not register the download progress i.e the progress bar is not displayed. It only states that the download list is empty. This occurs in both Windows and MacOS
  • on Windows, the download complete notification is not displayed, even though it is enabled in the extension’s settings. This works as intended on MacOS, though

See the attached screen captures for further details.

In essence, the issue is fixed, however, before closing the bug, I would require your feedback on the above observed behavior of the extension.

Flags: needinfo?(alexandru.cornestean) → needinfo?(lgreco)
Attached image DMS3 issues Windows.gif
Attached video DMS3 issues MacOS.mov
QA Whiteboard: [qa-triaged]

Hi Alex,
sorry for let you waiting, I had to dig into what the extension is doing a big deeper to see what is going on and if those two issues were still related to other issues in the downloads API. It looks that they are unrelated issues, more details below.


Empty download bar (and "Operation is insecure" error on the content script side)

The download bar is empty because the extension content script is retrieving an HTML fragment from a moz-extension url and then it is trying to append those elements directly into the webpage attached by the extension content script. This is triggered an "Operation is insecure" error on Nightly (which is visible in the Browser console by enabling the "Show Content Messages" checkbox, or in the tab webconsole), and so it is an issue unrelated with the downloads API.

It is definitely possible that we have become more strict about injecting extension elements directly into a third party webpages (I haven't tried to bisect when that change in behavior has been introduced), but in any case injecting extension UI elements directly into a third party webpage is considered an anti-pattern because it potentially leaks to the website information about the user and may introduce other security issues (e.g. if the extension provide to the user a way to control certain behaviors by interacting with the injected UI elements) and so in my opinion that part has to be investigated and fixed on the extension side (e.g. by injecting directly into the webpage only an iframe that points to a moz-extension url and render the extension UI directly inside that iframe, this wouldn't trigger the "Operation is insecure" error and the iframe content isn't going to be accessible to the webpage).

Notification API doesn't show notifications on Windows, no errors logged

The missing notification seems odd, but the notifications API is a feature that is using an underlying os-specific API and so it is not unlikely that it may be broken on some operating system and working correctly on the other ones.

The missing notification issue also looks unrelated to the downloads API, it looks to be a notifications API issue on windows as I briefly tried to reproduce it using the small that extension notify-link-clicks-i18n and it is showing a notification on Linux but not on Windows.

(we should file the notifications API issue on windows as a separate bugzilla issue, it may also be useful to run a mozregression using the notify-link-clicks-i18n example and bisect which is the regressing bug)

Flags: needinfo?(lgreco)

I confirm that the missing notification on windows is an issue with the Windows' "system backend" for the notifications, turning the "system backend" off from about:config (by setting "alerts.useSystemBackend" to false) makes Firefox to fallback to a "Firefox-integrated alert service" and that makes the notifications to be shown as expected.

I tried to use the Notification WebAPI, and the same behavior is reproducible with it, and so the notification issue on windows is not an issue specific to the WebExtensions API, and so I double-checked if we do have any know issues with the notifications "system backend" on windows already files and we do:
The missing notification issue is likely related to Bug 1500135 ("Action Center (Notification Center) notifications on Windows 10 are delayed sometimes by hours").

Anyway the notification issue should be a nightly-only problem because it seems that we haven't enabled them by default on release yet (See "Bug 1497425 - Turn on Toast Notification on release channel")

Hello Luca,

Thank you for the quick response ! Based on your feedback, the issue is fixed and thus I will mark it as Verified.

I also confirm that by setting the pref to false in Nightly, notifications are shown as expected and that on the Release version, the preference is false by default, hence the observed expected behavior.

Status: RESOLVED → VERIFIED
Flags: qe-verify+

(In reply to Luca Greco [:rpl] [:luca] [:lgreco] from comment #21)

It is definitely possible that we have become more strict about injecting extension elements directly into a third party webpages (I haven't tried to bisect when that change in behavior has been introduced),

Sounds like bug 1467970, which is about cross-document adoption, and is not specific to extensions.

Comment on attachment 9100533 [details]
Bug 1576508 - Fix downloads.onCreated DataClone error regression due to nsIURI referrer property. r?zombie!

Fix for high impact issue for addons users, verified in nightly.
OK for uplift for the RC2 build.

Attachment #9100533 - Flags: approval-mozilla-release? → approval-mozilla-release+

Hello,

Verified the fix using the latest Beta (70.0/20191016161957) under Windows 10 Pro 64-bit and MacOS Catalina 10.15.

The extension now properly detects the download and takes over the task from the browser. Progress bar and notifications are also properly displayed.

Version: Firefox 71 → 71 Branch
Duplicate of this bug: 1581270
You need to log in before you can comment on or make changes to this bug.