1829200 - Firefox 112 regression: network listener request missing valid tabId on twitch.tv

Status: VERIFIED FIXED

(Core :: DOM: Networking, defect, P2)

Description

[Michel Gutierrez]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/112.0

Steps to reproduce:

Using the Video DownloadHelper extension with Firefox 112.0.1 on https://twitch.tv/ , no video is detected while it works fine on Firefox < 112

The problem has been reported by VDH users on the support forum: https://groups.google.com/g/video-downloadhelper-q-and-a/c/O_bRwxrfqS8/m/3jvzeSPxDgAJ

After investigation, it appears that with Firefox 112, the network listener call corresponding to the network request for the HLS video M3U8 manifest has a field "tabId" equals to -1. Since VDH cannot identify the originating tab, it simply ignores the request.

Here is a log of the listened request with Firefox 112.0.1:
{
"requestId": "295",
"url": "https://usher.ttvnw.net/api/channel/hls/yogscast.m3u8?...",
"originUrl": "https://www.twitch.tv/",
"documentUrl": "https://www.twitch.tv/",
"method": "GET",
"type": "xmlhttprequest",
"timeStamp": 1682010441635,
"tabId": -1,
"frameId": 0,
"parentFrameId": -1,
"incognito": false,
"thirdParty": true,
"cookieStoreId": "firefox-default",
"fromCache": false,
"responseHeaders": [
...
],
"statusCode": 200,
"statusLine": "HTTP/2.0 200 OK",
"proxyInfo": null,
"ip": "...",
"urlClassification": {
"firstParty": [],
"thirdParty": []
},
"requestSize": 0,
"responseSize": 0
}

Note the "tabId" field equals to -1.

The same capture with Firefox 111.0.1:
{
"requestId": "311",
"url": "https://usher.ttvnw.net/api/channel/hls/yogscast.m3u8?...",
"originUrl": "https://www.twitch.tv/",
"documentUrl": "https://www.twitch.tv/",
"method": "GET",
"type": "xmlhttprequest",
"timeStamp": 1682010923860,
"tabId": 3,
"frameId": 0,
"parentFrameId": -1,
"incognito": false,
"thirdParty": true,
"cookieStoreId": "firefox-default",
"fromCache": false,
"responseHeaders": [
...
],
"statusCode": 200,
"statusLine": "HTTP/2.0 200 OK",
"proxyInfo": null,
"ip": "...",
"urlClassification": {
"firstParty": [],
"thirdParty": []
},
"requestSize": 0,
"responseSize": 0
}

Here, the "tabId" field has a valid value, 3.

So far, the problem has only been seen on twitch.tv. Other tested sites broadcasting HLS videos seem to work as expected with a valid originating tab.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Alice0775 White]

I can reproduce the issue on Windows10.

Regression window:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=99bb9f2c4e3edb90a72b80524b38f04b1681a952&tochange=0c260a33541a67b1c4f0ab83a1d1380f466fcf9b

Note:
Workaround is setting dom.workers.pFetch.enabled to false

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

:edenchuang, since you are the author of the regressor, bug 1351231, could you take a look? Also, could you set the severity field?

For more information, please visit auto_nag documentation.

Comment 4

[Frank Griffith Jr]

Eden has notified me that they are working on https://bugzilla.mozilla.org/show_bug.cgi?id=1819570 that may help this. Will link and monitor.

Comment 5

[publicemaildump751]

Folks,
Same bug on
https://www.winktv.co.kr/live
https://www.pandalive.co.kr/live
Upon upgrading Firefox. My installation is:
Linux Mint Cinnamon 19.3
Firefox 112.0.2
Issue is with Video Downloadhelper extension. It is no longer able to discover or download any streams from at least these two websites, while some other websites still work with extension (can download streams).
Setting dom.workers.pFetch.enabled to false restores my ability to download streams from those websites.
I haven't yet found a crystal clear description of repercussions of setting that parameter to false.

Comment 6

[Valentin Gosu [:valentin] (he/him)]

Bug 1819570 has landed in Nightly 114. Could you check if using a build from https://nightly.mozilla.org fixes your issue?
Thanks!

Comment 7

[Michel Gutierrez]

I may have missed something but using yesterday's nightly, 115.0a1 (2023-05-08) (64-bit), the problem seems to persist. On Titch.tv, the "tabId" field of the network listener callback parameter is still -1.
How confident are you that the issue is fixed ? I might re-run the tests if you think the bug is gone.

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1351231

Comment 9

[Eden Chuang[:edenchuang]]

I am working on a patch for this. But we still need the landed patch in bug 1819570 since it provides needed information for fetch() from workers.

Comment 10

[Valentin Gosu [:valentin] (he/him)]

Hi Eden, let me know if there's anything blocking us on this bug. Thanks!

Comment 11

[Pascal Chevrel:pascalc]

This is P2/S2, is there any progress on this one? Thanks!

Comment 12

[Eden Chuang[:edenchuang]]

A local patch is under the try testing.

Comment 13

[Wild Willy Kredentser]

Firefox 114.0 escaped into the wild a couple of days ago. The fix here has not made it into 114 despite the tracking flags on this report implying it would have been in 114. With the about:config preference for pFetch mentioned above set to the default value TRUE, VDH is unable to detect content on Twitch, of course on 114. That's the only site where I tested this but it's likely other sites continue to be adversely affected.

However, there has been some progress. Before 114, the Network Monitor showed no HLS manifests (.m3u8 files) on Twitch. With 114, .m3u8 files are once again appearing in the Network Monitor. But that simple fact does not seem to be enough to solve the VDH problem. This Bugzilla report remains unresolved & it is not clear at all when it will progress sufficiently to make it into a public release of Firefox. Perhaps there is activity occurring that is not being reported here. If so, it would be welcomed to see all relevant activity reported here.

Comment 14

[Julian Descottes [:jdescottes]]

Eden, not sure if you need any help with DevTools bits here (haven't seen patches yet, so I don't know where the issue/fix is), but don't hesitate to ni? myself or :Honza if we can assist with anything DevTools related.

Comment 15

[Donal Meehan [:dmeehan]]

Setting 114 to wontfix.
Any updates on the investigation/patch since Comment 9.
Wondering if we will have something in time for 115?

Comment 16

[Eden Chuang[:edenchuang]]

Attachment: Bug 1829200 - Get LoadContext/BrowsingContext from loadInfo.workerAssociatedBrowsingContext for WebRequests when the request is from Workers. r=#extension-reviewers

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9beaf8824b1a
Get LoadContext/BrowsingContext from loadInfo.workerAssociatedBrowsingContext for WebRequests when the request is from Workers. r=extension-reviewers,robwu

Comment 18

[Cristian Tuns]

Backed out for causing mochitest failures in /test_ext_webrequest_worker.html

• Backout link
• Push with failures
• Failure Log
• Failure line: TEST-UNEXPECTED-FAIL | toolkit/components/extensions/test/mochitest/test_ext_webrequest_worker.html | origin url is correct - got "test_ext_webrequest_worker.html", expected "test_ext_webrequest_worker.html?currentTestURL=toolkit%2Fcomponents%2Fextensions%2Ftest%2Fmochitest%2Ftest_ext_webrequest_worker.html&closeWhenDone=1&showTestReport=false&expected=pass"

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6386291adbed
Get LoadContext/BrowsingContext from loadInfo.workerAssociatedBrowsingContext for WebRequests when the request is from Workers. r=extension-reviewers,robwu

Comment 20

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/6386291adbed

Comment 21

[Eden Chuang[:edenchuang]]

Comment on attachment 9342183 [details]
Bug 1829200 - Get LoadContext/BrowsingContext from loadInfo.workerAssociatedBrowsingContext for WebRequests when the request is from Workers. r=#extension-reviewers

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: VedioDownload Helper extension could not get the correct tabId of the monitoring WebRequest when the request is from Workers.
• User impact if declined: Extensions might not work correctly.
• Fix Landed on Version: 118
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): The patch is low risky since it basically does not change the logic of the WebRequest code, but using the correct BrowsingContext for getting the correct information.

Comment 22

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 9342183 [details]
Bug 1829200 - Get LoadContext/BrowsingContext from loadInfo.workerAssociatedBrowsingContext for WebRequests when the request is from Workers. r=#extension-reviewers

Approved for 117.0b8 and 115.2esr.

Comment 23

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/dddd2c05631e

Comment 24

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr115/rev/035bba598ba8

Comment 25

[Alexandru Trif, Desktop Test Engineering [:atrif]]

Reproduced the issue with Firefox 114.0a1 (2023-04-20) on Windows 10x64. The Video Download Helper addon will not find any videos on Twitch.
The issue is verified fixed with Firefox 115.2.0esr, 118.0a1 (2023-08-21), and 117.0 on Windows 10x64, macOS 13, and Ubuntu 22. The Video Download Helper addon will find videos on Twitch.