Closed Bug 1740116 Opened 6 months ago Closed 5 months ago

HAR file export results in an empty file

Categories

(DevTools :: Netmonitor, defect, P2)

Firefox 94
defect

Tracking

(firefox-esr91 unaffected, firefox94 wontfix, firefox95 wontfix, firefox96 wontfix, firefox97 verified)

VERIFIED FIXED
97 Branch
Tracking Status
firefox-esr91 --- unaffected
firefox94 --- wontfix
firefox95 --- wontfix
firefox96 --- wontfix
firefox97 --- verified

People

(Reporter: kontakt, Assigned: ochameau)

References

(Blocks 1 open bug, Regression, )

Details

(Keywords: regression)

Attachments

(1 file)

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:94.0) Gecko/20100101 Firefox/94.0

Steps to reproduce:

  1. Open https://wiadomosci.onet.pl/bialystok/na-granicy-z-bialorusia-sa-tysiace-migrantow-na-zywo/sbkd6j8
  2. Open Devtools, navigate to network panel
  3. Refresh the page
  4. Click the cog -> Save all as HAR
  5. Save the file to disk

Actual results:

The file is empty (0B)

Expected results:

The file should be a JSON containing the information on all requests performed by the website

Has STR: --- → yes
Component: Untriaged → Netmonitor
Product: Firefox → DevTools

The stack in the browser console says:

Error: Error while calling method getRequestHeaders: No such actor for ID: server0.conn6.netEvent81
    _requestData resource://devtools/client/netmonitor/src/connector/firefox-data-provider.js:588
    requestData resource://devtools/client/netmonitor/src/connector/firefox-data-provider.js:505
    requestData resource://devtools/client/netmonitor/src/connector/index.js:605
    buildRequest resource://devtools/client/netmonitor/src/har/har-builder.js:177
    buildEntry resource://devtools/client/netmonitor/src/har/har-builder.js:113
    build resource://devtools/client/netmonitor/src/har/har-builder.js:64
    buildHarData resource://devtools/client/netmonitor/src/har/har-exporter.js:211
    fetchHarData resource://devtools/client/netmonitor/src/har/har-exporter.js:159
    save resource://devtools/client/netmonitor/src/har/har-exporter.js:78
    saveAllAsHar resource://devtools/client/netmonitor/src/har/har-menu-utils.js:39
    onClick resource://devtools/client/netmonitor/src/components/Toolbar.js:519
    React 38
    componentDidMount resource://devtools/client/shared/components/menu/MenuButton.js:126
    componentDidMount resource://devtools/client/shared/components/menu/MenuButton.js:122
    React 19
    bootstrap resource://devtools/client/netmonitor/src/app.js:77
    open resource://devtools/client/netmonitor/panel.js:21
    onLoad resource://devtools/client/framework/toolbox.js:2534
har-exporter.js:183:17

I'm also seeing this symptom with large sessions (~1gb). Although I think it's been an issue for a very long time, if not forever.

Thank for the report!

I was able to reproduce the problem in 94

Error: Error while calling method getRequestHeaders: No such actor for ID: server0.conn0.netEvent7
    _requestData resource://devtools/client/netmonitor/src/connector/firefox-data-provider.js:588
har-exporter.js:183:17

But, I am not seeing it in 95/96

@Kuba, could you try Firefox Nightly?
https://www.mozilla.org/cs/firefox/channel/desktop/

Flags: needinfo?(kontakt)

I've just checked in 96.0a1 (2021-11-11). The issue still persists. The error message is the same:

Error: Error while calling method getEventTimings: No such actor for ID: server0.conn0.netEvent4637 firefox-data-provider.js:600:15
Error: Error while calling method getEventTimings: No such actor for ID: server0.conn0.netEvent4684 firefox-data-provider.js:600:15
Error: Error while calling method getEventTimings: No such actor for ID: server0.conn0.netEvent4637 firefox-data-provider.js:600:15
Error: Error while calling method getEventTimings: No such actor for ID: server0.conn0.netEvent4684 firefox-data-provider.js:600:15
Error: Error while calling method getRequestHeaders: No such actor for ID: server0.conn0.netEvent4549
    _requestData resource://devtools/client/netmonitor/src/connector/firefox-data-provider.js:600
har-exporter.js:183:17
    onError resource://devtools/client/netmonitor/src/har/har-exporter.js:183
    (Async: promise callback)
    fetchHarData resource://devtools/client/netmonitor/src/har/har-exporter.js:182
    save resource://devtools/client/netmonitor/src/har/har-exporter.js:78
    saveAllAsHar resource://devtools/client/netmonitor/src/har/har-menu-utils.js:39
    onClick resource://devtools/client/netmonitor/src/components/Toolbar.js:519
    React 19

Flags: needinfo?(kontakt)

I should add that it happens on other websites, as well - most often when the tracking protection is disabled

I still have troubles to reproduce the problem. Did you try it in fresh new Firefox profile?
(no extensions installed)

@Bomsy, can you please try this on your machine?

Flags: needinfo?(hmanilla)

Hi, I tried it on Firefox Nightly in Safe Mode. Still the same. Here's a video of me reproducing this issue. It doesn't happen every time, but usually when I "accept" the cookies on this website and then refresh without cache and wait a while and then attempt to generate the HAR file, it happens on 2nd or 3rd try

https://video.kuba-orlik.name/videos/watch/808a142f-fc26-46b7-a67a-cf2333897c19

Thank you, Kuba!

I was able to reproduce the issue on my machine (Fx Nightly, Win10).

My STRs:

  1. Open Firefox in Private mode
  2. Load https://wiadomosci.onet.pl/bialystok/na-granicy-z-bialorusia-sa-tysiace-migrantow-na-zywo/sbkd6j8
  3. Accept the Cookies, click on "Przejdź do serwisu"
  4. Open DevTools, select the Network panel
  5. Reload the page and wait till it's loaded
  6. Right click on the list of request and pick "Save All As HAR"
  7. Check out the size of the file, it's empty -> BUG

There is an exception in the Browser Console

Error: Error while calling method getRequestHeaders: No such actor for ID: server0.conn6.netEvent9135
    _requestData resource://devtools/client/netmonitor/src/connector/firefox-data-provider.js:600
har-exporter.js:183:17

Honza

Severity: -- → S3
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(hmanilla)
Priority: -- → P3

What does the regressionwindow-wanted tag signify and how can I help with that?

It's a request for finding regression range that would help us to identify specific bug (patch) that caused this reported bug (I believe this worked before, so I think it's a regression).

There is a 'mozregression' tool that is helping to find that range. It supports auto downloading various Firefox builds and allowing you to test the feature on those builds quickly (by bisecting) and narrowing the range between: the feature works / the feature is broken.

The source of the tool + links to user docs: https://github.com/mozilla/mozregression
User docs: https://mozilla.github.io/mozregression/documentation/

It would be great help if you could find the culprit bug/patch!

I'm also having problem with HAR generation in Firefox 94, I wonder if its related. I use the https://github.com/firefox-devtools/har-export-trigger and the problem is described here: https://github.com/sitespeedio/browsertime/issues/1671

For me it's like this: I use Browsertime to start the browser with the HAR exporter installed. When I navigate to a page I can see in the network tools that requests happens but when the HAR is generated it is a JSON without any page and requests.

I've been able to verify that it worked fine in 93 so this happens in 94. However when I use the har-export-trigger test page, it's always work.

I found a workaround: If I open Firefox, wait some time (on my local machine 7 seconds is the magic number) and then navigate to the URL, the HAR file is generated correctly.

I'm having trouble using mozregression when I automate things with webdriver (as in Browsertime) but maybe there's a workaround?

Should I create a separate issue for this or do you think it's the same thing Jan?

Just want to clarify that this makes it impossible for us to upgrade to 94 for our WebPageReplay tests at Wikimedia + for Browsertime/sitespeed.io users their performance tests will fail when they run with 94.

QA Whiteboard: [qa-regression-triage]

Ok, I ran mozregression and the result is:

60:41.58 INFO: Last good revision: 9e713e49c00c14cc49032e828ad245f04be6447c
60:41.58 INFO: First bad revision: c1ba43b65415d33eeb716a191c2bb8395aed6224
60:41.58 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=9e713e49c00c14cc49032e828ad245f04be6447c&tochange=c1ba43b65415d33eeb716a191c2bb8395aed6224
Flags: needinfo?(odvarko)
Has Regression Range: --- → yes
QA Whiteboard: [qa-regression-triage]
Regressed by: 1439509

(In reply to Peter Hedenskog from comment #11)

I'm also having problem with HAR generation in Firefox 94, I wonder if its related. I use the https://github.com/firefox-devtools/har-export-trigger and the problem is described here: https://github.com/sitespeedio/browsertime/issues/1671

This feels like a different problem, but let's see if fix for this report helps or not.

Just want to clarify that this makes it impossible for us to upgrade to 94 for our WebPageReplay tests at Wikimedia +
for Browsertime/sitespeed.io users their performance tests will fail when they run with 94.

I am increasing priority for this bug

(In reply to Kuba Orlik from comment #13)

Ok, I ran mozregression and the result is:

Thank you Kuba, this is great help!

@Alex, it looks like this is a regression caused by Bug 1439509 - [dt-leak] NetworkEvent actors leak on reload
Can you please look at it, thank you.

Honza

Flags: needinfo?(odvarko)
Priority: P3 → P2
Flags: needinfo?(poirot.alex)

This exception seems to be related to old request made by the previous page load.
The netmonitor frontend receives these request a bit late, display them and try to fetch extra data about them.
As the HAR export is based on netmonitor frontend state, it also tries to manipulate these old, now defunct requests.

You may workaround these exception by toggling devtools.netmonitor.persistlog to true, but that may record more requests than you expect in the HAR file.

Another approach compared to the submitted patch would be to strengthen HAR codebase to be more resilient against unexpected errors with one request or one particular extra data fetching. So that, in case of exception we don't break the complete HAR export but only the one request or attribute we fail fetching.

Flags: needinfo?(poirot.alex)

These exception may happen when some pending request from the previous page
are received lated by the netmonitor, which tries to fetch extra data about these
old requests.
On the server side, these requests are destroyed when devtools.netmonitor.persistlog=false.
(which is the default behavior)

I can confirm that setting the preference to devtools.netmonitor.persistlog:true makes it work for me, that is great! Couldn't see any extra requests being picked up when I briefly looked,

What I cannot understand is what request that could be pending? In my use case, the browser opens, then navigates to data:text/html;charset=utf-8,<html><body></body></html> and then to the page that is measured (where we get the HAR). Could it be backend request (made by the browser) that was introduced in 94?

Assignee: nobody → poirot.alex
Status: NEW → ASSIGNED
Pushed by apoirot@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8bde59d048c8
[devtools] Fix fetching extra data about cancelled navigation request. r=nchevobbe
Status: ASSIGNED → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → 97 Branch

Thank you Alex for fixing this!

The patch landed in nightly and beta is affected.
:ochameau, is this bug important enough to require an uplift?
If not please set status_beta to wontfix.

For more information, please visit auto_nag documentation.

Flags: needinfo?(poirot.alex)

I just upgraded to Firefox 95 (from 94) and then the devtools.netmonitor.persistlog to true fix doesn't fix the problem anymore?

Blocks: 1747804

(In reply to Peter Hedenskog from comment #22)

I just upgraded to Firefox 95 (from 94) and then the devtools.netmonitor.persistlog to true fix doesn't fix the problem anymore?

Now, you might be hitting yet another issue specific to Fission.
Could you try disabling Fission, by turning fission.autostart preference to false?

Otherwise, in Firefox 97, I'm still able to reproduce comment 0 issues.
It looks like I only fixed one very particular failure and more are still reproducing.
I opened bug 1747804 to keep fixing HAR export code.

Finally, regarding the uplift request. Yes, it is worth uplifting, but I should figure out bug 1747804 first.

We're in RC week for 96, so too late for uplift.

Flags: needinfo?(poirot.alex)

Reproduced the initial issue on Firefox 96 on macOS Big Sur 11.6.

Verified as fixed on the latest Nightly 98.0a1 and Firefox 97 beta 5 - used the STR from the Description and from Comment 8 - the Har file is not empty anymore. Tested on macOS Big Sur 11.6, Windows 10 x64, and Ubuntu 20.04.

Status: RESOLVED → VERIFIED
Flags: qe-verify+

I verified that this work in Firefox 97.

You need to log in before you can comment on or make changes to this bug.