Closed Bug 1159078 Opened 5 years ago Closed 8 months ago

Display a useful message when a response body is truncated

Categories

(DevTools :: Netmonitor, defect)

defect
Not set

Tracking

(firefox67 verified)

VERIFIED FIXED
Firefox 67
Tracking Status
firefox67 --- verified

People

(Reporter: khuey, Assigned: koroknay)

References

(Blocks 1 open bug, )

Details

Attachments

(3 files)

We limit response logging to 1 MB.  http://mxr.mozilla.org/mozilla-central/source/toolkit/devtools/webconsole/network-monitor.js?force=1#32  When we hit this limit we should display a useful message rather than e.g. failing to parse a JSON blob.  Something like "response too large to display", etc.
See Also: → 1206061
In Firebug the response limit is controlled through extensions.firebug.netDisplayedResponseLimit. If that value is exceeded, the following message is shown:

"Firebug response size limit has been reached. Click [here] to open the entire response in a new Firefox tab."

Clicking on 'here' opens the response as data URL in a new browser tab.

Sebastian
See Also: → 1223726
Product: Firefox → DevTools
Still not really solved. How do we use the ability to bypass the limit in a javascript code. Actually I am loading multiples JSON from ADWORDS XML->JSON through php and cache files exposing the JSON in gzip compression to the javascript in a worker process using XHR object. Then I uncompress the zip content into variable using packo library and vue-worker. Then I transmit the datas to the parent process doing the JSON.parse. But with the 1MB limit my json is truncated at runtime. It is statistical data from AdWords about keywords and the size of the DATA in JSON format is 9MB. 

In Chrome and EDGE the JSON.parse on the 9MB structure to publish the data into boostrap vue tables is working fine and quick without many memory consomption

How can I achevied that with Firefox? I would not like to recommand to my 40 000 companies clients to not use firefox because it can't display their adwords datas in their web interface... 

PS: the whole loading time without caching is around 4 sec and 0.8s with caching under chrome for 5 adwords reports in real time campaign, account, adgroups, ad, keywords (the big one).

Regards.
Nadir, I think your issue is a little different to the one covered here. Yours rather sounds like bug 1281490. This one here is to display a proper message within the response preview of the Network panel when a response is clipped, independent of whether the response is JSON data or not.

Nonetheless, I've attached a screenshot of what's currently displayed within the response preview when selecting a request.

Sebastian
And to be able to quickly reproduce this issue, I've attached a JSON file which is a bit bigger than 1MB.

Steps to reproduce the issue:
1. Open the Network panel
2. Load the attached JSON
3. Select the request
4. Switch to the "Response" tab

Sebastian

Hi, I'd be happy to take this on as long as I could get some pointers please. The code the original reporter is referencing is no longer applicable, so it would be great to know where the code responsible for limiting response logging to 1MB lives.

Flags: needinfo?(sebastianzartner)

I am not the best person to ask about the related code, as I am currently not much involved in DevTools development, but as far as I can see the limitation of the code is happening in /devtools/server/actors/network-monitor/network-response-listener.js controlled by the devtools.netmonitor.responseBodyLimit preference and the preparation of the data and its display is done in /devtools/client/netmonitor/src/components/ResponsePanel.js. I redirected your request to Honza for further information on what exactly needs to be done, as he's the "master of the Network Monitor". :-)

Sebastian

Flags: needinfo?(sebastianzartner) → needinfo?(odvarko)

@Martin: assigned to you.
@Sebastian: thanks for the pointers.


Some more info:

  1. Sebastian's links are correct

  2. Here is the place where the current error message is rendered
    https://searchfox.org/mozilla-central/rev/e00ea598e52bbb35f8c45abf9c2eade17962bb5e/devtools/client/netmonitor/src/components/ResponsePanel.js#236

  3. I think we could display an error message (instead of the current error) saying e.g. "Response has been truncated" (any other better suggestions for the message?)

  4. The code can use the following condition to figure out whether the response has been truncated.

// In the render method
const limit = Services.prefs.getIntPref("devtools.netmonitor.responseBodyLimit");
if (limit <= request.responseContent.content.size) { ... }

Honza

Assignee: nobody → koroknay
Status: NEW → ASSIGNED
Flags: needinfo?(odvarko)

Previously when a JSON response's size was over the limit specified in pref devtools.netmonitor.responseBodyLimit, when inspecting this response in the Network Monitor one simply saw a JSON parse error message. This was due to the fact that the response was truncated in this view and thus no longer a valid JSON, even if the actual response was a valid JSON. Of course the actual response could have been invalid in the first place -- there was no way to know.

This patch will ensure that in such cases a truncation warning message is displayed instead of a parse error message, thus avoiding a likely misleading parse error message.

Thanks for the pointers Sebastian & Honza. I have submitted my patch for review, for now with the text "Response has been truncated." displaying.

@Martin: please look at my comments in Phabricator

Thanks,
Honza

Flags: needinfo?(koroknay)

Hi Honza, I have updated the diff in Phabricator.

Flags: needinfo?(koroknay)
Pushed by jodvarko@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/24a48c472242
Display a useful message in network monitor response tab when response body is truncated. r=Honza
Status: ASSIGNED → RESOLVED
Closed: 8 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 67
Flags: qe-verify+

I reproduced this issue with Fx 66.0a1 (2018-12-17) on Windows 10 x64.
The desired message ( "Response has been truncated") is displayed with Fx 67.0b8 on Windows 10 x64, Ubuntu 18.04 x64 and macOS 10.13.

Status: RESOLVED → VERIFIED
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.