1363222 - JSON Viewer is painfully slow

Status: NEW

(DevTools :: JSON Viewer, enhancement, P2)

Description

[Oriol Brufau [:Oriol]]

JSON Viewer is painfully slow to load.

To measure it, I loaded https://a.4cdn.org/boards.json

With JSON Viewer enable, the total load took more than 10 seconds. Among these, 4 seconds were wasted in this call:

    let theApp = render(MainTabbedArea(input), content);

(in render, not in MainTabbedArea).

Disabling the JSON Viewer loads that page in a fraction of a second.

We need to get rid of react and use fast vanilla javascript.

Not sure if this is the same as bug 1281490, the linked example in there is no longer available.

Comment 1

[Oriol Brufau [:Oriol]]

Attachment: with-jsonviewer.png

Comment 2

[Oriol Brufau [:Oriol]]

Attachment: without-jsonviewer.png

Comment 3

[Oriol Brufau [:Oriol]]

Attachment: profile-with-json-viewer.json

Comment 4

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(Copying from bug 1363224, since investigating the performance in a general sense seems worthwhile.)

Using React has greatly improved the team's efficiency and ability to share UI components.  It also greatly simplifies state management compared to pure JS UI code.  DevTools is a small team trying to do a lot, so we need as many wins like this as we can find.

I agree there are various places where React's performance is slower than we'd like.  Rather than trying to dump React entirely since it has other advantages we'd quite like to preserve, we should try our best to profile and optimize slow paths where we can.

Comment 5

[jonkoops]

I understand that there is a small team working on the DevTools, however it seems new features are prioritized over existing problems like this one. The one and only reason why I have to continue to switch back to Chrome for development is the fact that I cannot inspect JSON payloads in Firefox DevTools. Inspecting a reasonably sized JSON payload of ~20mb causes the inspector to freeze the whole browser for at least 4 seconds.

I strongly believe this issue should be prioritized as it would increase developer's willingness to use DevTools over Chrome. This is not even limited to large JSON files as simply clicking through the list of files I noticed a lot of them are quite slow to show the preview, blocking the main thread in the process.

Comment 6

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Raising the priority here.

Honza

Comment 7

[:Harald Kirschner :digitarald]

Test file that provides 6Mb to load: https://data.ct.gov/api/views/2fi9-sgi3/rows.json . Using this I have not seen any severe hangs in Nightly for either Network panel or JSON viewer.

Jon, do you have an STR that you could share?

Comment 8

[jonkoops]

Attachment: Video of browser freezing

Here is a video of the browser freezing up after selecting a large JSON file. As I move the cursor around the screen none of the hover states trigger and clicking the tabs has no effect whatsoever.

Comment 9

[jonkoops]

Attachment: ZIP file with JSON example

Here is the JSON file in question that is causing the browser to freeze up.

Comment 10

[jonkoops]

I am running this in Firefox Nightly 70.0a1 (2019-08-21) (64-bit) on Fedora Workstation 30.

Comment 11

[:Harald Kirschner :digitarald]

I wonder if codemirror is biting us here.

Honza, should we use this bug to disable code mirror syntax highlighting for truncated responses?

Comment 12

[Jan Honza Odvarko [:Honza] (always need-info? me)]

(In reply to :Harald Kirschner :digitarald from comment #11)

I wonder if codemirror is biting us here.

Honza, should we use this bug to disable code mirror syntax highlighting for truncated responses?

CodeMirror is already entirely disabled (not used) for rendering responses in the Response side panel bigger than devtools.netmonitor.response.ui.limit (10240B by default)

Note that I couldn't reproduce the UI Freeze when selecting the points JSON request in the request list.

Honza

Comment 13

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Attachment: image.png

@Alex: I tried to load the file from comment #9 (as top level document) to see how fast/slow the JSON viewer is.

Here is my recorded profile:
https://perfht.ml/2o7u5DN

I can see that ~5s is spent in reflow. But, not sure what is causing that. Any tips?

Honza

Comment 14

[Alexandre Poirot [:ochameau]]

you profiler link doesn't work.

If you go to MarkerChart, you will see a "Reflow" line.
Then, if you mouse over the reflows, you should see the stack.
If the stack is cut too short, you have a context menu to get the full one.

The stack will tell you what code is triggering the reflow.
If you many reflows, that's an issue and you should try to figure out how to do less.
If you have just one or a couple, you would need to understand why the reflow is so slow: slow CSS rule? too many DOM elements?

Comment 15

[Oriol Brufau [:Oriol]]

I think https://searchfox.org/mozilla-central/rev/35cc00a25c4471993fdaa5761952bd3afd4f1731/devtools/client/jsonview/converter-child.js#168 may be the culprit. (Well, one of the culprits, since I filed this bug before adding that line)

Before bug 1417035, the data was sent to the native HTML parser. To avoid parsing it as HTML, there was a <plaintext> element, but that had its own problems as well.

I used the paint flashing devtool and it seemed that using CharacterData.prototype.appendData to insert the data with JS would only repaint the last part. However, I suspect it causes synchronous reflows, unlike the native HTML parser.

So if you have a huge JSON file loaded from the disk, the stream converter receives lots of chucks in a short amount of time, and the JS inserts them synchronously, and this causes a reflow, which takes some time because the chunk is big, and before the reflow finishes, the converter receives another big chuck to insert. This ends up freezing the process.

I suspect performance could improve by caching the received data in some variable in converter-child.js instead of immediately inserting it into the page. And then only insert the data at the end if the data is loaded at high speed (like from the disk). But for slow servers that send tiny amounts of data little by little, we could still flush every second or so (using a setTimeout to detect that the process is not busy).

Comment 16

[:Harald Kirschner :digitarald]

However, I suspect it causes synchronous reflows, unlike the native HTML parser.

Just my 2c from looking at https://perfht.ml/2o7u5DN , it seems to show only 2 heavy reflows, so not sure if chunked responses are the issue here.