1786827 - highlighting can take so long it kills gunicorn worker

Status: RESOLVED FIXED

(Socorro :: Webapp, defect, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

The report view on Crash Stats shows a stack and for each frame, it links to a syntax highlighted version of the source file with the specified line indicated.

For long source files, the highlighting step (uses pygments) takes so long, gunicorn kills the worker and the user gets back a 502 bad gateway.

Example crash report:

https://crash-stats.mozilla.org/report/index/1c85d9c5-60d3-4591-89ac-8e9810220824

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

In Sentry: https://sentry.prod.mozaws.net/operations/socorro-new-prod/issues/19528510/

Maybe there are other faster highlighting systems than pygments? Maybe there are options we can adjust that reduce the work it does? Maybe we can point to another system at Mozilla instead for viewing the source code? Maybe if the file size is past some number, we don't do highlighting and instead adjust the HTML to indicate the specified line?

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

I've seen a few of these and I'm fiddling with the view code now, so I'm going to grab this to look into.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 6238: bug 1786827: use null lexer for large files

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #6238: "bug 1786827: use null lexer for large files" in 62b225e.

Comment 5

[Will Kahn-Greene [:willkg] ET needinfo? me]

This was deployed in bug #1799933. Marking as FIXED.