Closed Bug 1426456 Opened 7 years ago Closed 3 months ago

Add analyzer for python code via scip-python

Categories

(Webtools :: Searchfox, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: kats, Assigned: asuth)

Details

It should be possible to add analysis for python the same way we do c++ and js code. Some useful links:

https://docs.python.org/2/library/ast.html
https://greentreesnakes.readthedocs.io/en/latest/

And sample code to get the AST:

import ast
m = ast.parse(open("some-python-file.py", "r").read(), "some-python-file.py")
ast.dump(m)

https://github.com/sourcegraph/scip-python (based on pyright) is I think now the clear choice for us to use.

Summary: Add analyzer for python code → Add analyzer for python code via scip-python

I turned on some stuff in https://github.com/mozsearch/mozsearch-mozilla/pull/227 building on some work in https://github.com/mozsearch/mozsearch/pull/690 plus of course a ton of the SCIP work by Emilio, myself, and Nicolas, but with many, many thanks to the team at sourcegraph for their work on scip and scip-python and of course the underlying pyright team.

A lot of python code is now indexed, but not all of it. scip-python just falls down and eats CPU forever if we let it try and ingest the wpt tree and there was no easy way to get any insight about what was going on inside scip-python because the interaction with pyright ends up being to just turn a crank as type inference logic happens, but there are no instrumentation hooks to easily understand what data it's processing that is overwhelming it.

I dropped a note about this in the slack python channel but do not plan to update the root help.html matrix or send an update to dev-platform anytime soon because the level of python semantic indexing people should expect is none. I'm also going to stick to this being m-c only for now.

Assignee: nobody → bugmail
Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.