1616902 - AttributeError: module 'selectors' has not attribute 'SelectSelector' in `mach try fuzzy` preview pane

Status: RESOLVED FIXED

(Developer Infrastructure :: Try, defect, P3)

Description

[Andrew Halberstadt [:ahal]]

Attachment: Screenshot from 2020-02-20 12-15-08.png

See attached screenshot for the traceback.

Note this happens whether or not I pass --show-estimates into mach try fuzzy. Looks like it's happening on import :/. A quick Google search doesn't seem to lead to other results.

Comment 1

[Andrew Halberstadt [:ahal]]

Ah, subprocess must be finding the tools/tryselect/selectors directory instead of the proper one. Maybe it only tries to import from selectors in Python 3? Not sure what changed to start this happening.

Comment 2

[Simon Fraser [:sfraser] ⌚️GMT]

(In reply to Andrew Halberstadt [:ahal] from comment #1)

Ah, subprocess must be finding the tools/tryselect/selectors directory instead of the proper one. Maybe it only tries to import from selectors in Python 3? Not sure what changed to start this happening.

I think you're right: 2.7.15/lib/python2.7/subprocess.py has no mention of selectors. However, using requests from pip install doesn't cause this. Upgrading the vendored version from 2.9.1 to 2.23.0 seems to work.

Comment 3

[Tom Prince [:tomprince]]

Should tryselect be turned into a real package, rather than having a bunch of modules/packages at the top of the module hierarchy?

Comment 4

[Andrew Halberstadt [:ahal]]

I don't think a real package is necessary for tryselect. It only makes sense in the context of mozilla-central (you'd never want to pip install it for example).

I'm not sure how tools/tryselect is ending up on the PYTHONPATH. The tools directory should be there because it's listed in virtualenv_packages.txt (which is how we're able to do import tryselect).. but there's nowhere that I can see where we add tools/tryselect to the path.

Comment 5

[Simon Fraser [:sfraser] ⌚️GMT]

(In reply to Andrew Halberstadt [:ahal] from comment #4)

I don't think a real package is necessary for tryselect. It only makes sense in the context of mozilla-central (you'd never want to pip install it for example).

I'm not sure how tools/tryselect is ending up on the PYTHONPATH. The tools directory should be there because it's listed in virtualenv_packages.txt (which is how we're able to do import tryselect).. but there's nowhere that I can see where we add tools/tryselect to the path.

If a script is run using python scriptname then the file's directory is added to sys.path. https://docs.python.org/3/using/cmdline.html?highlight=pythonpath#interface-options indicates -I would help, if we didn't need to set PYTHONPATH for the preview script.

Comment 6

[Andrew Halberstadt [:ahal]]

Ah right, and the preview script is in the root tryselect dir.

I'd like to move that script to tools/tryselect/selectors/preview.py anyway, so I think that's the best solution here.

Comment 7

[Andrew Halberstadt [:ahal]]

Actually the problem with moving fuzzy.py to fuzzy/__init__.py is that we'll hit that issue where people who update to an old revision will have a leftover .pyc file lying around, and then ./mach try fuzzy will break.

So I guess I'll just move preview.py to selectors for now.

Comment 8

[Andrew Halberstadt [:ahal]]

Ok.. I have it working.. but even without -s the preview script is super slow to update. E.g, if I hit Ctrl-A it seems to just freeze. This used to update instantaneously :/.

Can we go back to using the old command (i.e not even load preview.py in the first place) if not using --show-estimates?

Comment 9

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1616902 - [try] Move tools/tryselect/preview.py to the 'selectors' directory, r?sfraser

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/28279f146bf8
[try] Move tools/tryselect/preview.py to the 'selectors' directory, r=sfraser

Comment 11

[Simon Fraser [:sfraser] ⌚️GMT]

(In reply to Andrew Halberstadt [:ahal] from comment #8)

Ok.. I have it working.. but even without -s the preview script is super slow to update. E.g, if I hit Ctrl-A it seems to just freeze. This used to update instantaneously :/.

I'm running something like this with 4000 tasks in the 'tasklist' file, and it seems quick, on the order of 0.3 seconds:

time python  tools/tryselect/selectors/preview.py -s -g ~/.mozbuild/srcdirs/mozilla-central-fd486ef90507/cache/taskgraph/target_task_dependencies -c ~/.mozbuild/srcdirs/mozilla-central-fd486ef90507/cache/taskgraph/ $(cat tasklist)

It looks like only providing the tasklist saves 0.08s. What does your preview command-line look like? Are we hitting the 'argument list too long' error again?

Can we go back to using the old command (i.e not even load preview.py in the first place) if not using --show-estimates?

Probably easiest if we have a separate file - the embedded code in a string was a little harder to read.

Comment 12

[Andrew Halberstadt [:ahal]]

Attachment: fuzzy.gif

Here's a (way too small) gif showing what I mean.

Basically just hit Ctrl-A on the default fuzzy screen and you'll see what I mean.

Comment 13

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/28279f146bf8

Comment 14

[Simon Fraser [:sfraser] ⌚️GMT]

(In reply to Andrew Halberstadt [:ahal] from comment #12)

Created attachment 9128207 [details]
fuzzy.gif

Here's a (way too small) gif showing what I mean.

Basically just hit Ctrl-A on the default fuzzy screen and you'll see what I mean.

We have slightly different numbers of tasks, but I had 3310 and it was near-instant when pressing Ctrl-A. Yours has 3332, I'm not sure if that would make a breaking difference without a bit more testing.