1852645 - mach try chooser doesn't do anything

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Timothy Nikkel (:tnikkel)]

I run ./mach try chooser, pick some tests, click submit, and just wait. Nothing happens. It doesn't even appear to get to the part where it talks to the try server (which sometimes can take a long time). ./mach try fuzzy works. I experience this on two machines at least, one macOS, one Windows. I've done a fresh pull, killed my objdir, and killed the _virtualenvs dir in .mozbuild, and the problem still happens.

bisected to
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=0489416a8aec949f888e29303c9bbe21ed1f99e7&tochange=e7a99f508f56aad179face2013609fc9498d0c8f
-> bug 1786490

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1786490

:Logan, since you are the author of the regressor, bug 1786490, could you take a look?

For more information, please visit BugBot documentation.

Comment 2

[Logan Rosen [:Logan]]

Attachment: Bug 1852645 - fix mach try chooser regression r?ahal

With the fix to Bug 1786490, we are now using Werkzeug 2.1.2, which removed the shutdown function that the chooser was relying on to shut down the Flask server after selecting tasks. To remedy this, I followed the instructions here for shutting down the server using multiprocessing instead.

Comment 3

[Logan Rosen [:Logan]]

Sorry about that! Fix is incoming.

Comment 4

[Timothy Nikkel (:tnikkel)]

Thanks for the quick fix!

Comment 5

[Marco Bonardo [:mak]]

I tried applying this patch, but when I run try chooser I get

ValueError: cannot find context for 'fork'

  File "D:\src\tools\tryselect\mach_commands.py", line 362, in try_chooser
    return run(command_context, **kwargs)
  File "D:\src\tools\tryselect\mach_commands.py", line 196, in run
    return mod.run(**kwargs)
  File "d:\src\tools\tryselect\selectors\chooser\__init__.py", line 85, in run
    multiprocessing.set_start_method("fork", force=True)
  File "D:\mozilla-build\python3\lib\multiprocessing\context.py", line 247, in set_start_method
    self._actual_context = self.get_context(method)
  File "D:\mozilla-build\python3\lib\multiprocessing\context.py", line 239, in get_context
    return super().get_context(method)
  File "D:\mozilla-build\python3\lib\multiprocessing\context.py", line 193, in get_context
    raise ValueError('cannot find context for %r' % method) from None

Comment 6

[Logan Rosen [:Logan]]

I see – I tried to force using forking to work around an issue with pickling objects on MacOS (where spawn is the default), but it looks like Windows only supports spawning anyway. Will need to find another way...

Comment 7

[Logan Rosen [:Logan]]

I just put up a new revision that doesn't force using fork. Please let me know if that works for you.

Comment 8

[Marco Bonardo [:mak]]

It is working on Windows 11 with the latest patch, with some additional output I don't remember seeing usually (but I may be misremembering).

$ ./mach try chooser
not found!
Starting trychooser on http://127.0.0.1:5000
 * Serving Flask app 'tryselect.selectors.chooser.app' (lazy loading)
 * Environment: development
 * Debug mode: off
 * Running on http://127.0.0.1:5000 (Press CTRL+C to quit)
127.0.0.1 - - [13/Sep/2023 09:26:55] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [13/Sep/2023 09:26:55] "GET /static/style.css HTTP/1.1" 200 -
127.0.0.1 - - [13/Sep/2023 09:26:55] "GET /static/filter.js HTTP/1.1" 200 -
127.0.0.1 - - [13/Sep/2023 09:26:55] "GET /static/select.js HTTP/1.1" 200 -
127.0.0.1 - - [13/Sep/2023 09:26:55] "GET /favicon.ico HTTP/1.1" 404 -
127.0.0.1 - - [13/Sep/2023 09:27:30] "POST / HTTP/1.1" 200 -
127.0.0.1 - - [13/Sep/2023 09:27:30] "GET /static/style.css HTTP/1.1" 304 -
estimates: Runs 252 tasks (226 selected, 26 dependencies)
estimates: Total task duration 4 days, 18:24:48
estimates: In the top 13% of durations
estimates: Should take about 1:41:19 (Finished around 2023-09-13 11:08)
Creating temporary commit for remote...
A try_task_config.json
pushing to ssh://hg.mozilla.org/try
searching for changes
...

Comment 9

[Logan Rosen [:Logan]]

Yeah, that's the output from the development server that's running. I'm not sure if it was suppressed intentionally before, but it's now getting logged because we're spawning a separate process. Let me know if you think it's disruptive, and I can try to suppress it.

Comment 10

[Marco Bonardo [:mak]]

As far as chooser works I don't mind, it just seemed a bit redundant to log all the GETs, and thus I pointed it out.

Comment 11

[Donal Meehan [:dmeehan]]

:ahal do you think you can review and land this patch this week?
Nightly soft freeze starts this week since next week if merge day for Fx119

Comment 12

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1786490

Comment 13

[Andrew Halberstadt [:ahal]]

Sorry for the delay there. I've queued it in lando.

Comment 14

[Kagami Rosylight [:saschanaz] (they/them)]

Seems it failed to land: https://lando.services.mozilla.com/D187952/

Can one of you rebase and reland it? (I can do that if you can't)

Comment 15

[Ryan VanderMeulen [:RyanVM]]

I did the rebase.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d1c09d187a7e
fix mach try chooser regression r=ahal,jgraham

Comment 17

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/d1c09d187a7e

Comment 18

[Olli Pettay [:smaug][[e-mail]]]

So I need to still always press ctrl+C after using try chooser. Is that expected?

Comment 19

[Timothy Nikkel (:tnikkel)]

I don't have to do that (on macos).

Comment 20

[Kagami Rosylight [:saschanaz] (they/them)]

Do you experience the same with ./mach try chooser --push-to-lando, Olli? (I never use the plain try submission nowadays, so just curious)

Comment 21

[Olli Pettay [:smaug][[e-mail]]]

Haven't used --push-to-lando yet (since I haven't managed to get that working with ./mach try perf, which is what I use all the time)