1761701 - Implement pipeline-server.rs to expose allow-listed searchfox-tool functionality built on tokio, tower, axum

Status: RESOLVED FIXED

(Webtools :: Searchfox, enhancement)

Description

[Andrew Sutherland [:asuth] (he/him)]

The next step of the plan at https://bugzilla.mozilla.org/show_bug.cgi?id=1749232#c0 (which will continue to be updated to show progress) is to expose specific portions of searchfox-tool functionality to the web via a new web server, pipeline-server.rs. Ideally this will eventually replace the existing web-server.rs which is, I think, stuck somewhat in the past on a very old version of hyper[1].

The current plan is to use the core tokio ecosystem stack of axum/tower_http/tower/(hyper)/tokio.

1: We tried to upgrade it once before and ran into some structural problems; quoting :kats in 9f11dc964ed625e4812a6d8dea8c630de0976928 "The newer version of hyper had worse behaviour in that any panic would take down the whole server instead of just one thread."

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

Attachment: PR: landed progress commits for pipeline-server

PR stack is https://github.com/mozsearch/mozsearch/pull/505 with pipeline-server.rs specific pieces primarily present in the with-bug-number commit https://github.com/mozsearch/mozsearch/pull/505/commits/1baf33c54f98b9b8dab79483b454e5144721d06f

The commit did leak a speculative nginx route for "/TREE/query" but this is harmless other than being a way for people to experience 5xx status codes.

This is landing incrementally with tests at each stage so people can follow along as things happen instead of having a single mega-landing at the end.

Comment 2

[Andrew Sutherland [:asuth] (he/him)]

Attachment: Landed PR: pipeline-server.rs MVP producing classic search-styled query JSON output

Comment 3

[Andrew Sutherland [:asuth] (he/him)]

Calling this sufficiently implemented to be marked fixed; I'll file a new bug for the UI hookup and templating discussed as next steps in the PR.