Bugzilla

While figuring out how to best implement the (static) settings UI in bug 1703656 without introducing new technical debt, I realized it's probably a good time to switch from the ad-hoc JS templating we're doing to rust logic using our liquid-templating (which also can be done from JS).

The general setup for output.sh right now is:
- parallel job invokes `output-file.rs` over `all-files`
- `output-dir.js` gets invoked to build the directory listings
- `output-template.js` gets invoked to build the "search.html" file that router.py uses
- `output-help.js` gets invoked to build the "help.html" that nginx gets pointed at to be our root /index.html

The JS logic is a combination of:
- `output.js` which has template string providing the HTML prolog plus a bunch of helpers like for breadcrumbs.
- app domain logic for output-dir.js

### Design Considerations

#### Displaying directory tree listings for specific revisions

Right now searchfox will happily show you files at historical revisions, but you'll get a 404 if you try and see a directory listing at a specific revision.  While the changes here probably should not require that we do the revision control magic, it would be good to architect so that this is easier to do when the time comes.

#### Dynamic queries returning files

Tying in with the above, for example, we already have a `PipelineValues::FileMatches` which is currently just a vec of `FileMatch` which is just a `path: String`, but one could imagine:
- augmenting the information to (extensibly) contain the information required for the current directory display listing.
- adding a searchfox-tool "batch" command that knows how to run an arbitrary (preconfigured?) pipeline multiple times and direct the output to disk.
  - It seems like this could be handled either by using search-files to return just a list of directories which could then drive the batch job, or alternately search-files could return everything but this could then be partitioned by grouping by directory and then running the batch against those resulting groups.  Partitioning seems like it should keep things more O(num files) rather than adding any scaling factors that could be magnified if we add extra metadata.
- allowing search-files or some alternate command to basically do `git ls-tree` to provide the information, although noting that it would obviously not have most extra metadata like test "skip-if" info or any knowledge of `__GENERATED__` files.

While figuring out how to best implement the (static) settings UI in bug 1703656 without introducing new technical debt, I realized it's probably a good time to switch from the ad-hoc JS templating we're doing to rust logic using our liquid-templating (which also can be done from JS).

The general setup for output.sh right now is:
- parallel job invokes `output-file.rs` over `all-files`
- `output-dir.js` gets invoked to build the directory listings
- `output-template.js` gets invoked to build the "search.html" file that router.py uses
- `output-help.js` gets invoked to build the "help.html" that nginx gets pointed at to be our root /index.html

The JS logic is a combination of:
- `output.js` which has template string providing the HTML prolog plus a bunch of helpers like for breadcrumbs.
- app domain logic for output-dir.js

The interaction with the static settings page comes from:
- The settings page ideally wants to not introduce yet another copy of the HTML prolog that we have to update all the time.
- There's interesting potential to let any examples in the settings UI be somewhat more realistic by actually embedding real searchfox data into them.
  - This would be something that happens down the road, but could be nice to plan for.
  - This would most sanely probably depend on the searchfox "tests" repo or an "examples" repo for most trees, although it could be neat for mozilla-central to have real-world examples.

### Design Considerations

#### Displaying directory tree listings for specific revisions

Right now searchfox will happily show you files at historical revisions, but you'll get a 404 if you try and see a directory listing at a specific revision.  While the changes here probably should not require that we do the revision control magic, it would be good to architect so that this is easier to do when the time comes.

#### Dynamic queries returning files

Tying in with the above, for example, we already have a `PipelineValues::FileMatches` which is currently just a vec of `FileMatch` which is just a `path: String`, but one could imagine:
- augmenting the information to (extensibly) contain the information required for the current directory display listing.
- adding a searchfox-tool "batch" command that knows how to run an arbitrary (preconfigured?) pipeline multiple times and direct the output to disk.
  - It seems like this could be handled either by using search-files to return just a list of directories which could then drive the batch job, or alternately search-files could return everything but this could then be partitioned by grouping by directory and then running the batch against those resulting groups.  Partitioning seems like it should keep things more O(num files) rather than adding any scaling factors that could be magnified if we add extra metadata.
- allowing search-files or some alternate command to basically do `git ls-tree` to provide the information, although noting that it would obviously not have most extra metadata like test "skip-if" info or any knowledge of `__GENERATED__` files.