My current plans for the query mechanism in bug 1762817 build very heavily on the searchfox-tool/insta check pipeline mechanism. While I think this is a very reasonable approach to structure the fundamentally complex things going on under the hood, it's still a lot of complexity under the hood! So it behooves me to implement the diagnostic tooling before moving forward with the query mechanism. It would then be a goal that this mechanism can be used when performing any server query by adding
&debug=1 to the query URL.
- Render to markdown that gets committed into the tree whenever test cases are updated. No one should have to run anything to understand what's going on!
- We're writing and committing to markdown since this allows for a relatively human-readable, diff-friendly output format, but that it can also be easily rendered to HTML.
- Graphs will be rendered to graphviz "dot" syntax which is what will be incorporated into the markdown.
- mermaid is another great diagram format with the advantage of easier dynamic rendering from its source syntax (no WASM), but we're making a deep commitment to graphviz and its sophisticated clustering and table rendering support. (In particular, the underlying dagre layout engine that was used by mermaid last time I checked was notably less powerful for my specific searchfox use-cases and development at that time had ceased.) So the choice of graphviz should allow more sophisticated diagrams that can potentially reuse in-tree code.
- Ensure the generated markdown files can be viewed with their diagrams starting from github. This may mean that we add an mdbook gh-pages setup.
- I'm fine if there's a little bit of manual effort required to keep this up-to-date initially. Specifically having to run a make command in the VM followed by some shell script outside the VM is an acceptable first step.
- Part of my rationale on this is that ideally we'll start having searchfox index itself, hosted on searchfox.org, and in that case that would be the step that we do the markdown rendering, in which case the github pages automation would be moot and potentially need to be removed.
- Graph of the pipeline topology; nice-to-have if we can hack things up so that clicking on the pipeline nodes jumps to the presumable heading anchor.
- Indication of what server actions were taken at each step. Ideally this could be done by integrating tokio tracing so that additional tracing instrumentation would show up automatically-ish.
- Stable JSON dump of the pipeline state after each step of the pipeline. We already have representations for all values already, so this is probably the easiest part, modulo perhaps some refactoring to avoid code duplication.