On glyph/SVG specifically:
My understanding so far is that we allow up to 8 worker threads in Rayon, and the same threadpool is used by the blob rasterizer and the glyph rasterizer. When we're busy doing long-running blobs (SVG), the high priority UI thread might ask for a glyph and stall on the result, with no way to help out and do the glyphs rather than block.
I can't find a clear yes/no answer if Rayon will allow the asking-and-blocking thread to enter the threadpool and help out, building the result that it's blocking on. But even if it does, it wouldn't help us, because request_glyphs_from_backend blocks on a glyph_rx.recv(), and nothing gets glyph_tx.send() until the entire par_iter+map+collect has finished. So it's invisible to Rayon that the blocking recv could help out with the work inside par_iter.
A quick fix would be to use try_recv instead and if not ready, add this_thread to the worker pool. I'm not sure if Rayon supports that (install() is the exact opposite), and even so there's a risk that the extra "worker" will pick up even more SVG work (likely if Rayon uses a task FIFO).
An alternative would be to manually implement work stealing; don't use par_iter, but put all glyphs in a shared vector, and use atomicAdd to take work from threads insert()'ed into the threadpool, or whatever. If resolve_glyphs sees outstanding work, it can also grab some and help out.
But what we really might want is priorities on the outstanding Rayon work. When we schedule UI glyphs, they should take precedence over SVG work. If we try to do it at the Rayon level, then we have two problems: 1/ the SVG work must be yieldable, ie. implemented as a chain of short lived tasks, and 2/ Rayon doesn't seem to have priorities :)
So an option might be to use the OS instead: have 2 worker threadpools of 8 workers each, a low priority pool and a high priority pool. In start_handler we can (hopefully) set the priorities, and also use thread affinity to make sure that each core only runs one or the other -- we don't want 16 threads total active. Then it becomes the OS's problem to context switch from SVG to glyph rendering, which it actually can do (unlike our atomic handling of entire tasks).
Skipping all this machinery for a small amount of glyphs is probably still worthwhile to reduce overhead, but it seems like we need something more general.
I hope I'm not analyzing and speculating widely off the mark here. Thanks!