1796258 - Raise Global Dispatcher Limit to 1_000_000

Status: RESOLVED FIXED

(Data Platform and Tools :: Glean: SDK, enhancement, P2)

Description

[Perry McManis [:perry.mcmanis]]

Once we use an unbounded queue, it should be possible to raise the global dispatch rate to 20k as it doesn't have to be pre-allocated.

Once confirmed that use of an unbounded queue is acceptable and does not cause any regressions, we will test raising the limit, which should hopefully help to address pre-init overflow on FOG.

Comment 1

[Chris H-C :chutten|PTO (back Oct 23)]

*address current pre-init overflow on FOG.

There are some changes waiting in the wings (notably bug 1790872) that'll require the limit to be even higher (or for a different solution to be found). I'd like to push for the limit to be configurable by embedders.

Yes, I realize that the main way embedders configure Glean is via init which is something that happens notably after pre-init. I'm not sure how best to configure this... cargo features would be a clumsy fit to try and tune this, maybe cargo config [env] blocks and the option_env! macro would work (or moz.build, honestly) (but how do we ensure the env is set when building the Rust crates), we could add another API that can be called during I dunno static init or any other time before the queue overfills....

I dunno how it'd work. But without it I see me having to shepherd Glean SDK releases regularly as new pre-init instrumentation is added to Firefox Desktop.

Maybe FOG just goes full unbounded?

Comment 2

[Perry McManis [:perry.mcmanis]]

From SDK meeting: use an optional flag to conditionally set the global dispatch queue limit for the pre-init queue to 1 million on desktop

Comment 3

[Chris H-C :chutten|PTO (back Oct 23)]

Attachment: pr 2239: bug 1796258 - Introduce cargo feature preinit_million_queue

Comment 4

[Chris H-C :chutten|PTO (back Oct 23)]

chutten merged PR #2239: "bug 1796258 - Introduce cargo feature preinit_million_queue" in e11e511.