1854218 - improve syminfo API uncached lookup time

Status: RESOLVED FIXED

(Tecken :: General, defect, P2)

Description

[Will Kahn-Greene [:willkg] ET needinfo? me]

The syminfo API takes too long to look up a record when it's not in the cache.

Data:

debug_file,debug_id,code_file,code_id
xul.pdb,6B8753C71BD035D54C4C44205044422E1,xul.dll,650B24A3EE56000
xul.pdb,37F7581DBA96E81D4C4C44205044422E1,xul.dll,650B23E18A1B000
xul.pdb,908C1D71C52124544C4C44205044422E1,xul.dll,650B25737A25000
xul.pdb,18CF28149AB527FB4C4C44205044422E1,xul.dll,650B24AF8362000
xul.pdb,699B1D693B39A66A4C4C44205044422E1,xul.dll,650B23319467000

Timings: 4.5s on average

The API is cached, so if we do a second request for the same thing, that's 0.5s on average--that's fine.

However, the uncached request needs to be faster than that. Preferably under 1s.

Comment 1

[Will Kahn-Greene [:willkg] ET needinfo? me]

Attachment: pr 2804: bug 1854218: add indexes to improve syminfo API times

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

willkg merged PR #2804: "bug 1854218: add indexes to improve syminfo API times" in 479c5fc.

We'll want to time how long the migration takes. Plus I'll do syminfo timings before and after.

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

Harold ran the migration. Stage has a copy of prod's database. The migration took 0.5s. Timings are a lot better now.

Before (1.6s - 2.0s):

2023-10-02 12:58:09,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650B24AF8362000,200,1.7558274269104004
2023-10-02 12:58:09,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650B23E18A1B000,200,1.876110315322876
2023-10-02 12:58:09,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650C1C9080B1000,200,2.046523094177246
2023-10-02 12:58:09,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650C1CC092D3000,200,1.8432397842407227
2023-10-02 12:58:09,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650B23319467000,200,1.7528691291809082
2023-10-02 12:58:09,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650B24A3EE56000,200,1.8975398540496826
2023-10-02 12:58:09,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650B25737A25000,200,1.773305892944336
2023-10-02 12:58:09,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650C1C8A919A000,200,1.6675662994384766

After indexes (0.13 - 0.14s):

2023-10-02 13:26:12,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650B24AF8362000,200,0.14202594757080078
2023-10-02 13:26:12,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650B23E18A1B000,200,0.13618707656860352
2023-10-02 13:26:12,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650C1C9080B1000,200,0.14935541152954102
2023-10-02 13:26:12,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650C1CC092D3000,200,0.1374979019165039
2023-10-02 13:26:12,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650B23319467000,200,0.13735079765319824
2023-10-02 13:26:12,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650B24A3EE56000,200,0.14732670783996582
2023-10-02 13:26:12,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650B25737A25000,200,0.1358046531677246
2023-10-02 13:26:12,479c5fced804e7f3a46b87e05a4d65da55d06dd2,xul.dll,650C1C8A919A000,200,0.1433696746826172

Stage runs faster than prod, but this suggests we'll see similar dramatic improvements in prod.

Comment 4

[Will Kahn-Greene [:willkg] ET needinfo? me]

This was deployed to prod in bug #1856685.

Before indexes (4.4s - 5.2s):

2023-10-03 14:30:27,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650B24AF8362000,200,4.4307310581207275
2023-10-03 14:30:27,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650B23E18A1B000,200,4.616220235824585
2023-10-03 14:30:27,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650C1C9080B1000,200,4.39372181892395
2023-10-03 14:30:27,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650C1CC092D3000,200,5.033223867416382
2023-10-03 14:30:27,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650B23319467000,200,4.4930195808410645
2023-10-03 14:30:27,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650B24A3EE56000,200,5.217177629470825
2023-10-03 14:30:27,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650B25737A25000,200,4.4167211055755615
2023-10-03 14:30:27,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650C1C8A919A000,200,4.66240930557251

After indexes (0.3s - 0.4s):

2023-10-03 16:13:40,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650B24AF8362000,200,0.38634610176086426
2023-10-03 16:13:40,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650B23E18A1B000,200,0.38417887687683105
2023-10-03 16:13:40,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650C1C9080B1000,200,0.36920976638793945
2023-10-03 16:13:40,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650C1CC092D3000,200,0.36362218856811523
2023-10-03 16:13:40,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650B23319467000,200,0.43492650985717773
2023-10-03 16:13:40,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650B24A3EE56000,200,0.36658358573913574
2023-10-03 16:13:40,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650B25737A25000,200,0.3866908550262451
2023-10-03 16:13:40,ee97ec8dcc6ce7c3e46ba718fe793830fb407e3d,xul.dll,650C1C8A919A000,200,0.3667166233062744

That's sub-1s, so that's good enough for now.

Marking as FIXED.