Closed Bug 1640978 Opened 6 years ago Closed 1 year ago

Searching for `message` produces blank page

Categories

(Webtools :: Searchfox, defect)

Product:
Webtools
Component:
Searchfox
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: kats, Unassigned)

References

(Regression,
URL
)

Details

(Keywords: regression)

https://searchfox.org/mozilla-central/search?q=message

This works sometimes, but sometimes iterating all the identifiers takes longer than the 15 seconds allotted to the request handler. There's code at https://github.com/mozsearch/mozsearch/blob/4e584d1e7f72f98809e492ddaac2fe3c996b96b0/router/router.py#L261 that only processes the first 500 identifiers, presumably in the hopes that it will complete within the time allotted, but sometimes it doesn't.

Probably a regression from running things in python3 as there's more byte/char conversions everywhere

Can confirm, always gives a 504 error on generic search queries with too many results.

If you can post a few more queries it would help verify this bug when we do eventually fix it. Some of the issues may also be caused by bug 1640958 which has very similar symptoms but was a totally different underlying problem.

FWIW searching for message works today. In the log I see it does emit WORK LIMIT HIT for the search, but that's after it iterates the 500 identifiers.

If you do have other generic search queries that still reproduce the error please do post them here.

The identifier result cap of 500 means that to maximize runtime you'd want to pick a left-hand side prefix query whose first 500 identifier results from the identifiers table have maximum JSON size. awk '{print length, $0}' crossref | sort -nr > line-lengths with a TMPDIR that's big enough nets us top hits of:

  • uint32_t: 33.6 MiB
  • builtin__expect: 23.6 MiB
  • int32_t: 23.6 MiB
  • nsresult: 23.4 MiB
  • size_t: 22.1 MiB
  • AnnotateMozCrashReason: 19.8 MiB
  • MOZ_AssertAssignmentTest: 19.6 MiB
  • AssertionConditionType: 19.1 MiB
  • abort: 18.8 MiB
  • JSObject: 17.5 MiB
  • MOZ_ReportAssertionFailure: 17.2 MiB
  • isValid: 16.7 MiB
  • __debugbreak: 15.5 MiB
  • MOZ_NoReturn: 15.4 MiB
  • Message: 15.2 MiB
  • JSContext: 14.8 MiB
  • operator: 13.2 MiB
  • uint8_t: 12.9 MiB
  • Handle: 12.0 MiB
  • <<: 10.0 MiB
  • RefPtr: 9.3 MiB
  • MOZ_ASSERT: 9.0 MiB

So it seems like "MOZ_" is the winner and "message" did so well owing to the presence of "Message" in the list above and its 1516 (== 4286035 - 4284519) candidate lines because the "message~" needle sorts just after the "messagestream_win::get_pipe_name audioipc::messagestream_win::get_pipe_name" line which means it gets a full 500 lookups.

But https://searchfox.org/mozilla-central/search?q=MOZ_&path= loads fine for me so maybe everything was just bug 1640958 after all?

Whoops, uint totally wins with a 504 for me just now: https://searchfox.org/mozilla-central/search?q=uint&path=

We actually perform pretty well in all of the example cases now, presumably partly due to performance optimizations, but I think we also may have fixed a logic error in the work limit mechanism at some point, possibly when we added reporting of when limits are hit? We definitely report this in all of the cases now. That said, bug 1794004 does exist, but there's no need for this bug to track that one.

Status: NEW → RESOLVED
Closed: 1 year ago
Keywords: regression
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.