Closed Bug 2091444 Opened 12 days ago Closed 10 days ago

TrackingFlags bug_check_can_change_field runs a database query for every custom field on show_bug.cgi, including fields that aren't tracking flags

Categories

(bugzilla.mozilla.org :: Extensions, enhancement)

Production
enhancement

Tracking

()

RESOLVED FIXED

People

(Reporter: dkl, Assigned: dkl)

Details

Attachments

(1 external file)

When show_bug.cgi renders, the bug page asks whether each field can be changed (Bugzilla::Bug::check_can_change_field, about 50 calls per page, mostly from bug_modal/field.html.tmpl). For every cf_ field, the TrackingFlags extension's bug_check_can_change_field hook then looks the field up as a tracking flag:

return if $field !~ /^cf_/ or $old_value eq $new_value;
return
  unless my $flag
  = Bugzilla::Extension::TrackingFlags::Flag->new({name => $field});

That causes two problems:

  1. Every lookup is a database query. Flag->new only uses the request cache (request_cache->{tracking_flags}, filled by Flag->get_all) for lookups by id. A lookup by name always goes to the database, even when get_all has already loaded every flag in the same request, which it does on show_bug.cgi through bug_create_cf_accessors.
  2. Most cf_ fields aren't tracking flags. They are other custom fields (for example cf_rank or cf_fx_points), so the query finds nothing. In the dev database, 19 of 27 custom fields aren't tracking flags.

Measured

Local dev stack, Devel::NYTProf, /show_bug.cgi?id=1, same data before and after:

Before After
Database queries (DBI::st::execute) 100 86
Flag->new by name from the hook 15 calls (6.1 ms) 0
Bugzilla::Bug::check_can_change_field 10.9 ms 3.4 ms

On this dev data none of the 15 lookups were tracking flags, so the whole saving comes from skipping non-tracking fields. Production has many more tracking flags and custom fields, so both parts of the fix should matter more there.

Steps to reproduce

  1. Start the dev stack with USE_NYTPROF=1.
  2. Load /show_bug.cgi?id=1.
  3. In the profile, Bugzilla::Extension::TrackingFlags::Flag::new is called from bug_check_can_change_field once per cf_ field whose value is being checked, and each call runs a query.

Expected

Fields that aren't tracking flags should be skipped without a query, and tracking flags already loaded in the request should not be loaded again.

Proposed fix

  1. bug_check_can_change_field: return early unless the field is a tracking flag, using a set built once per request from Bugzilla->tracking_flag_names (one query per request, already used the same way in Bugzilla::Search).
  2. Flag->new({name => ...}): if request_cache->{tracking_flags} exists, look the flag up in a name index built from it. Matching is case-insensitive, like the database lookup.
    • The index is rebuilt when the cache is replaced or flags are added or removed.
    • A hit is only returned if the flag is still cached under its id with the same name, because update() and remove_from_db() change the cache in place.
    • A miss falls back to the database, because the cache can be partial (see below).

A new test, extensions/TrackingFlags/t/flag-cache-by-name.t, covers cache hits, case-insensitive names, falling back to the database, renamed and removed flags, unchanged id lookups, and the hook skipping fields that aren't tracking flags.

Related, not fixed here

  • Flag::new_from_hash does request_cache->{tracking_flags} //= {}. If any flag object is built before get_all runs in a request, get_all finds the key, skips loading, and returns an empty or partial list. On show_bug.cgi get_all happens to run first. This is why the new name lookup falls back to the database on a miss instead of treating the cache as complete.
  • _tracking_flag_names reads tracking_flag_names from memcached but never writes it, so it always costs one query per request.

Authored by https://github.com/dklawren
https://github.com/mozilla/bmo/commit/5d48fa11daa171f2b4376b70356471246bc320fc
[master] Bug 2091444 - TrackingFlags bug_check_can_change_field runs a database query for every custom field on show_bug.cgi, including fields that aren't tracking flags

(via GitHub webhook, authenticated as shared-secret)

Status: ASSIGNED → RESOLVED
Closed: 10 days ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: