Closed Bug 1320140 Opened 7 years ago Closed 7 years ago

Network Predictor Performance Regressions in Nightly 2016-11-19

Categories

(Core :: Networking, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla53
Tracking Status
firefox53 --- fixed

People

(Reporter: chutten, Assigned: u408661)

Details

(Whiteboard: [necko-active])

Attachments

(1 file)

The Telemetry Alerts system detected two changes in network predictor probes. One for PREDICTOR_PREDICT_TIME_TO_ACTION[1] and another for PREDICTOR_PREDICT_WORK_TIME[2].

The changes appear to be slight performance regressions where the 75th percentile has risen from 0ms to 2ms, a level it hasn't been since June 16th: https://mzl.la/2fVQcGL

The changeset for that build[3] doesn't contain the words "predict" or "network" so I'm at a bit of a loss to narrow down what might have caused it.

Action Items:
1) Determine what caused the regression
2) Figure out if that's expected, or if it needs to be remedied
3) Add "alert_emails" fields to the two probes.

(( The alert_emails field[4] should contain an email address (usually for a mailing list of persons who would like to be notified when the probe changes). ))

[1]: http://alerts.telemetry.mozilla.org/index.html#/detectors/1/metrics/592/alerts/?from=2016-11-19&to=2016-11-19
[2]: http://alerts.telemetry.mozilla.org/index.html#/detectors/1/metrics/646/alerts/?from=2016-11-19&to=2016-11-19
[3]: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=28e2a6dde76ab6ad4464a3662df1bd57af04398a&tochange=f09e137ead39230eaa94f47988ccce2cfcda4195
[4]: https://developer.mozilla.org/en-US/docs/Mozilla/Performance/Adding_a_new_Telemetry_probe
Flags: needinfo?(hurley)
So, taking a look at the pushlog, the only thing in there that looks like a plausible culprit to me is the rusturl stuff (the predictor does do a fair amount of url creating). Valentin, do you think the rusturl stuff could have had this impact? Otherwise, I'm at a loss.
Flags: needinfo?(hurley) → needinfo?(valentin.gosu)
Yes, the rusturl did introduce a significant performance issue, but that should be fixed following bug 1318432 (we turned it off via a pref).
Chris, is the regression still present in the telemetry?
Flags: needinfo?(valentin.gosu) → needinfo?(chutten)
The patch in that bug should show up in 20161123

Unfortunately we tried switching something on the backend around then which has since been backed out (bug 1286868). We're backfilling, but for now the aggregate data behind https://mzl.la/2fVQcGL isn't there for the period in question.

I'll leave my ni? up so I remember to look at this tomorrow (~20 hours left in the backfill job, I'm told) so I can give you a better answer then.
Whiteboard: [necko-active]
Yup! This went away ( https://mzl.la/2gFmkNg ).

All that's left is to put appropriate email information in the alert_emails fields of the two probes. There are lots of examples in Histograms.json you can follow. Do you have a mailing list address that would be appropriate?
Flags: needinfo?(chutten) → needinfo?(valentin.gosu)
We have been using necko@mozilla.com
Flags: needinfo?(valentin.gosu)
Assignee: nobody → hurley
Comment on attachment 8823346 [details]
Bug 1320140 - add reporting emails for some predictor telemetry.

https://reviewboard.mozilla.org/r/101896/#review102284
Attachment #8823346 - Flags: review?(valentin.gosu) → review+
Pushed by hurley@todesschaf.org:
https://hg.mozilla.org/integration/autoland/rev/53cb3928ff23
add reporting emails for some predictor telemetry. r=valentin
https://hg.mozilla.org/mozilla-central/rev/53cb3928ff23
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: