Improve build telemetry submission logic

RESOLVED FIXED in Firefox 65

Status

enhancement
RESOLVED FIXED
9 months ago
8 months ago

People

(Reporter: ted, Assigned: sheehan)

Tracking

(Blocks 2 bugs)

Trunk
mozilla65
Dependency tree / graph

Firefox Tracking Flags

(firefox65 fixed)

Details

Attachments

(3 attachments)

Currently our logic for submitting build telemetry is "roughly every 10 mach invocations":
https://dxr.mozilla.org/mozilla-central/rev/c291143e24019097d087f9307e59b49facaf90cb/build/mach_bootstrap.py#245-247

This was chosen arbitrarily, I think we should revisit it. I currently have 22 samples sitting in my outgoing directory:
$ ls -1 ~/.mozbuild/telemetry/outgoing/ | wc -l
22

I'd propose something like:

If any of the following are true, submit telemetry:
* There are >N samples in the outgoing directory (choose an arbitrary N here, perhaps 10)
* The last submission date was > 1 week ago (we could check the mtime on ~/.mozbuild/telemetry/telemetry.log to measure this)
* The current 10% random check
I'm going to hack on this.
Assignee: nobody → sheehan
A subsequent commit will require similar logic to the
checks found in this command. Moving to mozbuild will
allow us to re-use the function instead of re-writing
similar logic.
While attempting to improve the build telemetry submission
logic, I found a bug in the way telemetry submission
works. Essentially the submission script was failing to
import any of the required packages (specifically
`mozbuild.telemetry` in this case) as the method used to
modify path was incorrect and the script was running outside
of the virtualenv. The invocation is also sending stdout
and stderr to `/dev/null`, making this problem even less obvious.
When I fixed the path modifications, I realized that `mozbuild`
imports will require a long chain of other imports
(and transitively, more `sys.path` modifications)
such as `which`, `mach`, `mozautomation`, etc to complete.

When I tested the submission script, I did so by running
`mach python build/submit_telemetry_data.py`, which runs the
script in a virtualenv with all required packages installed.
That's likely part of the reasons I overlooked this issue in testing.
Rather than go through the process of importing every dependency
of `mozbuild`, this commit changes the invocation of the submission
script to go through `mach python`. Things seem to work as
expected with this change.
Currently, build telemetry submits at random, approximately
every 10 `mach` invocations. This choice was made arbitrarily,
with no real reason in mind for that level of frequency.
This commit adds a few new conditions which will trigger
submission of telemetry pings:

    - If there are more than 10 pings in the `outgoing` dir
    - If the last submission date was over 1 week ago
    - The old logic of approximately every 10 `mach` runs
Attachment #9023504 - Attachment description: Bug 1505086: add new conditions under which to submit build telemetry r?ted → Bug 1505086: remove build telemetry submission logic r?ted
Pushed by cosheehan@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/12c35ad7e844
move `verify_statedir` from telemetry submission script to `mozbuild.telemetry` r=ted
https://hg.mozilla.org/integration/autoland/rev/9a2f0d443bb1
use `mach python` to submit build telemetry r=ted
https://hg.mozilla.org/integration/autoland/rev/96046dff0022
remove build telemetry submission logic r=ted
Blocks: 1506619
https://hg.mozilla.org/mozilla-central/rev/12c35ad7e844
https://hg.mozilla.org/mozilla-central/rev/9a2f0d443bb1
https://hg.mozilla.org/mozilla-central/rev/96046dff0022
Status: NEW → RESOLVED
Closed: 8 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla65
You need to log in before you can comment on or make changes to this bug.