1505086 - Improve build telemetry submission logic

Status: RESOLVED FIXED

(Firefox Build System :: General, enhancement)

Description

[(not currently active) Ted Mielczarek]

Currently our logic for submitting build telemetry is "roughly every 10 mach invocations":
https://dxr.mozilla.org/mozilla-central/rev/c291143e24019097d087f9307e59b49facaf90cb/build/mach_bootstrap.py#245-247

This was chosen arbitrarily, I think we should revisit it. I currently have 22 samples sitting in my outgoing directory:
$ ls -1 ~/.mozbuild/telemetry/outgoing/ | wc -l
22

I'd propose something like:

If any of the following are true, submit telemetry:
* There are >N samples in the outgoing directory (choose an arbitrary N here, perhaps 10)
* The last submission date was > 1 week ago (we could check the mtime on ~/.mozbuild/telemetry/telemetry.log to measure this)
* The current 10% random check

Comment 1

[Connor Sheehan [:sheehan]]

I'm going to hack on this.

Comment 2

[Connor Sheehan [:sheehan]]

Attachment: Bug 1505086: move `verify_statedir` from telemetry submission script to `mozbuild.telemetry` r?ted

A subsequent commit will require similar logic to the
checks found in this command. Moving to mozbuild will
allow us to re-use the function instead of re-writing
similar logic.

Comment 3

[Connor Sheehan [:sheehan]]

Attachment: Bug 1505086: use `mach python` to submit build telemetry r?ted

While attempting to improve the build telemetry submission
logic, I found a bug in the way telemetry submission
works. Essentially the submission script was failing to
import any of the required packages (specifically
`mozbuild.telemetry` in this case) as the method used to
modify path was incorrect and the script was running outside
of the virtualenv. The invocation is also sending stdout
and stderr to `/dev/null`, making this problem even less obvious.
When I fixed the path modifications, I realized that `mozbuild`
imports will require a long chain of other imports
(and transitively, more `sys.path` modifications)
such as `which`, `mach`, `mozautomation`, etc to complete.

When I tested the submission script, I did so by running
`mach python build/submit_telemetry_data.py`, which runs the
script in a virtualenv with all required packages installed.
That's likely part of the reasons I overlooked this issue in testing.
Rather than go through the process of importing every dependency
of `mozbuild`, this commit changes the invocation of the submission
script to go through `mach python`. Things seem to work as
expected with this change.

Comment 4

[Connor Sheehan [:sheehan]]

Attachment: Bug 1505086: remove build telemetry submission logic r?ted

Currently, build telemetry submits at random, approximately
every 10 `mach` invocations. This choice was made arbitrarily,
with no real reason in mind for that level of frequency.
This commit adds a few new conditions which will trigger
submission of telemetry pings:

    - If there are more than 10 pings in the `outgoing` dir
    - If the last submission date was over 1 week ago
    - The old logic of approximately every 10 `mach` runs

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/12c35ad7e844
move `verify_statedir` from telemetry submission script to `mozbuild.telemetry` r=ted
https://hg.mozilla.org/integration/autoland/rev/9a2f0d443bb1
use `mach python` to submit build telemetry r=ted
https://hg.mozilla.org/integration/autoland/rev/96046dff0022
remove build telemetry submission logic r=ted

Comment 6

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/12c35ad7e844
https://hg.mozilla.org/mozilla-central/rev/9a2f0d443bb1
https://hg.mozilla.org/mozilla-central/rev/96046dff0022