1699350 - Reduce mozilla-central config1 nightly padding once we have a good idea of the worst-case coverage aggregation artifact upload time

Status: NEW

(Webtools :: Searchfox, task)

Description

[Andrew Sutherland [:asuth] (he/him)]

During the process of bug 1677903 I ended up configuring the searchfox lambda cron triggers to give 6 hours for coverage to run and be aggregated and uploaded. Current documentation for that should always be found at https://github.com/mozsearch/mozsearch-mozilla#how-searchfoxorg-stays-up-to-date

We can probably reduce this value once we figure out when the aggregation is being reliably uploaded.

It looks like we can use taskcluster api queue listArtifacts to figure out when the artifact expires and then subtract off the known to be 14 days expiration period. I've overhauled my previous script to give us https://gist.github.com/asutherland/61e44bee962a15a9154bbd76838493c4

Initial data for the runs that have happened since bug 1697857 look good but suggest we currently only have 1h15 - 2h10m of padding and so probably don't need/want to immediately change anything:

• 0d51fdccaa965d303b4a50e6ef245906642bc5f7 artifact project.relman.code-coverage.production.repo.mozilla-central.0d51fdccaa965d303b4a50e6ef245906642bc5f7
  • 2021-03-16T22:40:14.016Z - indexing completed in job EDu-lKz2Q0m95IpLr7LKiQ
  • 2021-03-17T01:49:18Z - artifact uploaded
• 9ad67cd4d216ffec61bbaf8430d5fb9dfb537407 artifact project.relman.code-coverage.production.repo.mozilla-central.9ad67cd4d216ffec61bbaf8430d5fb9dfb537407
  • 2021-03-17T10:53:26.785Z - indexing completed in job UWO8bv4QSOygnAatYw_q2Q
  • 2021-03-17T14:46:03Z - artifact uploaded
• 4d4bc56f77a199234c625dfdd85d06c62caae273 artifact project.relman.code-coverage.production.repo.mozilla-central.4d4bc56f77a199234c625dfdd85d06c62caae273
  • 2021-03-17T22:44:15.383Z - indexing completed in job K-5dloFCR1uoG_MCy5r2WA
  • 2021-03-18T01:35:38Z - artifact uploaded

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

Attachment: script run 2021-04-03 covering 14 days

It looks like our current buffer values are probably still ideal for the time being. In this run there were cases where we only had spare (relative to the searchfox AWS cron jobs) of ~10min 2x, plus a 30min. The great news is that in every case the coverage data was uploaded before the searchfox AWS cron jobs started, so we should have had coverage in every run for the past 14 days! Woo!

I'm going to leave this bug open for now, but it's likely appropriate in the medium term to commit the tc-coverage-timestamps script to the mozsearch repo and add a little more documentation that references the script and how to use it to evaluate the buffer and then mark this bug as fixed.