1268862 - Use a shorter artifact lifetime for try jobs

Status: RESOLVED FIXED

(Taskcluster :: Services, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

This should be implemented as a taskgraph generation filter, similar to stripping caches, to set try job task and artifact lifetimes to, say, 2 weeks.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

Nick, can you let me know what the current retention times are?

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a42ebb77d4a5

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

Historically we've kept try for 2 weeks, as we were limited by NetApp disk availability. Now that we're on S3 it's softer due to the way the billing is aggregated, or the discussions just happen above my level. FWIW, we currently have a lot more for buildbot builds due to an oversight setting up expiration in S3.

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

re 2 vs 4 weeks - I don't know that we have any data for 'usefulness' of longer retention. Maybe the http logs for archive.m.o or TC artifacts could do that, or look for repushes somehow. My guess is that most pushes are useful for just a day - they either work, or fail and get superceded fast - with a tail where people come back to projects later. With Treeherder retention being longer the logs are still available in that case, just not the binaries. At any rate lets agree a common value.

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

Well, it's an in-tree configuration :)

I'll start with 2 weeks, and if users want to bump it, they can (hopefully with some discussion of potential cost).  Also, individual try pushes can be bumped!

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: MozReview Request: Bug 1268862: expire try jobs in a shorter time-period than default; r=nthomas

Review commit: https://reviewboard.mozilla.org/r/51093/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/51093/

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8749679 [details]
MozReview Request: Bug 1268862: expire try jobs in a shorter time-period than default; r=nthomas

https://reviewboard.mozilla.org/r/51093/#review48405

r+ on the 14 day retention, the rest eyeballs OK but I don't claim any background with it.

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

https://reviewboard.mozilla.org/r/51093/#review48425

::: testing/taskcluster/tasks/decision/try.yml:22
(Diff revision 1)
>    - taskId: '{{#as_slugid}}decision task{{/as_slugid}}'
>      reruns: 3
>      task:
>        created: '{{now}}'
>        deadline: '{{#from_now}}1 day{{/from_now}}'
> +      expires: '{{#from_now}}28 day{{/from_now}}'

Whoops, this should have been 14 days

Comment 10

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8749679 [details]
MozReview Request: Bug 1268862: expire try jobs in a shorter time-period than default; r=nthomas

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/51093/diff/1-2/

Comment 11

[Dustin J. Mitchell [:dustin] (he/him)]

Yay, and it magically carried forward the r+.  Not sure that's according to policy, but it works for me :)

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/79abc7d212099dbb54f6723b8dad94e1421e1cbb
Bug 1268862: expire try jobs in a shorter time-period than default; r=nthomas

Comment 13

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/79abc7d21209