1280641 - Implement a two-tier S3 cache for sccache

Status: NEW

(Firefox Build System :: General, defect, P3)

Description

[(not currently active) Ted Mielczarek]

Currently sccache on try uses a different S3 bucket from inbound etc, because we don't want try builds to be able to poison the inbound cache with junk. Unfortunately this means that try builds don't benefit from cached objects that have already been built on other branches.

Once we fix bug 1280640 such that cross-branch cache hits are possible, we can add support to sccache for a two-tier S3 cache and configure try builds to use the inbound bucket as a read-only cache, and then still write to a separate bucket. This should make some try builds faster because they'll be able to pull objects from cache where they would have had to build.

Comment 1

[Armen [:armenzg]]

ted might be able to tackle this by the end of this month.
More info in dependent bug.

Comment 2

[Armen [:armenzg]]

I also want to point out that it is unclear how much gain we will get from this work.
However, since it is not a lot of work to get it done as we will most likely get *some* gains even if not large.

Comment 3

[Armen [:armenzg]]

P1 == we're commiting to complete this within this quarter.

Comment 4

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

This is an assigned P1 bug without activity in two weeks. 

If you intend to continue working on this bug for the current release/iteration/sprint, remove the 'stale-bug' keyword.

Otherwise we'll reset the priority of the bug back to '--' on Monday, August 28th.

Comment 5

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

Is this still being worked on?

Comment 6

[Armen [:armenzg]]

ted: Is this still being worked on?

Comment 7

[(not currently active) Ted Mielczarek]

Not at present, but it's still something we'd like to have.