1548810 - Managing obsolete indexes

Status: RESOLVED WONTFIX

(Taskcluster :: General, enhancement)

Description

[Mitchell Hentges [:mhentges] 🦀]

Use case:
Fenix used the same index for a while to point to the latest raptor-ready app: index.project.mobile.fenix.branch.master.latest.greenfield/aarch64-release-raptor.

As the releng team, we realized that some of this information is redundant, so we refactor the index (and add a v2 prefix to differentiate the new structure): index.project.mobile.fenix.v2.fenix.master.latest.raptor/aarch64

Between these two stages, downstream consumers start using our original index - some tell releng informally, others just silently depend on the index.
We in releng notify those that we know/remember are using the index before we make the change.

However, we still have the following issues:

• We aren't officially keeping track of who is using each index, so some consumers aren't notified of the transition
• Consumers that are using an old index have no way to know that an index isn't used anymore, except they won't be overridden/created anymore - which, to consumers, is ambiguous between "builds have been failing, therefore not creating indexes" or "the index has changed"
• Consumers who realize that they're using an old index don't know where the new one is: they need to comb through commits/PRs to find out who changed the index, then need to manually find the new index (or talk to the team that caused the change).

---

TL;DR:
When indexes are obsolete, I think we need a way to:

• Notify index-consumers that they're out-of-date (warn on-GET? Fail the index-GET-request?)
• Inform index-consumers of the new index that they probably want
• Ensure that obsolete indexes are properly marked as "obsolete"

I don't know the best way of solving this. Native versioning that isn't just appending a string? Manual "mark-as-obsolete"? 🤔

Comment 1

[Nick Alexander :nalexander [he/him]]

One thing that might make consumers both notice and curse less loudly is if you 303 (See Other) but accept with a specific HTTP header (&allow_obselete=1) or similar. That will cause clients to require updates (so consumers notice) but not make them fix the issue right that minute.

Comment 2

[Dustin J. Mitchell [:dustin] (he/him)]

I like Nick's idea, but allowing arbitrary 303's (even if adding them requires scopes) needs some thought from a security perspective -- we don't want to become an open redirector.

On the other hand, this feels like a kind of normal "things change, it's software, get used to it" situation, so it doesn't feel like a situation we must address.

Mitch, if you're interested in pursuing this, maybe put together a proposal and develop it into an RFC? Otherwise, let's close this up.

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

I won't be able to pursue this, so I'll close it.