1316948 - make it possible to evaluateRules at a certain point in history

Status: RESOLVED MOVED

(Release Engineering Graveyard :: Applications: Balrog (backend), defect, P3)

Description

[[e-mail] (:bhearsum)]

One of the ideas behind storing full history of changes to Balrog's database was to make it possible for us to turn back the clock and see what a given update request was served at a certain point in History. Technically, this is possible for us to do by inspecting the History of Rules and Releases, but in practice it is very difficult - unless you know the full state of the Rules table at the time, you can't be sure which Rule an update request matched.

I think this could be made a lot easier if we had a helper at the database layer that worked liked evaluateRules, but used an older version of the entire table instead. There might be some clever SQL features we can make use of to create a temporary table based on rules_history, but in the worst case scenario we can do it all in Python. Once you have a rewound version of the Rules, evaluating them against the update request should be easy. If a Mapping is found, we'd have to rewind it as well.

A few caveats:
* This really won't work at all for nightlies, because we barely store any history for them.
* This won't _always_ return accurate results, because our matching logic changes over time. Eg: we didn't always have csv matching support in the locale field.

There may be other caveats as well, but this still may be useful for many cases.

Comment 1

[[e-mail] (:bhearsum)]

It would be great to be able to query the future too, by looking at scheduled changes. This might be better off as a separate bug.

Comment 2

[[e-mail] (:bhearsum)]

A UI for rewinding the Rules table would be useful, too. We've been trying to figure out why we did something for bug 1415557, and because there's so many Rules involved it's not humanly possible to understand the state of things at the time.

Comment 3

[[e-mail] (:bhearsum)]

The UI part of this has been explored in https://github.com/mozilla-frontend-infra/balrog-ui/pull/135, which was the main driving force. If use cases beyond that come up, let's file them as separate Github Issues.