1564917 - wpt-update --full removes conditions targetting non-nightly builds

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect)

Description

[James Graham [:jgraham]]

Even when we do a full try run of all configurations we don't run against beta or release. So conditions designed to target these ought not to be updated. Probably the easiest way to do this is to always retain conditions using properties outside the set that we use for updates.

Comment 1

[James Graham [:jgraham]]

Attachment: Bug 1564917 - Don't remove conditions we can't create in wpt-update,

With the changes in bug 1545143 we were accidentially deleting
conditions like if release_or_beta: FAIL from metadata with mach wpt-update --full because they didn't correspond to anything in the
"full" run. With this change a full update will preserve any
conditions that involve variables we aren't using in the update and
that also don't match any of the runs we have. The reason for the
latter requirement is cases like

expected:
if nightly_build: PASS
FAIL

In this case if we preserve the condition we'll end up with the wrong
behaviour for nightly; by removing it we end up with the wrong
behaviour for release_or_beta, which is a problem but one that can be
solved asynchronously (i.e. it doesn't block sync).

In general this also means that one should prefer to write conditions
that match only non-nightly builds first and have the default fallback
be correct for nightly i.e. the above condition would be better
expressed as:

expected:
if release_or_beta: FAIL

or

expected:
if not nightly_build: FAIL

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/203e09f37d1f
Don't remove conditions we can't create in wpt-update, r=maja_zf

Comment 3

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/203e09f37d1f

Comment 4

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Attachment: Bug 1564917 - Use default value when there are no conditions; r=jgraham

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8cb1c0d76dda
Use default value when there are no conditions; r=jgraham

Comment 6

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/8cb1c0d76dda

Comment 7

[Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/18229 for changes under testing/web-platform/tests

Comment 8

[Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)]

Upstream PR merged