Closed Bug 1564917 Opened 3 months ago Closed 3 months ago

wpt-update --full removes conditions targetting non-nightly builds

Categories

(Testing :: web-platform-tests, defect)

Version 3
defect
Not set

Tracking

(firefox-esr60 unaffected, firefox-esr68 unaffected, firefox68 unaffected, firefox69 wontfix, firefox70 fixed)

RESOLVED FIXED
mozilla70
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox68 --- unaffected
firefox69 --- wontfix
firefox70 --- fixed

People

(Reporter: jgraham, Assigned: jgraham)

References

(Regression)

Details

(Keywords: regression)

Attachments

(2 files)

Even when we do a full try run of all configurations we don't run against beta or release. So conditions designed to target these ought not to be updated. Probably the easiest way to do this is to always retain conditions using properties outside the set that we use for updates.

With the changes in bug 1545143 we were accidentially deleting
conditions like if release_or_beta: FAIL from metadata with mach wpt-update --full because they didn't correspond to anything in the
"full" run. With this change a full update will preserve any
conditions that involve variables we aren't using in the update and
that also don't match any of the runs we have. The reason for the
latter requirement is cases like

expected:
if nightly_build: PASS
FAIL

In this case if we preserve the condition we'll end up with the wrong
behaviour for nightly; by removing it we end up with the wrong
behaviour for release_or_beta, which is a problem but one that can be
solved asynchronously (i.e. it doesn't block sync).

In general this also means that one should prefer to write conditions
that match only non-nightly builds first and have the default fallback
be correct for nightly i.e. the above condition would be better
expressed as:

expected:
if release_or_beta: FAIL

or

expected:
if not nightly_build: FAIL

Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/203e09f37d1f
Don't remove conditions we can't create in wpt-update, r=maja_zf
Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla70
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/8cb1c0d76dda
Use default value when there are no conditions; r=jgraham
Status: REOPENED → RESOLVED
Closed: 3 months ago3 months ago
Resolution: --- → FIXED
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/18229 for changes under testing/web-platform/tests
Upstream PR merged
See Also: → 1569559
You need to log in before you can comment on or make changes to this bug.