Update web-platform-tests to 4836688f614cff16a03cbd922f3cc3394b5e5013

RESOLVED FIXED in Firefox 63

Status

enhancement
RESOLVED FIXED
Last year
Last year

People

(Reporter: wptsync, Unassigned)

Tracking

unspecified
mozilla63
Points:
---

Firefox Tracking Flags

(firefox63 fixed)

Details

(Whiteboard: [wptsync landing error])

No description provided.
Whiteboard: [wptsync landing] → [wptsync landing error]
Retriggered failing web-platform-test tasks on try before final metadata update.
Pushing update to remote failed:
Cmd('git') failed due to: exit code(1)
  cmdline: git push --porcelain mozilla sync/landing/open/1477472/0:inbound
  stderr: 'error: refs/heads/sync/downstream/open/12158/0 does not point to a valid object!
error: refs/syncs/sync/downstream/open/12158/0 does not point to a valid object!
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 63 changesets with 628 changes to 548 files (+1 heads)
remote:
remote:
remote: ************************** ERROR ****************************
remote: Multiple heads detected on branch 'default'
remote: Only one head per branch is allowed!
remote: *************************************************************
remote:
remote:
remote: transaction abort!
remote: rollback completed
Traceback (most recent call last):
  File "/app/git-cinnabar/git-remote-hg", line 46, in <module>
    run(main)
  File "/app/git-cinnabar/cinnabar/util.py", line 601, in run
    retcode = func(sys.argv[1:])
  File "/app/git-cinnabar/git-remote-hg", line 40, in main
    helper.run()
  File "/app/git-cinnabar/cinnabar/remote_helper.py", line 88, in run
    func(*args)
  File "/app/git-cinnabar/cinnabar/remote_helper.py", line 396, in push
    self._branchmap.names(), self._dry_run)
  File "/app/git-cinnabar/cinnabar/hg/repo.py", line 723, in push
    hint=part.params.get('hint'))
mercurial.error.Abort: pretxnchangegroup.b_singlehead hook failed
error: failed to push some refs to 'hg::ssh://hg.mozilla.org/integration/mozilla-inbound''
Pushed by wptsync@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/1cca14cefaef
[wpt-sync] Update web-platform-tests to 4836688f614cff16a03cbd922f3cc3394b5e5013, a=testonly
There were also failures on script-src-strict_dynamic_double_policy_honor_whitelist.sub.html

Log link: https://treeherder.mozilla.org/logviewer.html#?job_id=189934264&repo=mozilla-inbound&lineNumber=3127
Pushed by archaeopteryx@coole-files.de:
https://hg.mozilla.org/integration/mozilla-inbound/rev/dd958f1110d6
[wpt-sync] Update web-platform-tests to 4836688f614cff16a03cbd922f3cc3394b5e5013, a=testonly
https://hg.mozilla.org/integration/mozilla-inbound/rev/dde885a62577
Remove result expectations for content-security-policy/img-src/img-src-self-unique-origin.html. r=wpt-fix on a CLOSED TREE
Pushed by archaeopteryx@coole-files.de:
https://hg.mozilla.org/integration/mozilla-inbound/rev/d8f622781168
Update expectations of Windows Wr svg tests. r=Wr-fix on a CLOSED TREE
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/7fe4cb4c4eb7
Fixup some bogus wpt metadata misset from intermittents, a=testonly
Flags: needinfo?(james)
Can we have it so that the bot doesn't touch the expectations for tests that are not actually new or modified? Because for intermittent failures the bot will just keep flip-flopping the expectation values between the various intermittently produced states on every update. Specifically from this push, bug 1464302, bug 1477606, bug 1473544, and bug 1474649 were all affected for no good reason. I expect the intermittent failure rates on those bugs to go up significantly now because the bot changed the expectation to what I believe is the lower-frequency state, so the "failure" will happen more often.

Also in general changing the expectation for existing annotations can hide real failures, and reduces the value of having this suite in the first place.
Flags: needinfo?(james)
It;s generally hard to figure out if a test was modified. We don't have enough information to figure out if a particular support file is used by a particular test, so we can't reliably work out which tests were actually modified by a particular push. In addition harness changes may also affect test results, and we have no way to determine which tests are expected to be affected. I know this situation isn't ideal but I can't think of a realistic approach to improve it in the short term.

We already go to some lengths to determine which results are intermittents (multiple try pushes + retriggering failing jobs after the metadata is updated to see if we flipped things the wrong way). In this case we accidentially skipped that second try push, so the result was worse than usual. But in general having many intermittent tests enabled is a problem, not just for the metadata update. We should try to fix the underlying problems where possible.
Flags: needinfo?(james)
You need to log in before you can comment on or make changes to this bug.