Web-Platform-Tests fails verify the manifest on Android

RESOLVED FIXED in Firefox 64

Status

defect
RESOLVED FIXED
9 months ago
9 months ago

People

(Reporter: automatedtester, Assigned: automatedtester)

Tracking

unspecified
mozilla64
Points:
---

Firefox Tracking Flags

(firefox64 fixed)

Details

Assignee

Description

9 months ago
I tried to run wpt against an artifact build of fennec

dburns in ~/development/mozilla-central λ ./mach web-platform-tests
the MozReview service has been disabled; stop loading the reviewboard/mozreview extension from your hgrc files to make this warning go away
 0:17.46 INFO Downloading manifest from https://index.taskcluster.net/v1/task/gecko.v2.mozilla-central.revision.c5a9878baf35a354cb913b4f06542e233685ea9a.source.manifest-upload/artifacts/public/manifests.tar.gz
 0:21.00 INFO Manifest downloaded
 0:21.02 INFO Updating manifests
 0:06.96 INFO Closing logging queue
 0:06.96 INFO queue closed
Error running mach:

    ['web-platform-tests']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

KeyError: u'verify'

  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands.py", line 377, in run_web_platform_tests
    return wpt_runner.run(**params)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands_base.py", line 31, in run
    result = wptrunner.start(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 326, in start
    return not run_tests(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 155, in run_tests
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 78, in get_loader
    skip_timeout=kwargs["skip_timeout"])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 501, in __init__
    self._load_tests()
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 567, in _load_tests
    for test_path, test_type, test in self.iter_tests():
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 552, in iter_tests
    inherit_metadata, test_metadata = self.load_metadata(manifest_file, metadata_path, test_path)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 535, in load_metadata
    metadata_path, test_path, test_manifest.url_base, self.run_info)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/manifestexpected.py", line 301, in get_manifest
    url_base=url_base)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 223, in compile
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 216, in compile_ast
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 35, in compile
    self.visit(tree)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 57, in visit_KeyValueNode
    value = self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 75, in visit_ConditionalNode
    if self.visit(node.children[0]):
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 111, in visit_BinaryExpressionNode
    operand_0 = self.visit(node.children[1])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 91, in visit_VariableNode
    value = self.expr_data[node.data]
Assignee

Comment 1

9 months ago
This appears to happen when you do ./mach web-platform-tests --update-manifest.

dburns in ~/development/mozilla-central λ ./mach web-platform-tests --manifest-update
 0:00.00 INFO Skipping manifest download because existing file is recent
 0:00.00 INFO Updating manifests
 0:05.04 INFO Closing logging queue
 0:05.05 INFO queue closed
Error running mach:

    ['web-platform-tests', '--manifest-update']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

KeyError: u'verify'

  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands.py", line 377, in run_web_platform_tests
    return wpt_runner.run(**params)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands_base.py", line 31, in run
    result = wptrunner.start(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 326, in start
    return not run_tests(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 155, in run_tests
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 78, in get_loader
    skip_timeout=kwargs["skip_timeout"])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 501, in __init__
    self._load_tests()
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 567, in _load_tests
    for test_path, test_type, test in self.iter_tests():
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 552, in iter_tests
    inherit_metadata, test_metadata = self.load_metadata(manifest_file, metadata_path, test_path)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 535, in load_metadata
    metadata_path, test_path, test_manifest.url_base, self.run_info)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/manifestexpected.py", line 301, in get_manifest
    url_base=url_base)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 223, in compile
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 216, in compile_ast
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 35, in compile
    self.visit(tree)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 57, in visit_KeyValueNode
    value = self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 75, in visit_ConditionalNode
    if self.visit(node.children[0]):
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 111, in visit_BinaryExpressionNode
    operand_0 = self.visit(node.children[1])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 91, in visit_VariableNode
    value = self.expr_data[node.data]
That isn't about the manifest not being downloaded; it's about `verify` not being in the RunInfo object sent in to evaluate the wpt ini files. Presumably it's present on desktop but not on mobile. So we perhaps need to default it to false in wptrunner/wpttest.py.
Assignee

Updated

9 months ago
Summary: Web-Platform-Tests fails to get manifest downloaded → Web-Platform-Tests fails verify the manifest on Android

Comment 3

9 months ago
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/443579275cc8
Handle more test info details for web platform tests on android r=jgraham
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/13281 for changes under testing/web-platform/tests
Upstream web-platform-tests status checks passed, PR will merge once commit reaches central.

Comment 6

9 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/443579275cc8
Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla64
Upstream PR merged
Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/13281
* Taskcluster (pull_request) (https://tools.taskcluster.net/task-group-inspector/#/MaTWB-dGSnyI6mgez3dQUg)
Version: Version 3 → unspecified
Assignee: nobody → dburns
You need to log in before you can comment on or make changes to this bug.