Closed Bug 1494636 Opened 7 years ago Closed 7 years ago

Web-Platform-Tests fails verify the manifest on Android

Categories

(Testing :: web-platform-tests, defect)

Product:
Testing
Component:
web-platform-tests
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox64 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla64
Tracking Flags:
Tracking Status
firefox64 --- fixed

People

(Reporter: automatedtester, Assigned: automatedtester)

Details

I tried to run wpt against an artifact build of fennec dburns in ~/development/mozilla-central λ ./mach web-platform-tests the MozReview service has been disabled; stop loading the reviewboard/mozreview extension from your hgrc files to make this warning go away 0:17.46 INFO Downloading manifest from https://index.taskcluster.net/v1/task/gecko.v2.mozilla-central.revision.c5a9878baf35a354cb913b4f06542e233685ea9a.source.manifest-upload/artifacts/public/manifests.tar.gz 0:21.00 INFO Manifest downloaded 0:21.02 INFO Updating manifests 0:06.96 INFO Closing logging queue 0:06.96 INFO queue closed Error running mach: ['web-platform-tests'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: KeyError: u'verify' File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands.py", line 377, in run_web_platform_tests return wpt_runner.run(**params) File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands_base.py", line 31, in run result = wptrunner.start(**kwargs) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 326, in start return not run_tests(**kwargs) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 155, in run_tests **kwargs) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 78, in get_loader skip_timeout=kwargs["skip_timeout"]) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 501, in __init__ self._load_tests() File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 567, in _load_tests for test_path, test_type, test in self.iter_tests(): File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 552, in iter_tests inherit_metadata, test_metadata = self.load_metadata(manifest_file, metadata_path, test_path) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 535, in load_metadata metadata_path, test_path, test_manifest.url_base, self.run_info) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/manifestexpected.py", line 301, in get_manifest url_base=url_base) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 223, in compile **kwargs) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 216, in compile_ast **kwargs) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 35, in compile self.visit(tree) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode self.visit(child) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode self.visit(child) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 57, in visit_KeyValueNode value = self.visit(child) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 75, in visit_ConditionalNode if self.visit(node.children[0]): File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 111, in visit_BinaryExpressionNode operand_0 = self.visit(node.children[1]) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 91, in visit_VariableNode value = self.expr_data[node.data]
This appears to happen when you do ./mach web-platform-tests --update-manifest. dburns in ~/development/mozilla-central λ ./mach web-platform-tests --manifest-update 0:00.00 INFO Skipping manifest download because existing file is recent 0:00.00 INFO Updating manifests 0:05.04 INFO Closing logging queue 0:05.05 INFO queue closed Error running mach: ['web-platform-tests', '--manifest-update'] The error occurred in code that was called by the mach command. This is either a bug in the called code itself or in the way that mach is calling it. You should consider filing a bug for this issue. If filing a bug, please include the full output of mach, including this error message. The details of the failure are as follows: KeyError: u'verify' File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands.py", line 377, in run_web_platform_tests return wpt_runner.run(**params) File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands_base.py", line 31, in run result = wptrunner.start(**kwargs) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 326, in start return not run_tests(**kwargs) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 155, in run_tests **kwargs) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 78, in get_loader skip_timeout=kwargs["skip_timeout"]) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 501, in __init__ self._load_tests() File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 567, in _load_tests for test_path, test_type, test in self.iter_tests(): File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 552, in iter_tests inherit_metadata, test_metadata = self.load_metadata(manifest_file, metadata_path, test_path) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 535, in load_metadata metadata_path, test_path, test_manifest.url_base, self.run_info) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/manifestexpected.py", line 301, in get_manifest url_base=url_base) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 223, in compile **kwargs) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 216, in compile_ast **kwargs) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 35, in compile self.visit(tree) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode self.visit(child) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode self.visit(child) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 57, in visit_KeyValueNode value = self.visit(child) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 75, in visit_ConditionalNode if self.visit(node.children[0]): File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 111, in visit_BinaryExpressionNode operand_0 = self.visit(node.children[1]) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit return func(node) File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 91, in visit_VariableNode value = self.expr_data[node.data]
That isn't about the manifest not being downloaded; it's about `verify` not being in the RunInfo object sent in to evaluate the wpt ini files. Presumably it's present on desktop but not on mobile. So we perhaps need to default it to false in wptrunner/wpttest.py.
Summary: Web-Platform-Tests fails to get manifest downloaded → Web-Platform-Tests fails verify the manifest on Android
Pushed by james@hoppipolla.co.uk: https://hg.mozilla.org/integration/autoland/rev/443579275cc8 Handle more test info details for web platform tests on android r=jgraham
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/13281 for changes under testing/web-platform/tests
Upstream web-platform-tests status checks passed, PR will merge once commit reaches central.
https://hg.mozilla.org/mozilla-central/rev/443579275cc8
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox64: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla64
Version: Version 3 → unspecified
Assignee: nobody → dburns
You need to log in before you can comment on or make changes to this bug.