Closed Bug 1494636 Opened 2 years ago Closed 2 years ago

Web-Platform-Tests fails verify the manifest on Android

Categories

(Testing :: web-platform-tests, defect)

defect
Not set
normal

Tracking

(firefox64 fixed)

RESOLVED FIXED
mozilla64
Tracking Status
firefox64 --- fixed

People

(Reporter: automatedtester, Assigned: automatedtester)

Details

I tried to run wpt against an artifact build of fennec

dburns in ~/development/mozilla-central λ ./mach web-platform-tests
the MozReview service has been disabled; stop loading the reviewboard/mozreview extension from your hgrc files to make this warning go away
 0:17.46 INFO Downloading manifest from https://index.taskcluster.net/v1/task/gecko.v2.mozilla-central.revision.c5a9878baf35a354cb913b4f06542e233685ea9a.source.manifest-upload/artifacts/public/manifests.tar.gz
 0:21.00 INFO Manifest downloaded
 0:21.02 INFO Updating manifests
 0:06.96 INFO Closing logging queue
 0:06.96 INFO queue closed
Error running mach:

    ['web-platform-tests']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

KeyError: u'verify'

  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands.py", line 377, in run_web_platform_tests
    return wpt_runner.run(**params)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands_base.py", line 31, in run
    result = wptrunner.start(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 326, in start
    return not run_tests(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 155, in run_tests
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 78, in get_loader
    skip_timeout=kwargs["skip_timeout"])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 501, in __init__
    self._load_tests()
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 567, in _load_tests
    for test_path, test_type, test in self.iter_tests():
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 552, in iter_tests
    inherit_metadata, test_metadata = self.load_metadata(manifest_file, metadata_path, test_path)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 535, in load_metadata
    metadata_path, test_path, test_manifest.url_base, self.run_info)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/manifestexpected.py", line 301, in get_manifest
    url_base=url_base)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 223, in compile
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 216, in compile_ast
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 35, in compile
    self.visit(tree)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 57, in visit_KeyValueNode
    value = self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 75, in visit_ConditionalNode
    if self.visit(node.children[0]):
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 111, in visit_BinaryExpressionNode
    operand_0 = self.visit(node.children[1])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 91, in visit_VariableNode
    value = self.expr_data[node.data]
This appears to happen when you do ./mach web-platform-tests --update-manifest.

dburns in ~/development/mozilla-central λ ./mach web-platform-tests --manifest-update
 0:00.00 INFO Skipping manifest download because existing file is recent
 0:00.00 INFO Updating manifests
 0:05.04 INFO Closing logging queue
 0:05.05 INFO queue closed
Error running mach:

    ['web-platform-tests', '--manifest-update']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

KeyError: u'verify'

  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands.py", line 377, in run_web_platform_tests
    return wpt_runner.run(**params)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands_base.py", line 31, in run
    result = wptrunner.start(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 326, in start
    return not run_tests(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 155, in run_tests
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 78, in get_loader
    skip_timeout=kwargs["skip_timeout"])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 501, in __init__
    self._load_tests()
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 567, in _load_tests
    for test_path, test_type, test in self.iter_tests():
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 552, in iter_tests
    inherit_metadata, test_metadata = self.load_metadata(manifest_file, metadata_path, test_path)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 535, in load_metadata
    metadata_path, test_path, test_manifest.url_base, self.run_info)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/manifestexpected.py", line 301, in get_manifest
    url_base=url_base)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 223, in compile
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 216, in compile_ast
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 35, in compile
    self.visit(tree)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 57, in visit_KeyValueNode
    value = self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 75, in visit_ConditionalNode
    if self.visit(node.children[0]):
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 111, in visit_BinaryExpressionNode
    operand_0 = self.visit(node.children[1])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 91, in visit_VariableNode
    value = self.expr_data[node.data]
That isn't about the manifest not being downloaded; it's about `verify` not being in the RunInfo object sent in to evaluate the wpt ini files. Presumably it's present on desktop but not on mobile. So we perhaps need to default it to false in wptrunner/wpttest.py.
Summary: Web-Platform-Tests fails to get manifest downloaded → Web-Platform-Tests fails verify the manifest on Android
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/443579275cc8
Handle more test info details for web platform tests on android r=jgraham
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/13281 for changes under testing/web-platform/tests
Upstream web-platform-tests status checks passed, PR will merge once commit reaches central.
https://hg.mozilla.org/mozilla-central/rev/443579275cc8
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla64
Version: Version 3 → unspecified
Assignee: nobody → dburns
You need to log in before you can comment on or make changes to this bug.