1494636 - Web-Platform-Tests fails verify the manifest on Android

Assignee

Description

•

6 years ago

I tried to run wpt against an artifact build of fennec

dburns in ~/development/mozilla-central λ ./mach web-platform-tests
the MozReview service has been disabled; stop loading the reviewboard/mozreview extension from your hgrc files to make this warning go away
 0:17.46 INFO Downloading manifest from https://index.taskcluster.net/v1/task/gecko.v2.mozilla-central.revision.c5a9878baf35a354cb913b4f06542e233685ea9a.source.manifest-upload/artifacts/public/manifests.tar.gz
 0:21.00 INFO Manifest downloaded
 0:21.02 INFO Updating manifests
 0:06.96 INFO Closing logging queue
 0:06.96 INFO queue closed
Error running mach:

    ['web-platform-tests']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

KeyError: u'verify'

  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands.py", line 377, in run_web_platform_tests
    return wpt_runner.run(**params)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands_base.py", line 31, in run
    result = wptrunner.start(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 326, in start
    return not run_tests(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 155, in run_tests
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 78, in get_loader
    skip_timeout=kwargs["skip_timeout"])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 501, in __init__
    self._load_tests()
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 567, in _load_tests
    for test_path, test_type, test in self.iter_tests():
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 552, in iter_tests
    inherit_metadata, test_metadata = self.load_metadata(manifest_file, metadata_path, test_path)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 535, in load_metadata
    metadata_path, test_path, test_manifest.url_base, self.run_info)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/manifestexpected.py", line 301, in get_manifest
    url_base=url_base)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 223, in compile
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 216, in compile_ast
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 35, in compile
    self.visit(tree)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 57, in visit_KeyValueNode
    value = self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 75, in visit_ConditionalNode
    if self.visit(node.children[0]):
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 111, in visit_BinaryExpressionNode
    operand_0 = self.visit(node.children[1])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 91, in visit_VariableNode
    value = self.expr_data[node.data]

David Burns :automatedtester

Assignee

Comment 1

•

6 years ago

This appears to happen when you do ./mach web-platform-tests --update-manifest.

dburns in ~/development/mozilla-central λ ./mach web-platform-tests --manifest-update
 0:00.00 INFO Skipping manifest download because existing file is recent
 0:00.00 INFO Updating manifests
 0:05.04 INFO Closing logging queue
 0:05.05 INFO queue closed
Error running mach:

    ['web-platform-tests', '--manifest-update']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

KeyError: u'verify'

  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands.py", line 377, in run_web_platform_tests
    return wpt_runner.run(**params)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/mach_commands_base.py", line 31, in run
    result = wptrunner.start(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 326, in start
    return not run_tests(**kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 155, in run_tests
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptrunner.py", line 78, in get_loader
    skip_timeout=kwargs["skip_timeout"])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 501, in __init__
    self._load_tests()
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 567, in _load_tests
    for test_path, test_type, test in self.iter_tests():
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 552, in iter_tests
    inherit_metadata, test_metadata = self.load_metadata(manifest_file, metadata_path, test_path)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/testloader.py", line 535, in load_metadata
    metadata_path, test_path, test_manifest.url_base, self.run_info)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/manifestexpected.py", line 301, in get_manifest
    url_base=url_base)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 223, in compile
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 216, in compile_ast
    **kwargs)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 35, in compile
    self.visit(tree)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 47, in visit_DataNode
    self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 57, in visit_KeyValueNode
    value = self.visit(child)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 75, in visit_ConditionalNode
    if self.visit(node.children[0]):
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 111, in visit_BinaryExpressionNode
    operand_0 = self.visit(node.children[1])
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/node.py", line 7, in visit
    return func(node)
  File "/Users/dburns/development/mozilla-central/testing/web-platform/tests/tools/wptrunner/wptrunner/wptmanifest/backends/static.py", line 91, in visit_VariableNode
    value = self.expr_data[node.data]

James Graham [:jgraham]

Comment 2

•

6 years ago

That isn't about the manifest not being downloaded; it's about `verify` not being in the RunInfo object sent in to evaluate the wpt ini files. Presumably it's present on desktop but not on mobile. So we perhaps need to default it to false in wptrunner/wpttest.py.

David Burns :automatedtester

Assignee

Updated

•

6 years ago

Summary: Web-Platform-Tests fails to get manifest downloaded → Web-Platform-Tests fails verify the manifest on Android

Pulsebot

Comment 3

•

6 years ago

Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/autoland/rev/443579275cc8
Handle more test info details for web platform tests on android r=jgraham

Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)

Comment 4

•

6 years ago

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/13281 for changes under testing/web-platform/tests

Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)

Comment 5

•

6 years ago

Upstream web-platform-tests status checks passed, PR will merge once commit reaches central.

Noemi Erli[:noemi_erli]

Comment 6

•

6 years ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/443579275cc8

Status: NEW → RESOLVED

Closed: 6 years ago

status-firefox64: --- → fixed

Resolution: --- → FIXED

Target Milestone: --- → mozilla64

Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)

Comment 7

•

6 years ago

Upstream PR merged

Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)

Comment 8

•

6 years ago

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/13281
* Taskcluster (pull_request) (https://tools.taskcluster.net/task-group-inspector/#/MaTWB-dGSnyI6mgez3dQUg)

Sylvestre Ledru [:Sylvestre]

Updated

•

6 years ago

Version: Version 3 → unspecified

Sylvestre Ledru [:Sylvestre]

Updated

•

6 years ago

Assignee: nobody → dburns

Bugzilla

Quick Search

Web-Platform-Tests fails verify the manifest on Android

Categories

(Testing :: web-platform-tests, defect)

Tracking

(firefox64 fixed)

People

(Reporter: automatedtester, Assigned: automatedtester)

References

Details

Crash Data

Security

(public)

User Story

Description

Comment 1

Comment 2

Updated

Comment 3

Comment 4

Comment 5

Comment 6

Comment 7

Comment 8

Updated

Updated