Marionette tests don't emit group_result, group_start, or group_end in structured logging
Categories
(Testing :: Marionette Client and Harness, enhancement, P3)
Tracking
(firefox149 fixed)
| Tracking | Status | |
|---|---|---|
| firefox149 | --- | fixed |
People
(Reporter: florian, Assigned: florian)
Details
(Whiteboard: [webdriver:m19][webdriver:external])
Attachments
(1 external file)
Marionette test jobs are missing group_result entries in their marionette_errorsummary.log files and group_start/group_end messages in their structured log output. This means manifest-level timing and status data isn't available for marionette tests, and resource usage profiling can't show markers for manifests. Additionally, fetch-manifest-data.js (being added in bug 2015029) doesn't even try to process marionette jobs.
There are three issues:
-
Missing
group_resultentries in errorsummary: The marionette runner'sStructuredTestResult(inherited frommoztest) emits test events with IDs in the format"filepath TestClass.test_method"(viatest.id()), but the error summary formatter'stest_to_groupmapping built duringsuite_startonly has plain filepath keys. Since_clean_test_nameonly strips"(finished)"suffixes, the lookup always fails,groupis alwaysNone, andself.groupsis never populated from running test events. Atsuite_end, nogroup_resultlines are emitted. -
Missing
group_start/group_endmessages in structured logging: The marionette runner never callslogger.group_start()/logger.group_end()around manifest execution, unlike mochitest and xpcshell. This prevents resource usage profiling from attributing consumption to specific manifests. -
fetch-manifest-data.jsskips marionette jobs:getArtifactPrefix()doesn't recognize marionette job names, so it returnsnulland marionette jobs are silently skipped during runtime data collection.
| Assignee | ||
Comment 1•9 months ago
|
||
Pass the test group (manifest name) through StructuredTestResult so that
test_start/test_end log events include the group field. Without this, the
errorsummary formatter cannot match test events to their groups because
marionette test IDs include "TestClass.test_method" after the filepath,
which doesn't match the filepath-only keys in test_to_group.
Also emit group_start/group_end around each manifest's tests in
run_test_set, and add "marionette" to the prefix list in
fetch-manifest-data.js so runtime collection includes marionette jobs.
Updated•9 months ago
|
Comment 3•9 months ago
|
||
| bugherder | ||
Updated•9 months ago
|
Description
•