Change data format for unittest index, result.test field

NEW
Assigned to

Status

P3
normal
8 months ago
3 months ago

People

(Reporter: gabriel-v, Assigned: ekyle)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

8 months ago
We're using this query[1] to create a mapping from chunk to test file. The problem is that the query returns results in the form '$filename $testname', and we just want '$filename'.

Is there a field we could use instead of 'result.test' that only contains the filename, and not the test name?

[1] https://github.com/mozilla/release-services/blob/5a5eb07c0d48e0dbfcec30ab54d11bd5be5be482/src/shipit_code_coverage/shipit_code_coverage/chunk_mapping.py#L50
Or maybe there's a way to schedule the specific sub-test? I wouldn't expect so though.
(Assignee)

Comment 2

8 months ago
The `result.test` is provided by the structured logging, and each suite has a different naming strategy; with reftests being the most confusing. 

We can add the filename as a property where it makes sense.  I can attempt to identify the test names that can be split into filename/testname pairs, and record he filename. A few examples would help ensure I do it correct.

Code that digests the unittest records - https://github.com/klahnakoski/ActiveData-ETL/blob/d106c6ad9e8710ae01c5b2099c492764ae28ed8d/activedata_etl/transforms/unittest_logs_to_sink.py#L143
(Reporter)

Comment 3

8 months ago
(In reply to Kyle Lahnakoski [:ekyle] from comment #2)
> The `result.test` is provided by the structured logging, and each suite has
> a different naming strategy; with reftests being the most confusing. 

Actually, this bug was opened in regard to marionette. We're using unittest to map between chunks and the files that are part of those chunks. Right now, only  information about marionette is being pulled for this.

> We can add the filename as a property where it makes sense.  I can attempt
> to identify the test names that can be split into filename/testname pairs,
> and record he filename. A few examples would help ensure I do it correct.

Here's an example `result.test` entry:
    testing/marionette/harness/marionette_harness/tests/unit/test_addons.py TestAddons.test_install_nonexistent_addon

We would want this the filename relative to topsrcdir to be extracted in another field:
    testing/marionette/harness/marionette_harness/tests/unit/test_addons.py

This 'filename' field should map to the filename relative to topsrcdir, so we can use the data for pushing to try.
See Also: → bug 1480141
Let's figure out bug 1480142 first, maybe there will be a way to run a specific sub-test (although I doubt it).
(Assignee)

Comment 5

4 months ago
We could change the query to parse the test name into components:

> {
> 	"from":"unittest",
> 	"select":[
> 		"run.suite",
> 		"result.test",
> 		{
> 			"name":"file",
> 			"value":{"between":{"result.test":[null," "]}}
> 		},
> 		{
> 			"name":"test",
> 			"value":{"between":{"result.test":[" ",null]}}
> 		}
> 	],
> 	"where":{"eq":{"result.test":"testing/marionette/harness/marionette_harness/tests/unit/test_addons.py TestAddons.test_install_nonexistent_addon"}}
> }


[1] Query - https://activedata.allizom.org/tools/query.html#query_id=bYCJ5IZU
[2] "between" - https://github.com/mozilla/ActiveData/blob/dev/docs/jx_expressions.md#between-operator
Flags: needinfo?(klahnakoski)
No longer blocks: 1429463
(Assignee)

Updated

3 months ago
Flags: needinfo?(klahnakoski)
Priority: -- → P3
You need to log in before you can comment on or make changes to this bug.