Closed Bug 1720230 Opened 3 months ago Closed 2 months ago

Gtest update changed the gtest reports, losing gtest details in all.sh reports.

Categories

(NSS :: Libraries, defect, P1)

3.67

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: rrelyea, Assigned: rrelyea)

References

Details

Attachments

(2 files)

gtests.sh and ssl_gtest.sh parses the report outputted from gtests and reflects that to the all.sh reporting mechanism. Mostly this results in properly counting each gtest as one test, rather than the entire ssl_gtest as one test. This looked like we lost 10's of thousands of tests in NSS to our qa team.

The attached patch updated the sed script which parses the output. It should be short enough to review, but I'm willing to push it to phabricator if the reviewer would prefer.

Assignee: nobody → rrelyea
Status: NEW → ASSIGNED

Any reason not to try to make the ignored result and timestamp attributes to the other regexes? I find the regex handling in sed incomprehensible, but I think that you can use \(timestamp="[^"]*"\ )* to ignore an attribute; the only trick is in ignoring the extra value that produces ((?:...) groupings would be ideal, but we can't get nice things).

I would be totally fine with a more robust regex for these, but like you, I'm pretty much at the limit of my comprehension on exactly on the sed script works. Intuitively it seems like we should be able to come up with something that can handle new additions and changes in order, but I'm not sure how to do it with sed.

Severity: -- → S3
Priority: -- → P1

This patch includes the updated .sed script, and an experiment using bash instead to see how hard it would be to make a more robust parser.

The robust parser generates identical output as sed, but takes about 30x longer, so instead of subsecond operations, it takes almost half a minute. With that result, I think we can stay with sed and continue to update when we get new versions of gtests. (sigh).

time cat report.xml.0 | sed -f parsegtestreport.sed > r1

real 0m0.710s
user 0m0.705s
sys 0m0.008s

time cat report.xml.0 | sh parsegtestreport.sh > r2

real 0m25.066s
user 0m17.759s
sys 0m9.506s
[rrelyea@localhost common]$ diff r1 r2

Status: ASSIGNED → RESOLVED
Closed: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → 3.69
You need to log in before you can comment on or make changes to this bug.