1060439 - Mochitest should only call suite_start once for a run

Status: REOPENED

(Testing :: Mochitest, defect)

Description

[Chris Manchester (limited bugmail, email directly)]

Mochitests sometimes calls buildTestPath multiple times, at least when using the bisect chunk code. Right now this is where we call structured logging's suite_start, but mozlog expects just one suite_start and issues an error if it gets called more than once.

Comment 1

[Vaibhav (:vaibhav1994)]

Reference Log: https://tbpl.mozilla.org/php/getParsedLog.php?id=47027650&tree=Try&full=1

This line: INFO -  26883 ERROR Got second suite_start message before suite_end

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

hey guys, lets pick this up and move this along.  Do we know if this is an issue due to instantiating the browser >1 inside a single test job or the issue is somewhere else?

Comment 3

[Chris Manchester (limited bugmail, email directly)]

From IRC sounds like Vaibhav would like to take a look at this. I think this will be a matter of logging the suite_start once outside of how buildTestPath is called. The idea is to log just once at the very beginning with a list of all test names to be run. In the case of run-by-dir this sounds like doing so before splitting into directories, and in the case of bisect chunk this sounds like doing so once for the initial run before any rerun during bisection analysis.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

assigning to Vaibhav, do let me know if that should not be the case.  There are a handful of bugs like this to get fixed.

Vaibhav, if you feel this is a fairly simple bug, feel free to make this a good first bug and add some links to code and suggestions of what to do!

Comment 5

[Vaibhav (:vaibhav1994)]

https://tbpl.mozilla.org/?tree=Try&rev=ac137232d8bd -- try push with normal mochitest

https://tbpl.mozilla.org/?tree=Try&rev=c6d839b467b2 -- try push with --bisect-chunk turned on for mochitests

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Vaibhav, thanks for working on this bug.

In the bisect chunk bug, i see a bc1 failure and a name error for 'log':
https://tbpl.mozilla.org/php/getParsedLog.php?id=48310873&tree=Try

seems like we could fix that easily :)

Comment 7

[Vaibhav (:vaibhav1994)]

https://tbpl.mozilla.org/?tree=Try&rev=ff4de9cd9012 -- try push with --bisect-chunk turned on.

The "Bisection Summary" is getting printed but the summary(passed, failed, todo) is not getting printed for individual runs: https://tbpl.mozilla.org/php/getParsedLog.php?id=48357405&tree=Try&full=1

Comment 8

[Chris Manchester (limited bugmail, email directly)]

We discussed some strategies for debugging why the summary isn't printed by the bc harness, but the issue with logging suite_start can be addressed independent of that.

Comment 9

[Chris Manchester (limited bugmail, email directly)]

I believe this was fixed with my patch for bug 1075240, please reopen if there are any issues found related to this.

Comment 10

[Vaibhav (:vaibhav1994)]

The following is the new push to verify if it is working ok:

https://treeherder.mozilla.org/ui/#/jobs?repo=try&revision=d948d8c9119b

https://tbpl.mozilla.org/?tree=Try&rev=d948d8c9119b

Comment 11

[Chris Manchester (limited bugmail, email directly)]

I guess the fix doesn't work with runByDir.

Comment 13

[Vaibhav (:vaibhav1994)]

Don't think I would be working on it anytime soon, clearing the assigned flag.