If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

B2G emulator unittests should dump logcat to stdout on failure

RESOLVED FIXED in Firefox 18

Status

Testing
General
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: ahal, Assigned: ahal)

Tracking

unspecified
mozilla20
Points:
---

Firefox Tracking Flags

(firefox18 fixed, firefox19 fixed, firefox20 fixed)

Details

Attachments

(4 attachments)

(Assignee)

Description

5 years ago
There have been several issues that could only be reproduced on the test slaves, not locally (e.g bug 809437). It would have been helpful for us to have the logcat dumped to the logs to help with debugging these kinds of errors.

The Mn tests already do this, but the others (mochitest/reftest) don't.
(Assignee)

Updated

5 years ago
Assignee: nobody → ahalberstadt
Status: NEW → ASSIGNED
(Assignee)

Comment 1

5 years ago
This will require a patch to the b2g automation and the mozharness script.
(Assignee)

Comment 2

5 years ago
Created attachment 683190 [details] [diff] [review]
Patch 1.0 (gecko) - Add logcat_dir option to b2g unittests

I also updated how we start the logcat process (we no longer need to call waitForFinish so there's no reason to start it in a new thread).

We'll need to land this patch before the mozharness change on all branches that we have b2g running on (which means m-c, aurora and probably beta since today is merge day).
Attachment #683190 - Flags: review?(jgriffin)
(Assignee)

Comment 3

5 years ago
Some other notes:
The above patch worked locally, but I pushed to try anyway (due to mozprocess dependency changes): https://tbpl.mozilla.org/?tree=Try&noignore=1&rev=72b2e3c96783

I forgot to remove import multiprocessing.. won't bother with a new patch, but I'll do it before landing.
(Assignee)

Comment 4

5 years ago
Created attachment 683235 [details] [diff] [review]
Patch 1.0 (mozharness) - Bits to enable and dump logcat

Aki, I'm wondering if the line:
> (parser.passed == 0 and parser.failed == 0) or parser.failed > 0

should catch all of the cases where we might want to dump the logcat. Would this include infra failures?
Attachment #683235 - Flags: review?(aki)

Comment 5

5 years ago
(In reply to Andrew Halberstadt [:ahal] from comment #4)
> Created attachment 683235 [details] [diff] [review]
> Patch 1.0 (mozharness) - Bits to enable and dump logcat
> 
> Aki, I'm wondering if the line:
> > (parser.passed == 0 and parser.failed == 0) or parser.failed > 0
> 
> should catch all of the cases where we might want to dump the logcat. Would
> this include infra failures?

I'm not entirely sure, but it looks like it's parser.fail_count and parser.pass_count ?

I think -1 for either means we haven't reached the pass/fail/known output.
If you're looking for a logcat for any failure condition, parser.fail_count != 0 might do the trick.

Comment 6

5 years ago
Comment on attachment 683235 [details] [diff] [review]
Patch 1.0 (mozharness) - Bits to enable and dump logcat

I think this is good other than the parser.passed line.
r=me with that fixed; feel free to either do that or send me another review if you want.
Attachment #683235 - Flags: review?(aki) → review+
(Assignee)

Comment 7

5 years ago
(In reply to Aki Sasaki [:aki] from comment #6)
> Comment on attachment 683235 [details] [diff] [review]
> Patch 1.0 (mozharness) - Bits to enable and dump logcat
> 
> I think this is good other than the parser.passed line.
> r=me with that fixed; feel free to either do that or send me another review
> if you want.

Ok, I just copied what marionette.py did. I'll update that as well and do an inc review.

Comment 8

5 years ago
I think they're different because they're using different parsers -- marionette has its own, b2g_emulator_unittest.py uses the DesktopUnittestOutputParser.
Comment on attachment 683190 [details] [diff] [review]
Patch 1.0 (gecko) - Add logcat_dir option to b2g unittests

Review of attachment 683190 [details] [diff] [review]:
-----------------------------------------------------------------

Cool, thanks!
Attachment #683190 - Flags: review?(jgriffin) → review+
(Assignee)

Comment 10

5 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/7a0bf63a5268

I missed the merge (aurora/beta are currently closed) so I'll have to land in both those branches before landing the mozharness change. I may need some help landing on beta.. not sure if the same deal we had with aurora will still hold or not.
Whiteboard: [automation-needed-in-aurora][automation-needed-in-beta]
https://hg.mozilla.org/mozilla-central/rev/7a0bf63a5268
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla20
(Assignee)

Comment 12

5 years ago
https://hg.mozilla.org/releases/mozilla-aurora/rev/d0c7324bc500

I'm going to hold off on beta since:
a) we don't have tests running there yet
b) I don't know what the approval process is

But if we do plan on enabling tests there we'll need this patch before doing so.
(Assignee)

Updated

5 years ago
Whiteboard: [automation-needed-in-aurora][automation-needed-in-beta] → [automation-needed-in-beta]
(Assignee)

Updated

5 years ago
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 13

5 years ago
Created attachment 683608 [details] [diff] [review]
Patch 2.0 (mozharness) - Address review comment, carry r+ forward
(Assignee)

Comment 14

5 years ago
The required gecko change has landed on inbound, central and aurora. Not yet on beta but we don't have automation there anyway.

If I land the mozharness change now it will break b2g emulator tests on all of the core project branches that haven't yet been merged with m-c (e.g fx-team, services-central, cedar, etc). Is this acceptable? How often do these get merged? I can merge cedar at the least, but not sure about the others.
(Assignee)

Comment 15

5 years ago
According to philor b2g tests should be enabled on beta after the next reconfig.. so I should land the gecko patch there before landing the mozharness one.
FYI, landing process on beta is same for aurora, just a=NPOTB for automation changes.
(Assignee)

Comment 17

5 years ago
https://hg.mozilla.org/releases/mozilla-beta/rev/80afbd9cf1e5
status-firefox18: --- → fixed
status-firefox19: --- → fixed
status-firefox20: --- → fixed
Whiteboard: [automation-needed-in-beta]
(Assignee)

Comment 18

5 years ago
https://hg.mozilla.org/build/mozharness/rev/595d0844f8ef

This is one of those times where it would be nice to have some sort of mozharness try. I'll keep a close watch on the tree and be ready to back this out if there are any problems.
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED
(Assignee)

Comment 19

5 years ago
Woops, https://tbpl.mozilla.org/php/getParsedLog.php?id=17217191&tree=Mozilla-Inbound&full=1
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 20

5 years ago
Created attachment 683727 [details] [diff] [review]
Patch 2.1 (mozharness) - oops
Attachment #683727 - Flags: review?(aki)

Updated

5 years ago
Attachment #683727 - Flags: review?(aki) → review+
(Assignee)

Comment 21

5 years ago
https://hg.mozilla.org/build/mozharness/rev/5f523a752d57
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.