Closed Bug 841566 Opened 8 years ago Closed 8 years ago

Turn on testserver logging for WebRTC-related systems

Categories

(Testing :: Mochitest, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla22

People

(Reporter: abr, Assigned: abr)

References

Details

Attachments

(1 file, 1 obsolete file)

Currently, progress on intermittent bug 841496, bug 841150, and bug 839677
has proven difficult; these bugs are difficult to reproduce, and TBPL logs
provide only the roughest estimation of where something may have gone wrong.
For as long as these (and similar) intermittant bugs exist, we would benefit
greatly from having relevant logging turned on.
This patch should activate logging on the test servers for the signaling,
webrtc, ice, and transport subsystems.
If all goes well, our additional debugging should show up in the logs for this push:

https://tbpl.mozilla.org/?tree=Try&rev=4247a24bb8e5
Blocks: 841496
Blocks: 839677
Blocks: 841150
Re-try after fixing the patch for bug 841457:
https://tbpl.mozilla.org/?tree=Try&rev=a437dbcdc5f7
Okay, that that looks like it actually broke automation. Third time's a charm...
https://tbpl.mozilla.org/?tree=Try&rev=8f874b4ae911
Component: Release Engineering: Automation (General) → Mochitest
Product: mozilla.org → Testing
QA Contact: catlee
Version: other → Trunk
I told Adam that the sheriffs were the people most likely to care about stuffing extra debug logging into the tinderbox logs.
Looks like Ethan just took care of some of the FORCE_PR_LOG stuff that I was going to have to work with -- I'm going to depend this bug on that one, since I've just rebased my (local, work-in-progress) patch on top of that one.
Depends on: 841641
Okay, this did manage to turn on ICE debugging, so it's now clear that the environment variables are being propagated to the tests:

https://tbpl.mozilla.org/?tree=Try&rev=3e2ffe882d36
Attachment #714120 - Attachment is obsolete: true
I think this most recent patch probably does what we want. It uses a reasonable log level for ICE, signaling, and mtransport. I've also demoted a couple of relatively noisy log messages that weren't adding any value at this level, as well as lengthened a buffer that was too short (and generating a metric ton of error logs as a consequence).

Proof's in the pudding -- the peerconnection mochi tests should contain both ICE logging and "cpr" logging in both opt and debug builds:

https://tbpl.mozilla.org/?tree=Try&rev=25e59929e596
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #5)
> I told Adam that the sheriffs were the people most likely to care about
> stuffing extra debug logging into the tinderbox logs.

Roughly how many additional lines of output are we talking about per mochitest chunk)? (However, if this is just temporary then I don't mind regardless of length) :-)
Ed -- here's the resulting output. All three of these should look roughly the same, and gives you a good feel for how much extra output we'll be introducing.

And yes, the intention is very much that this is temporary.

Example linux opt log:
https://tbpl.mozilla.org/php/getParsedLog.php?id=19974947&tree=Try&full=1

Example windows opt log:
https://tbpl.mozilla.org/php/getParsedLog.php?id=19979225&tree=Try&full=1

Example linux debug log:
https://tbpl.mozilla.org/php/getParsedLog.php?id=19975741&tree=Try&full=1
Ryan -- can you take a look at the logs in comment 11 and let me know if you have any objection to my turning on this level of logging temporarily until we nail down the root cause of some intermittent bugs?
Flags: needinfo?(ryanvm)
Phil -- same question as I have for Ryan above.
Flags: needinfo?(philringnalda)
Looks fine to me.
Flags: needinfo?(ryanvm)
Ed -- given that the plan is for this to be temporary, I'm basing your position on the statement "I don't mind regardless of length".
Yup, no qualms with this at all; but thank you for checking :-)
Attachment #716926 - Flags: review?(rjesup)
Yeah, mochitest-3 logs are half or a third the size of things like mochitest-1 and browser-chrome, you'd have to get stuck spewing in a loop for a while to run into any problems.
Flags: needinfo?(philringnalda)
Comment on attachment 716926 [details] [diff] [review]
Turn on testserver logging for WebRTC-related systems

Review of attachment 716926 [details] [diff] [review]:
-----------------------------------------------------------------

r+, though a build peer should look at automation.py.in
Attachment #716926 - Flags: review?(ted)
Attachment #716926 - Flags: review?(rjesup)
Attachment #716926 - Flags: review+
Comment on attachment 716926 [details] [diff] [review]
Turn on testserver logging for WebRTC-related systems

Review of attachment 716926 [details] [diff] [review]:
-----------------------------------------------------------------

Technically you need a test harness peer, not a build peer, but conveniently I own both of those modules anyway.
Attachment #716926 - Flags: review?(ted) → review+
https://hg.mozilla.org/mozilla-central/rev/86c4d6a9775a
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla22
Depends on: 850128
You need to log in before you can comment on or make changes to this bug.