Open Bug 386218 Opened 13 years ago Updated 9 years ago

make debug/trace logging circular and automatic

Categories

(Thunderbird :: General, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

People

(Reporter: bugzilla, Unassigned)

References

(Blocks 1 open bug)

Details

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4
Build Identifier: 2.0.0.4 (20070604)

I need to use the debug/trace facility to isolate a possible IMAP-related problem and follow the instructions at:

http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap

and now have this small .bat file to start TB for me:

cd "\Documents and Settings\don\Desktop\imap-log"
set NSPR_LOG_MODULES=imap:5
set NSPR_LOG_FILE=log.txt
start thunderbird

The log file can grow unacceptably large while I wait for my "strange event" to show itself. (Of course I can stop/start TB to overwrite the log file, but that's just wrong. :-) )

I propose a slight enhancement to the logging code:

If NSPR_LOG_FILE is a *directory*, then the logging mechanism should create a new file in that directory, and periodically close/open a new file.
(Or, perhaps a new option: NSPR_LOG_DIRECTORY=)

Perhaps a name like <date><time>.txt (i.e. 2007-06-28T14-30-00.txt) or something that sorts nicely in chronological order... hmmm, maybe UTC in case somebody is tracing across a Daylight Saving Time change.

The "close/open new" event could be periodic (every hour?), and/or based on log file size. Perhaps when the file reaches 1MB or 10,000 "events", or ?? Obviously care is needed so no logging events are lost.

Start a new file each time TB is started, regardless of the size of the previous log file.

Reproducible: Always

Steps to Reproduce:
1. n/a - this is an RFE
2. See 1
3. See 2 :-)
Keywords: mail6
Version: unspecified → 2.0
Assignee: nobody → wtc
Component: Mail Window Front End → NSPR
Product: Thunderbird → NSPR
QA Contact: front-end → nspr
Version: 2.0 → other
what size is "unacceptably large"?  and why is "large" a problem?
(In reply to comment #1)
> what size is "unacceptably large"?  and why is "large" a problem?

That's rather subjective.... I suggested a limit of 1MByte or so, then start a new file. Maybe 1M is too small... what's a reasonable size file for vim to deal with in a reasonably efficient manner? 

Large is a nuisance, if I have an event that occurs rarely, I could get a trace file that's many many megabytes... if files are kept to a size of a few meg, then it's easier to delete old files than it is to delete old data from a file while that file may be written to.

Just suggestions... :-)

I keep logging on all the time for imap, and have encountered the "big file" editor issue. In fact I normally forget it's on - the file is currently 214MB.

But I'm not sure the value of what you are asking is worth the effort, because ...  "tail" utility immediately addresses the editor issue. The program of course exists in unix world. You can get a free one on windows (and less, find, grep, etc). I don't know what's available for Mac.
I don't mind if you want to just write continuously to a single file. I'm mostly looking for the ease of turning debugging on/off. :-)

Perhaps it could be added to "logrotate" like other log files...
(In reply to comment #4)
> I don't mind if you want to just write continuously to a single file. I'm
> mostly looking for the ease of turning debugging on/off. :-)

we are finally at what you *really* want - which is Bug 193873 – Add Mozilla logging to UI

whereas mozilla is not a "logging" system, suggest we dupe this bug, morph it to Thunderbird, or file a new one for thunderbird. 

This is an important, and I think strategic QA issue for Thunderbird.
cc: Bryan and dmose for comment
I think this is a thunderbird or mail/news RFE, and not an NSPR bug.
A certain other mail/news program has the property that it ALWAYS keeps a 
short log of the previous protocol activity, and any time there is an 
error, the user has the option, at the click of a link, to display the
log.  The log typically includes the protocol activity (requests & responses)
last sent over the particular connection on which the failure occurred.

There's no long list of directions on a web page to go find.  No need to 
- stop the product, 
- set environment variables to enable logging,
- restart the product,
- attempt to reproduce the failure,
- stop the product again,
- look at log files.

Instead, there's a one-click activity, while the product is still running,
that requires no advanced configuration or setup, that opens a window 
containing the relevant log info.  That UI is easy & simple for users to use, 
useful for them to supply useable debugging into.  

So, I suggest we raise the vision here, from "tweak NSPR a little", and 
from "allow users to configure logs to run continuously so that they can 
catch the rare bug", to logs that always run continuously, need no 
configuration, never grow to be huge, and are available at a click.  
That's clearly a mail/news RFE.
Nelson, I'm glad you raise these good issues, many of which I noted a few years ago in bug 257942 comment 4.

For advanced logging - are you are referring to The Bat! log http://www.ritlabs.com/en/products/thebat/ - which is also noted in bug 257942?


Moving to Thunderbird general. But I'm not convinced there are not backend NSPR bits needed like
- is there a way for NSPR logging to be on *by default* for trunk, debug or production builds?
- is there a mechanism for the an in-memory trace file
- size of the in memory log, and can/should it be adjustable size
- turning same on and off

Not duping to bug 257942 because people have been asking for 3 things if not more and we need to sort out what bugs will address what :
a) wraping trace log and control of tracing via UI
b) queryable snapshot/summary of each account's status
c) continuous display in the UI of an account's status

Perhaps discussion is needed in m.d.a.Thunderbird or netscape.public.mozilla.mail-news to determine which, if any, of the above are needed and how to best deliver the UI.

*Frank DiLecce's Progress History extension provided part of b) 
I tried it several times, but it is in disrepair and doesn't work in TB 2 or 3 afaik. http://forum.addonsmirror.net/index.php?showtopic=2031&mode=threaded
"records all status messages from all TB windows in a list and lets you see progress messages."
Assignee: wtc → nobody
Component: NSPR → General
Product: NSPR → Thunderbird
QA Contact: nspr → general
Summary: RFE to allow debug/trace logging to directory → allow debug/trace logging of mail events to directory
Version: other → unspecified
I've just looked at this and it seems like the best course of action might be to dump the NSPR logging and implement a specialized C++ circular logger class which exposes an XPCOM interface to get at its data.  Slap some xul/javascript on the front, and we have a poor man's firebug, or possibly a poor man's firebug plugin which can pastebin with just a click.

The logger instance itself can be associated with the account, and some slight meta-data can be associated with each 'row' of data.  Namely, it seems useful to explicitly track sending versus receiving and maybe versus assertions of state.  Otherwise, what is currently colon-delimited could potentially just become nul-delimited, with the initial logger class instantiation mapping the positional arguments to names for the XPCOM interface.  We use a fixed-size 'row' of 512 (which is in keeping with NSPR if the IMAP comments are to believed), control the number of 'rows' per logger via preferences with account overrides, choose a low default number, and we're good.

Is there any existing test infrastructure or real-world usage that would balk at removing the existing PR_LOG stuff?  We could add a command-line handler and logging to disk off the bat, but it seems like circular logging and perhaps some kind of snapshot capability triggered by the protocol code when it observes an inconsistency (with JS UI) would be ideal.
I agree that we should really do better here.  I don't have a terribly strong opinion on how we should go about it.  

I would go so far as to say this problem wants to be generally solved for the platform, rather than specifically for messaging.  That said, messaging may well be a reasonable place to start.  

Other recent platform-wide discussion about logging has included consideration of log4moz as well as Flock's C++ logging code.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: mail6
Summary: allow debug/trace logging of mail events to directory → make debug/trace logging circular and automatic
I did a circ trace log for an online application some years ago, kermit-cics, and was immensely useful for debugging purposes. However, it was in addition to the CICS monitor's trace and augmented the CICS trace. For similar reasons, it may not be possible for circular trace in TB to fully replace PR_LOGging.

The "per account" and pastebin ideas are great. (and anxious for anything with potential time savings, faster prob resolution, and saving of hair for those for who have trouble with PR_LOGging)

(wow, firebug has 174,827 weekly downloads!)
Blocks: tb-logging
You need to log in before you can comment on or make changes to this bug.