1687549 - Allow to limit log file size when using MOZ_LOG=append,...

Status: RESOLVED FIXED

(Core :: XPCOM, enhancement)

Description

[Nick Alexander :nalexander [he/him]]

Use case first. I want to capture logs for multiple independent Gecko/Firefox invocations happening over time. That is, I'm going to run Firefox in a special background task/maintenance mode every X hours, and I want to be able to log each run. I want to capture a cumulative historical log, but I don't want the log to grow without bound.

The existing append option allows to accumulate logs. The rotate:SIZE option added in Bug 1244306 allows to limit the growth of the log, but it's both incompatible with append and not well suited for a single cumulative historical log across multiple runs, since it's unclear how to manage the current log file index across invocations.

I propose an additional flag, say maxsize:SIZE, that would trim a log file of N >= SIZE Mb to, say, N//2 Mb before opening it. Since my background tasks are expected to be short-lived (but frequent) and to mostly be no-ops with very little logging, this should solve my immediate problem. By doing any trimming at file open time I expect to avoid most threading issues.

Comment 1

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1687549 - Allow to limit MOZ_LOG file with `maxsize:N` when using `append`. r?#xpcom-reviewers

The existing append option allows to accumulate logs. The rotate:N
option added in Bug 1244306 allows to limit the growth of the log, but
it's both incompatible with append and not well suited for a single
cumulative historical log across multiple runs, since it's unclear how
to manage the current log file index across invocations. Continuously
limiting the file size for each log message is technically
challenging.

This commit pursues a much simpler approach. When maxsize:N is
specified with append, before a log file is opened, it "tails" the
file to be at most N // 2 bytes long. It tries to keep lines intact
at low runtime memory cost, but allows very long lines may be cut.

Comment 2

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1687549 - Allow to limit MOZ_LOG file with `maxsize:N` when using `append`. r?nika

The existing append option allows to accumulate logs. The rotate:N
option added in Bug 1244306 allows to limit the growth of the log, but
it's both incompatible with append and not well suited for a single
cumulative historical log across multiple runs, since it's unclear how
to manage the current log file index across invocations. Continuously
limiting the file size for each log message is technically
challenging.

This commit pursues a much simpler approach. When maxsize:N is
specified with append, before a log file is opened, it "tails" the
file to be at most N // 2 bytes long. It tries to keep lines intact
at low runtime memory cost, but allows very long lines may be cut.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/da3640fcab7b
Allow to limit MOZ_LOG file with `maxsize:N` when using `append`. r=xpcom-reviewers,kmag

Comment 4

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/da3640fcab7b