1045722 - Increase Console Log Limit

Status: RESOLVED FIXED

(DevTools :: Console, defect)

Description

[Rob Campbell [:rc] (:robcee)]

Right now, devtools.hud.loglimit.console is set to 200. We should bump this to 2000 or 20000. It should be more than it is. (See Chrome)

Comment 1

[Lin Clark [:linclark]]

Looking at the Chrome devtools code, I think that its limit is only 300.

from devtools/front_end/console/ConsoleView.js, line 160

> WebInspector.ConsoleView.persistedHistorySize = 300;

Comment 2

[Lin Clark [:linclark]]

Attachment: Bug1045722.patch

This patch bumps the DEFAULT_LOG_LIMIT to match Chrome's.

I also removed what I think is cruft from browser/app/profile/firefox.js. The preferences were hardcoded with a default. It seemed like the code in logLimitForCategory was meant to fallback to DEFAULT_LOG_LIMIT if nothing was set, so hardcoding these preferences should be unnecessary.

Comment 3

[Brian Grinstead [:bgrins]]

(In reply to Lin Clark from comment #1)
> Looking at the Chrome devtools code, I think that its limit is only 300.
> 
> from devtools/front_end/console/ConsoleView.js, line 160
> 
> > WebInspector.ConsoleView.persistedHistorySize = 300;

I'm guessing that is the number of history entries that stay in the stack (for pressing up in the console to go through the last commands you've entered).  AFAICT they keep at least 10000 entries in the log based on my test case here:  https://bgrins.github.io/devtools-demos/console/stress.html.

I remember we have perf issues when trying to render that many in the view (https://bugzilla.mozilla.org/show_bug.cgi?id=1045715#c17).  We have a meta bug opened to improve this (Bug 1156747).

That said, I think increasing this limit a bit could make sense, since 200 seems like a pretty low number that you could pretty easily run into during normal development.

Comment 4

[Brian Grinstead [:bgrins]]

We may be able to tweak DEFAULT_LOG_LIMIT / MESSAGES_IN_INTERVAL / MAX_CLEANUP_TIME / THROTTLE_UPDATES, etc to allow a much bigger log limit.  It's been a while since we've looked into this, and the performance tools available in the Browser Toolbox are much better now so should be able to give better results.

Comment 5

[Brian Grinstead [:bgrins]]

Comment on attachment 8645837 [details] [diff] [review]
Bug1045722.patch

Review of attachment 8645837 [details] [diff] [review]:
-----------------------------------------------------------------

Bumping this to 1000 doesn't seem to cause any noticeable change in performance (at least in my simple test case at https://bgrins.github.io/devtools-demos/console/stress.html).  Let's go with that for now (while keeping the prefs and updating them to the new number)

::: browser/app/profile/firefox.js
@@ -1605,5 @@
>  pref("devtools.webconsole.timestampMessages", false);
>  
> -// The number of lines that are displayed in the web console for the Net,
> -// CSS, JS and Web Developer categories.
> -pref("devtools.hud.loglimit.network", 200);

I think we should actually keep these but bump their values up to the new value, see: https://dxr.mozilla.org/mozilla-central/source/browser/devtools/webconsole/webconsole.js#4821.

It is sort of awkward to have the same const defined twice, but we use this feature sometimes in tests: https://dxr.mozilla.org/mozilla-central/source/browser/devtools/webconsole/test/browser_webconsole_bug_644419_log_limits.js#136

Comment 6

[Lin Clark [:linclark]]

When devtools.hud.loglimit.X is used in tests, is the hardcoded preference that is set in the profile ever used? It looks like the tests set the preference via the API (and then clear the preference afterwards), so that the test would never end up using the preference that the profile initializes. I might be wrong about how this works though. I ran the full browser test suite with those lines removed and there were no failures, but three were skipped.

Comment 7

[Brian Grinstead [:bgrins]]

(In reply to Lin Clark from comment #6)
> When devtools.hud.loglimit.X is used in tests, is the hardcoded preference
> that is set in the profile ever used? It looks like the tests set the
> preference via the API (and then clear the preference afterwards), so that
> the test would never end up using the preference that the profile
> initializes. I might be wrong about how this works though. I ran the full
> browser test suite with those lines removed and there were no failures, but
> three were skipped.

You are right, after looking again it does look like it's always set for the test.

Even still, I'd probably keep them around just because of convention (and it makes them easier to find if you were inclined or told to change them to work around the limit).  The duplicated number is weird, but it's not too bad to keep in sync since this changes infrequently.  Maybe it would help to add a comment above the prefs in firefox.js saying it should be kept in sync with DEFAULT_LOG_LIMIT in the webconsole frontend.

Comment 8

[Lin Clark [:linclark]]

Attachment: Bug1045722.patch

That makes sense, thanks for the feedback. 

This patch updates the limit to 1000 and adds the comment to keep them synced in profile.js.

Comment 9

[Brian Grinstead [:bgrins]]

Comment on attachment 8646314 [details] [diff] [review]
Bug1045722.patch

Review of attachment 8646314 [details] [diff] [review]:
-----------------------------------------------------------------

Nice :).  Try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=8daa9790c850

Comment 10

[Brian Grinstead [:bgrins]]

Try push looks generally good

Comment 11

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/1edc9ce636a4

Comment 12

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/integration/fx-team/rev/1edc9ce636a4

Comment 13

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/1edc9ce636a4

Comment 14

[Jim Michaels]

what about x64 boxes and the fact that when a stop is offered on a forever-loop script, the first log output is dropped off the planet?

Comment 15

[Jim Michaels]

are any of these fixes something I can simply plop into the console and run to fix the console history length?

Comment 16

[Brian Grinstead [:bgrins]]

(In reply to Jim Michaels from comment #15)
> are any of these fixes something I can simply plop into the console and run
> to fix the console history length?

I'm not sure I understand the use case: is the console opened while the logs are happening or do you want to get access to logs that happened before the console was opened?  And also are you using the web console or browser console?

Comment 17

[Jim Michaels]

my log finally stopped (at a race of 12s reaction time) 80,000 entries ago and long since lost the original reason why my script is failing. possible to keep first few entries? in c++ you simply:

if (index<deque.size()) {
    deque.erase(deque.begin()+index);
}
to delete one entry,

if (index<deque.size()) {
    deque.erase(deque.begin()+index1,deque.begin()+index2);
}
to delete a range starting at index1 and up to but not including index2.

and show a ... between. those 1st entries can make a difference. so a new profile entry is needed, one that keeps a count of how many history entries to keep from the start of the log. like console_history_log_start_keep_count or something.
and reduce by same amount the rest of the entries. but it should be ignored if it's over the log history size of course. 


in response to comment#16, whatever it is that it's called when you ctrl-shift-i and click console, where console.log statements output to.

Comment 18

[Brian Grinstead [:bgrins]]

(In reply to Jim Michaels from comment #17)
> in response to comment#16, whatever it is that it's called when you
> ctrl-shift-i and click console, where console.log statements output to.

The number of messages that get stored before you open the console is hardcoded to 1000 here: https://dxr.mozilla.org/mozilla-central/source/dom/console/ConsoleAPIStorage.js#17.  This is capped since they are stored for any page even if you never open the console. I don't see a reason why that couldn't be a pref though.

And the number of messages shown in the frontend once the console is opened is controlled by the devtools.hud.loglimit pref if you are on nightly or the devtools.hud.loglimit.console pref if you are devedition/beta/stable.  This can be > 1000 as long as the console is opened when they occur since those messages are observed as they happen.

Comment 19

[Jim Michaels]

can't get the source code for ff or any nice binaries of any version number like in the past or the x64 version of ff, ftp server has been permanently down for years.

Comment 20

[Jim Michaels]

even if certain special versions were offered, like the maximal browser for certain versions of platforms, like
3.1? if it goes that far back, 98, XP, 2000, vista, (7, 8, 8.1 10)?.
with directory names that are labeled as such. even the web site should offer that.
sorry for being off-topic. under these there should be 32 and 64-bit platforms where possible, mac versions for appropriate macos versions (I know nothing about that, but I am sure there's something to that, because there was a cpu changeover several times - 68000->68020->68040->ppc->intel I think and then there's cocoa vs the other gui you can code for).
then there's ubuntu, linux, bsd, bsdi, gentoo, linux platforms in 32 and 64-bit with differing GUIs like xfce, kde, cinnamon, mate, differing versions of that gui I am not allowed to say.
I would really like to see an wattcp-based DOS browser if DOS sticks around long enough (I like coding for it, and DJGPP has fsu pthreads and pth that I would like to try once they get an x64-hosted compiler working).