Make Network Monitor log requests persistent

VERIFIED FIXED in Firefox 31

Status

defect
VERIFIED FIXED
6 years ago
11 months ago

People

(Reporter: contact, Assigned: chrishood)

Tracking

unspecified
Firefox 31

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [good first bug][mentor=vporof@mozilla.com][lang=js])

Attachments

(2 attachments, 5 obsolete attachments)

(Reporter)

Description

6 years ago
Every time a new webpage is loaded (manually or through javascript), the network monitor log is cleared, which is annoying to debug a succession of webpages.

In firebug, there's a "record" button to make those persistent.

In the Firefox devtools, there's an option Web Console -> Enable persistent logs. The Network Monitor should probably honor this preference.
Good idea!
Status: UNCONFIRMED → NEW
Ever confirmed: true
OS: Linux → All
Hardware: x86_64 → All
Whiteboard: [good first bug][mentor=vporof@mozilla.com][lang=js]
See Also: → 859057
Duplicate of this bug: 862864
(Assignee)

Comment 3

6 years ago
I'd be interested in tackling this as a way to learn more about how the network monitor works.
Hi Chris! Thanks for the interest :D Let me know if you need help with building Firefox from source [0] and running browser chrome mochitests [1].

The Network Monitor's code lives here: [2]. The tool is currently clearing its views every time a page navigation starts happening. This is done in the controller, here: [3].

This bug is about respecting the "Enable persistent logs" pref (devtools.webconsole.persistlog), used only by the WebConsole for now. It would be nice if the Network Monitor would also honor this pref.

Pref management in the Network Monitor is also done in the controller here: [4], using our shared helper [5]. You can use that to access the pref.

Afterwards, a test should be added, here: [5]. Browse through the existing tests to see how things work; you can duplicate an existing test or start writing one from scratch. Here's a very small template: [7].

Ideally, the entry in the Options panel should be changed as well, since the WebConsole won't be the only tool using the pref anymore. But I don't have any good ideas of how to structure it. What do you think we should do about that?

[0] https://developer.mozilla.org/en-US/docs/Simple_Firefox_build
[1] https://developer.mozilla.org/en/docs/Browser_chrome_tests
[2] http://dxr.mozilla.org/mozilla-central/source/browser/devtools/netmonitor
[3] http://dxr.mozilla.org/mozilla-central/source/browser/devtools/netmonitor/netmonitor-controller.js#309
[4] http://dxr.mozilla.org/mozilla-central/source/browser/devtools/netmonitor/netmonitor-controller.js#586
[5] http://dxr.mozilla.org/mozilla-central/source/browser/devtools/shared/widgets/ViewHelpers.jsm#354
[6] http://dxr.mozilla.org/mozilla-central/source/browser/devtools/netmonitor/test
[7] http://dxr.mozilla.org/mozilla-central/source/browser/devtools/netmonitor/test/browser_net_aaa_leaktest.js
Assignee: nobody → chrishood
Status: NEW → ASSIGNED
(Assignee)

Comment 5

6 years ago
Thanks for the info, I'll start looking into it.  I think the ideal solution would be to have two separate checkboxes under two separate headings, each of which would enable persistent logs for their own respective controls.  But that sounds more like a feature that could be requested later.  What I would like to do is to change Web Console to something like "Common Settings" or "Log Preferences" and possibly move it to be in between the JavaScript Profiler heading and the Advanced Settings heading.
Sounds good to me!
(Assignee)

Comment 7

6 years ago
I'm able to get the tests specific to the network monitor to pass, but I'm having problems getting the chrome test suite to pass.  There's about 300 tests that are failing and I know that 4 of them are related to fonts and there are a large number that are failing due to the datepicker popup not getting the right field.
Some of them are failing due to menu movement.
I'm currently building with out any options by simply using ./mach build and my development environment is Ubuntu running in Virtual Box.  I'll look on the introduction IRC to see if I can get some help with the tests failing.
Attach a WIP patch and I'll have a look! The other failing tests might have nothing to do with your change.
(Assignee)

Comment 9

6 years ago
They are failing regardless of whether or not any changes have been applied so I'm inclined to believe that there's an issue with my development environment.  However none of tests in the network monitor test are failing so I'm going to go ahead with development.
(In reply to Chris from comment #9)

Yes! That's a better plan :)

At most, just run the developer tool tests once you've finished. We'll push to the try servers at that point to verify that nothing else breaks globally.
(Assignee)

Comment 11

6 years ago
Posted patch bug-925275-WIP1.patch (obsolete) — Splinter Review
Uploading this WIP patch, I will finish off the TODOs in the next two to three days when I sit down to work on it again.
(Assignee)

Comment 12

6 years ago
Posted patch bug-925275-WIP2.patch (obsolete) — Splinter Review
Everything is finished except for the unit tests.
Attachment #8344969 - Attachment is obsolete: true
Comment on attachment 8346891 [details] [diff] [review]
bug-925275-WIP2.patch

Review of attachment 8346891 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good so far!
Make sure you don't leave trailing whitespace in the edited files.
Attachment #8346891 - Flags: feedback+
(Assignee)

Comment 14

6 years ago
Thanks for the feedback.  I'll go back and double check for trailing whitespace.
Chris, are you still working on this bug?  It's fine if you are! :) Just double checking, in case others may want to give it a shot.
Flags: needinfo?(chrishood)
(Assignee)

Comment 16

5 years ago
Sorry for taking so long, I've been bogged down with work.  I'll try to get around to finish it this weekend.  After that if I still don't get to it then if someone else wants to take over the only thing left is to just write the test case and fix that one extra trailing whitespace that I put in by accident in /browser/locales/en-US/chrome/browser/devtools/toolbox.dtd
Flags: needinfo?(chrishood)
Okay, no worries, keep working on it for sure!  Just wanted to double check. :)
(Assignee)

Comment 18

5 years ago
The way I was thinking about going about doing this is to:
1: turn on the preference (Or perform some equivalent action)
2: Log some data by loading a sample web page.
3: Navigate to a separate sample web page.
4: Check the log to make sure that the data is still there (I'll probably just check for a string that should appear in the logs from the old webpage but not the new one.)

Right now I'm just manually turning on the preference, is there some other way I should be going about this?  I've noticed that none of the other tests seem to be actually setting the preferences outside of head.js.

Please let me know if there's anything else I should be testing with this.
Flags: needinfo?(vporof)
(In reply to Chris from comment #18)

Don't feel rushed into fixing this :) Thanks for still being interested.

> The way I was thinking about going about doing this is to:
> 1: turn on the preference (Or perform some equivalent action)
> 2: Log some data by loading a sample web page.
> 3: Navigate to a separate sample web page.
> 4: Check the log to make sure that the data is still there (I'll probably
> just check for a string that should appear in the logs from the old webpage
> but not the new one.)
> 
> Right now I'm just manually turning on the preference, is there some other
> way I should be going about this?  I've noticed that none of the other tests
> seem to be actually setting the preferences outside of head.js.
> 
> Please let me know if there's anything else I should be testing with this.

Manual testing is good, but to land this new feature we need an automated test. Basically, all the steps described above should be automated in a mochitest. See the other netmonitor tests for some inspiration; you can clone one of them and modify it to do what you wrote above.
Flags: needinfo?(vporof)
There may be some merit to making this a per-panel pref instead of a global pref. Chrome does this, for example.


If I get time I may be able to help with the testcase (I'm new to writing tests as well).
(Assignee)

Comment 21

5 years ago
I'm going to take another look at it this weekend to see if I can get a working test case.  Is it okay to make this a per-panel preference as a part of this bug or should that be done as apart of another bug?
Flags: needinfo?(vporof)
I would say that it's best to continue with your current patch, and we can look into making a separate panelwise button later. (I'm not your mentor on this, though, you may wish to wait for him)

I think you can go ahead with the testcase though, if you do switch the functionality or add new functionality it would be easy to extend the textcase to work.
(In reply to Chris from comment #21)
> I'm going to take another look at it this weekend to see if I can get a
> working test case.  Is it okay to make this a per-panel preference as a part
> of this bug or should that be done as apart of another bug?

Either way is fine by me. Best thing is to hack something that's landable, and file a followup bug if there's anything else that needs fixing.
Flags: needinfo?(vporof)
(Assignee)

Comment 24

5 years ago
Manish you can take over for this if you'd like.
Extremely busy at the moment, got a lot of coursework. Will look at later if you've not finished it.

Comment 26

5 years ago
Posted patch 925275.patch (obsolete) — Splinter Review
Hello,

I 've managed to put a test together and updated the patch.
Attachment #8388219 - Flags: feedback?(vporof)
Comment on attachment 8388219 [details] [diff] [review]
925275.patch

Review of attachment 8388219 [details] [diff] [review]:
-----------------------------------------------------------------

Woooo! This is awesome! Thank you Christos! I just want you to fix a few small nits, and this should be good to go.
Pushed to try: https://tbpl.mozilla.org/?tree=Try&rev=5e6773371123

r+ with the changes below and a green try run.

::: browser/devtools/framework/toolbox-options.xul
@@ +75,5 @@
>            <checkbox label="&options.showPlatformData.label;"
>                      tooltiptext="&options.showPlatformData.tooltip;"
>                      data-pref="devtools.profiler.ui.show-platform-data"/>
>          </vbox>
> +        <label value="&options.commonPrefs.label;"/>

How about making the common prefs appear immediately after the "Theme" option, instead of here? Would that take a lot of work or is it straightforward. If it's easy, do the change here please.

::: browser/devtools/netmonitor/netmonitor-controller.js
@@ +417,5 @@
>    _onTabNavigated: function(aType, aPacket) {
>      switch (aType) {
>        case "will-navigate": {
>          // Reset UI.
> +        if (!Services.prefs.getBoolPref("devtools.webconsole.persistlog")) {

Please use a Prefs instance to interact with preferences [0] in this file.
Place the following below [0]:

let WebConsolePrefs = new ViewHelpers.Prefs("devtools.webconsole", {
  persistLog: ["Bool", "persistlog"],
});

and use it here:

if (!WebConsolePrefs.persistLog) {
  ...
}

[0] http://dxr.mozilla.org/mozilla-central/source/browser/devtools/netmonitor/netmonitor-controller.js#732

@@ +424,4 @@
>  
> +          // Switch to the default network traffic inspector view.
> +          if (NetMonitorController.getCurrentActivity() == ACTIVITY_TYPE.NONE) {
> +            NetMonitorView.showNetworkInspectorView();

This stuff should be outside the above conditional. Switching to the network inspector view should always happen on refresh.

::: browser/locales/en-US/chrome/browser/devtools/toolbox.dtd
@@ +144,5 @@
> +<!ENTITY options.commonPrefs.label           "Common Preferences">
> +
> +<!-- LOCALIZATION NOTE (options.enablePersistentLogging.label): This is the
> +  -  label for the checkbox that toggles persistent logs in the Web Console and
> +  -  network monitor,  i.e. devtools.webconsole.persistlog a boolean preference in 

Nit: please remove the whitespace here.
Attachment #8388219 - Flags: feedback?(vporof) → feedback+

Comment 28

5 years ago
> ::: browser/devtools/netmonitor/netmonitor-controller.js
> @@ +417,5 @@
> >    _onTabNavigated: function(aType, aPacket) {
> >      switch (aType) {
> >        case "will-navigate": {
> >          // Reset UI.
> > +        if (!Services.prefs.getBoolPref("devtools.webconsole.persistlog")) {
> 
> Please use a Prefs instance to interact with preferences [0] in this file.
> Place the following below [0]:
> 
> let WebConsolePrefs = new ViewHelpers.Prefs("devtools.webconsole", {
>   persistLog: ["Bool", "persistlog"],
> });
> 
> and use it here:
> 
> if (!WebConsolePrefs.persistLog) {
>   ...
> }
> 
> [0]
> http://dxr.mozilla.org/mozilla-central/source/browser/devtools/netmonitor/
> netmonitor-controller.js#732

After this change, the WebConsolePrefs.persistLog always remember the initial value it had when the browser is loaded, ignoring any change afterwards. I don't know why this is happening. It makes the test fail. I am attaching the patch for investigation.

Comment 29

5 years ago
Posted patch 925275.patch (obsolete) — Splinter Review
Attachment #8388219 - Attachment is obsolete: true
Attachment #8391852 - Flags: feedback?(vporof)
Comment on attachment 8391852 [details] [diff] [review]
925275.patch

Review of attachment 8391852 [details] [diff] [review]:
-----------------------------------------------------------------

Hmm, you're right. I forgot about that. I seems premature to make Prefs handle updated preferences, so go ahead and keep using Services.prefs for now :)
Attachment #8391852 - Flags: feedback?(vporof)
(Assignee)

Comment 31

5 years ago
Thanks for writing up the test case for this Christos, It'll be nice to see this finally getting added.

Comment 32

5 years ago
Posted patch 925275.patchSplinter Review
Reverted back to version using Services.prefs for accessing preferences.
Attachment #8391852 - Attachment is obsolete: true
Attachment #8395347 - Flags: review?(vporof)
Comment on attachment 8395347 [details] [diff] [review]
925275.patch

Review of attachment 8395347 [details] [diff] [review]:
-----------------------------------------------------------------

Alright! Thank you! I'll push to try, and if everything is green, please add a checkin-needed keyword.
Attachment #8395347 - Flags: review?(vporof) → review+
Attachment #8346891 - Attachment is obsolete: true
Chris, this seems to be failing, so we can't land it just yet. Can you please take a look at this?
Flags: needinfo?(chrishood)
(Assignee)

Comment 36

5 years ago
I'll run it through the unit tests on my box and see what's failing.
Flags: needinfo?(chrishood)
(Assignee)

Comment 37

5 years ago
Nevermind, I just saw which test was failing, I'll take a look and see if I can fix it.
(Assignee)

Comment 38

5 years ago
It looks like persistent prefs just needed to be turned off prior to exiting the new test case, all netmonitor tests are passing on my box as of now.
Attachment #8398949 - Flags: review+
(Assignee)

Comment 39

5 years ago
Turned off persistent logs before exiting browser_net_persistent_logs test.
Attachment #8398949 - Attachment is obsolete: true
Attachment #8398950 - Flags: review?(vporof)
Attachment #8398950 - Flags: review?(vporof) → review+
Thanks!
Keywords: checkin-needed
https://hg.mozilla.org/integration/fx-team/rev/aaeff6d3bb3b
Keywords: checkin-needed
Whiteboard: [good first bug][mentor=vporof@mozilla.com][lang=js] → [good first bug][mentor=vporof@mozilla.com][lang=js][fixed-in-fx-team]
https://hg.mozilla.org/mozilla-central/rev/aaeff6d3bb3b
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Whiteboard: [good first bug][mentor=vporof@mozilla.com][lang=js][fixed-in-fx-team] → [good first bug][mentor=vporof@mozilla.com][lang=js]
Target Milestone: --- → Firefox 31
Depends on: 1006299
QA Whiteboard: [good first verify]
Starting with 2017-09-01, the Log Persistence checkbox was implemented in Firefox 57, so we can consider this bug fixed and verified across platforms.
Status: RESOLVED → VERIFIED

Updated

11 months ago
Product: Firefox → DevTools
You need to log in before you can comment on or make changes to this bug.