Open Bug 1472581 Opened 6 years ago Updated 2 years ago

Debugger reloads current URL rather than displaying source of current document

Categories

(DevTools :: Debugger, defect, P3)

Product:
DevTools
Component:
Debugger
Version:
62 Branch
Type:
defect

Tracking

(firefox62 affected)

Status:
REOPENED
Tracking Flags:
Tracking Status
firefox62 --- affected

People

(Reporter: roy-orbison, Unassigned)

References

(Blocks 1 open bug)

Details

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:62.0) Gecko/20100101 Firefox/62.0
Build ID: 20180628175621

Steps to reproduce:

Pressed F12.


Actual results:

Saw changed document at same URL reloaded from network.


Expected results:

Seen source of actual current document.
I know there are headers that expire the document, for good reason, so new requests to the URL should not fetch a cached result. However, conceptually, using the debugger should not be treated as a reload, another tab, or any other user-initiated request. The current document has to be allowed to display as a DOM in the browser so I don't see how it's a security/cache violation to have an in-memory-only copy of its source for use in the debugger, etc. If the document is reloaded by the user, this memory only copy should not be used anyway as it may have been modified by document.write(). Frankly, I'd rather see a notice that the document has expired and your tool is incapable of showing the response source, and provide an option to request again, rather than assume and make another network request. If it's a performance issue, then let us opt-in to keeping source markup in memory.

I noticed this because of a separate but related issue that the debugger is not maintaining the current Container's context, so when (re-)loading the URL, as it shouldn't, its getting redirected to an entirely different page and the resulting source displayed in the debugger has nothing to do with the currently displayed document.
I noticed it because of bug 1375036.
Regarding your STR: Can you be a bit more specific about what you mean with "actual current document" and "changed document"? It is unclear what you were expecting. 

Regarding the functionality you described: The debugger must fetch the sources independently of the browser for performance, we throw away the source as soon as it is parsed, and we load sources lazily. We make the second fetch in order to get and save the source texts. I am not sure if there are other reasons. I believe that bug has an explanation regarding the thinking behind reloading sources at https://bugzilla.mozilla.org/show_bug.cgi?id=1060732#c19 -- There is a mention of changing our approach to how we do things, but it isn't clear if this was implemented (it wasn't part of the patch that resolved that bug) I will confirm with Jim.
Flags: needinfo?(jimb)
Sorry, I mean...

Actual results:

See document source in the debugger that does not exactly match the one rendered in the current tab.


Expected results:

See the source of the currently rendered document, exactly as it was fetched (ideally including modifications made with document.write(), etc. before the document was closed for writing).


I disagree with "The debugger must fetch the sources independently of the browser for performance". It's not a "must" it's a design decision and it breaks accurate debugging as it requests a different document. If a URL can only be used once, it gets broken, if unique data is put in a response, then the source in the debugger doesn't have it. It's unexpected and unobvious. It means, when developing with Firefox, we're debugging only an approximation. I'd rather cop the extra kilobytes of RAM usage than be baffled as to why the breakpoint I'm paused at has values that don't match the code.
Hi Roy,
You might have missed the last part of my reply: 

> There is a mention of changing our approach to how we do things, but it isn't clear if this was implemented (it wasn't part of the patch that resolved that bug)

You will find more context regarding this in the link from my previous comment.
:jimb -- to clarify: can you give a bit of context about why the debugger is making a second request to get sources? specifically in the case of cookies / containers, but it looks like we have already seen similar issues with post requests.
ping :jimb, :yulia what is the status of this bug?
I don't think there has been any movement on it
I agree we ideally should not be re-fetching documents if we don't have to; in practice, fetches are side effects.

This is not something we're going to be able to look into immediately, so I'm setting its priority to P3.
Flags: needinfo?(jimb)
Priority: -- → P3

I believe that this is dup of bug 1375036.

Feel free to reopen if you see a difference.

Honza

Status: UNCONFIRMED → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE

This is not the same as bug 1375036. That one is about the debugger not following the tab's container. This one is about the fact that - even if the container is followed - the main document shown in the debugger is not the exact one from which the current page was created, the debugger instead makes a second network request for it which can result in different source code served, or other unwanted server-side effects.

Thanks for the update Roy, reopening.

Honza

Status: RESOLVED → REOPENED
Ever confirmed: true
Resolution: DUPLICATE → ---

Encountering this issue on all our current in development websites. Our organization's current security scheme results in an api auth error when the same resource is called from the debugger, thus, barring the devs from debugging the source.

I had never thought I would ever use chrome for development but how Firefox is being so unpredictable nowadays, I guess it was inevitable.

I'm sorry to hear that Umair. We're looking to remove the second fetch in bug 1538056. Hopefully when that lands this will no longer be an issue.

So 68 got installed today, the issue is still there along with some unexpected bugs introduced like view corruptions, botched extensions and stuck menus. Nevertheless, clicking on a console error leads to a plain text view of the source which is essentially not at all helpful as I can't add a break point at it.

Can someone suggest a version that has the debugger working like a debugger is supposed to? We have a critical issue reported for Firefox which we can't fix without it.

(In reply to Umair Ahmed from comment #15)

Can someone suggest a version that has the debugger working like a debugger is supposed to? We have a critical issue reported for Firefox which we can't fix without it.

Same problem here. If the debugger doesn't send a session anymore, we can't use it more. That really means bye bye Firefox :-(

I am using Firefox 69.0b11 64-bit and the bug is still present. I too regularly need to debug pages that are behind a login and the only option I have is to use Chrome - please fix this as soon as possible. Thanks.

The patch mentioned above has landed on Nightly. If folks here could test it out there to see how things look, it would be great. As long as the devtools were open before you loaded the page, they devtools should show the correct content now.

Fantastic! I used a script that generates a random string into a JS variable:

<script>
	var foo = <?= json_encode(randomchars()) ?>;
	document.write(foo);
</script>

In Firefox Dev Edition, the strings in the document and the debugger never match; in Nighty, they do!

*Nightly.

Can also confirm that the fix holds up inside a container tab. The debugger does not show content written with document.write(), but it's still a major improvement.

Yes the nightly fixes this bug, not essentially everything that was destroyed in 62. Like the fix does and does not do the following:

  1. The current content loads in the debugger
  2. Errors in console lead to plain text representation of source where one can't add break points
  3. Before document ready state, request response exceptions are not paused upon. Even if you add a break point on the same line the error occurs by manually navigating to the file and line no in the debugger (due to no. 2), it will not function before ready state is achieved.

So essentially this bug is fixed, but the debugger is till pretty botched. But at least one can debug to an extent once again. Not me though, our request client is failing on the first request (before ready state) so essentially our sites are still broken.

Thank you Umair,

and sorry that you have such a broken experience. I have not seen similar bugs filed in the past months and I have been heavily testing breakpoints and sources as well.

Could you help us by breaking out 2 (sounds like opening view-source incorrectly) and 3 (bps during loading are not hit?) into bugs with STRs. If you don't have test cases, describing the setup could also provide enough information to diagnose the issue.

Flags: needinfo?(syedumairahmed)

I have been working for 40 hours straight, please excuse any mistakes or unclear words. I'll review and investigate some more after a good rest.

Thanks for replying quickly. It wasn't very clear when I last tested but it now seems that the issues 2 and 3 may be very different in nature. Essentially this needs more investigation.

  • Our base libraries are throwing errors only in Firefox when a logged in user switches pages.
  • Always at least one request to our real-time communications server gets cancelled when a user changes pages. For some reason, Firefox interprets that behavior as an invalid JSON response error caught in the promise catcher, even though the server had not responded yet with anything.
  • This appears to happen just before Firefox changes the page, I maybe wrong as I am only depending on the visual state of the browser for this assumption. I assume JS of the new page hadn't even arrived before the current view is discarded.
  • Regardless of persistence setting, this error persists in the console. Thus it insinuates that it originates from the new location.
  • If a bp is added just before or on the error's origin, it does not trigger any pause in the flow, . But I assume debugger does not function once a location change is already in progress.
  • Trying to access this error's source, results in that plaint text view, a hint maybe that it may not belong to the new page. Our sources are consistent for all pages though so the same are present in all pages.

I have also encountered a rare strange behavior on clicking this kind of error's source in nightly only: (happened only twice in my current work stint.)

  • Clicking the source reference throws an exception of source not found.
  • While the error is displayed, the reference in console changes to a jumbled string.
  • Clicking OK on the error dialog, opens a new white space tab and Firefox asks me for line no (?) in a cancellable prompt.
  • Canceling the dialog, closes the new tab.
  • Entering a line no, displays the same dialog yet again.

The error and plain text source view: https://imgur.com/a/EwpcJrg

Flags: needinfo?(syedumairahmed)

I have also noticed that as the Inspector does get updated with new dynamic content, debugger though does not. For example getting new HTML embedded to a node, shows up correctly in the inspector but the debugger content remains the same. Perhaps its how it has been always, I only noticed it now while debugging some dynamic modals.

(In reply to Umair Ahmed from comment #24)

I have also noticed that as the Inspector does get updated with new dynamic content, debugger though does not.

That should be expected because there are infinite ways script can modify a document after it's served. Trying to keep track of them all in a debugger source would be crazy. Anything could be inserted and removed at any time, changing code blocks' line numbers and thus any breakpoints, causing scroll events, etc. It would be painful.

The only way I'm aware of for the source to be changed and still have it be interpreted as part the original document (not DOM manipulation), e.g. adding literal markup, is with document.write(). I know it's a kind of eval() and should be avoided, but it still works and anything inserted that way is treated as if it were sent over the network, so I believe it should be included in debugger sources.

Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.