893696 - Sources don't update after being edited

Status: RESOLVED FIXED

(DevTools :: Debugger, defect, P2)

Description

[Victor Porof [:vporof][:vp]]

Attachment: test.html

This is really bad. Basically, after prototyping something and installing it in the Simulator, the debugger seems to never use the new sources, not even after a refresh or reconnect. Sounds like we're dealing with a perpetually cached but never invalidated source.

STR:

1. Create a simple test.html document that contains a script which does something (use the attached file for example).
2. Start a server in that directory, for example using python
> python -m SimpleHTTPServer 1234
3. Open the simulator dashboard
4. "Add URL" for http://localhost:1234/test.html as an in the simulator
5. Click "Connect" for the newly installed app
6. Go to debugger and observe the code (in the case of the attached file, see that "console.log(1)" line. Also, notice that the webconsole franticly prints 1.
7. Open your favorite text editor and replace that line with something else, like "console.log(2)"
8. Refresh the app in the simulator (either via the Refresh button in the dashboard, or cmd/ctrl+R on the app itself)

Now, if you go to the debugger and see that line didn't change and still shows "console.log(1)". That's quite sad, since the webconsole now franticly prints 2.

If you go a step further, and click connect again, the debugger *still* didn't update the source contents. That's increasingly sad.

After getting annoyed, you decide to close the toolbox and connect again. The debugger *still* didn't update the source contents. That's incredibly sad.

Without investigating, it sounds to me like either we're not handling (or not receiving) the tab navigation event as we should. Another possibility could be that the source is cached on the server and everything else is blown into oblivion.

What do you think, Panos?

Comment 1

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

I suspect this has to do with the heavy handed caching we use when loading sources. It has bitten me a few times with source mapped sources. I had to add a `loadFromCache` flag which (if false) bypasses the cache completely. It seems equally heavy handed but on the other side of the spectrum. Unfortunately I wasn't able to solve the problem using the "load from cache normally" flag with the channel we open.

See http://mxr.mozilla.org/mozilla-central/source/toolkit/devtools/server/actors/script.js#2829

Comment 2

[Victor Porof [:vporof][:vp]]

This should be retested after bug 886848 lands.

Comment 3

[Panos Astithas (he/him) [:past] (please ni?)]

Not going to work on this any time soon.

Comment 4

[Eddy Bruel [:ejpbruel]]

It looks like I've ran into the same bug several times when testing worker debugging. I have a test html page with some worker scripts that I serve using python's SimpleHTTPServer. When I change any of them the debugger keeps using the old sources. Nothing I do seems to coerce the debugger to use the new version, not even restarting Nightly and removing the file. The only thing that worked for me is renaming the file.

I'm surprised we haven't seen more reports of this. I can't imagine I'm the only one who wants to debug a few test files in a directory behind a server. This seems like a fairly basic use case, and we should get it fixed.

I also suspect that the heavyhanded caching we use for loading sources might be at fault here, but I don't fully understand *why* that would cause this behavior. Nick, do you have any idea what might be going on here?

Comment 5

[Nick Fitzgerald [:fitzgen] [⏰PST; UTC-8]]

I don't know anything about how the simulator works.

Eddy, for your case, what kind of sources were these?

How were they linked/included/eval'd in the page?

Are you seeing RDP requests for the sources on reload? If not, then we aren't invalidating the client's source cache at the appropriate time.

If you are seeing the RDP requests then its either D.Source.p.text not being updated (seems unlikely to me since these will be different JSScript/D.Script and ScriptSource/D.Source instances) or this is a source that the debugger doesn't believe it can use D.Source for yet (see bug 905700) and is retrieving the source from the browser's network cache. The latter seems more plausible, but still doesn't make sense to me. The whole reason we rely on the network cache is to get exactly the source that the browser fetched and passed to the JS engine. Are you sure that you are executing the script with the expected changes, and your server isn't just returning a not-modified response?

Comment 7

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

I am not able to reproduce the original issue here as far as debugging apps goes with WebIDE.  It appears to work correctly: 

* Start debugging
* Edit JS
* Push new app version
* Debugger shows updated source

So, Eddy's issue seems separate in some way from apps, but I am not sure what the difference is.

Comment 8

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Eddy, can you adjust the summary to reflect your situation in comment 4, or perhaps we should close this bug and file a new one for it?

I don't believe there is a bug here about apps at the moment, so the summary is confusing.

Comment 12

[Jim Michaels]

sources don't update after ftping the file to the server and an F5 either. for a while it took 2 F5's to refresh it, now it just doesn't do it at all. debugger completely hosed.

Comment 13

[Jim Michaels]

and, I am looking at some sort of old cached code somehow... the line numbers it is giving don't even closely match what I have now. what's in debugger shows the debug line as being much farther down than my newer code, yet it sees script code in that place.

Comment 14

[James Long (:jlongster)]

Jim, can you confirm that this still happens in latest nightly? If so, can you post a test case?

Comment 15

[Eddy Bruel [:ejpbruel]]

I ran into this problem again when testing worker debugging. I'm not sure yet what the exact STR are, but it seems to happen when I debug files over a connection with Python's simple HTTP server.

This is a really annoying problem. I intend to get back to it when I get back from my vacation.

Comment 16

[Geoffrey D.]

I came across a similar situation on FF 42.0

I'm not doing anything fancy: no web worker, no simulator, just plain html/js. I'm not even editing the JS client side.

I've been having this weird behavior where the debugger won't reflect the changes I just made server side (F5, CTRL+F5, as well as closing and reopening the dev tools won't change anything), however the page is executing the new version of the code. Note that the "Disable Cache (when toolbox is open)" option is enabled.

The weird thing is that if disable (untick) "Disable Cache (when toolbox is open)" the changes will be loaded and visible in the debugger (seems counter-intuitive and to work backwards).

Comment 17

[David D Lowe]

I still experience this bug.

Comment 18

[Bryan Clark (DevTools PM) [:clarkbw]]

Is there a simplified test case for this?

Comment 19

[Serhiy]

Any workaround for this?
In my case it's Firefox 55.0.1, Django dev server (manage.py runserver) and JS code embedded into the page. While it's updated if viewed through Ctrl+U it's still old in the JS debugger.

Comment 20

[Serhiy]

But code being executed is new, so you can blindly set breakpoints by line number.

Comment 21

[Edouard PELLERIN]

I can attest that the comment from Goeffrey D. (16) is correct, enabling caching (by un-checking "Disable HTTP cache") works, the debugger tab has up to date sources ...

Comment 22

[Jim Michaels]

ok, but what I have seen in the past is that a bird flying by or just about any event like an upgrade will cause this setting to be lost.

Comment 24

[Jason Laster [:jlast]]

Closing this as I believe we fixed the root issue. Please let me know this is still reproducible.