48845 - Conn: [RFE] Offline browsing engine

Status: RESOLVED FIXED

(Core :: Networking: Cache, enhancement, P3)

Description

[miesgram]

You should integrate a real offline browsing engine into the new Mozilla 5.0 if 
you want to kick big billys IE5.5 ou of business. That's one thing I know many 
people miss in Netscape Communicator.

Comment 1

[Sebastian Späth]

What kind off offline browsing engine do you imagine? If it is a kinf of thing
like bug 18213 proposes, I suggest marking this a dupe of it.
(adapting summary to match request. NEW FEATURE isn't really intuitive :)

Comment 2

[Doron Rosenberg (IBM)]

imho, offline browsing is kinad useless, and the report is not very informative.
 Probably dupe of 18213

Comment 3

[Joseph Elwell]

This isn't the same as 18213. bug 18213, basically says, "if you're already
browsing offline, when you click on a link that you don't have cached, popup a
dialog and do stuff".

This bug, I believe is more of a, I want the ability to browse through my cache
as seemlessly as browsing the internet.

Example:
If I have most of www.mozilla.org in my cache - and I'm offline - when I type in
www.mozilla.org it should pull from my cache, and I should be able to click on
links and read info that is in my cache without having to wade through the cache
files. A front end for displaying the cache, and allowing access to it - Like
History - but offline.

the last time I used the history menu, it was because I was on my laptop in the
airport, and I couldn't view any of the pages in my history - despite the fact
they lived in my cache. I had to open up my cache folders and manually view
every page, hoping I would find the right one. (often viewing fat.db to try to
expediate the process).

Comment 4

[David Krause]

Resolving as dup per Asa.

*** This bug has been marked as a duplicate of 18213 ***

Comment 5

[miesgram]

I think Joseph Elwell is the first one who really understands me. It is like he
has said and that's why I think you should think about it again.

Comment 6

[Asa Dotzler [:asa]]

OK, reopening.

Comment 7

[Asa Dotzler [:asa]]

reassign. addign helpwanted and targeting to Future.

Comment 8

[Doron Rosenberg (IBM)]

why is this in xpapps GUI?

--> networking : cache

Comment 9

[Doron Rosenberg (IBM)]

*** Bug 52935 has been marked as a duplicate of this bug. ***

Comment 10

[R.K.Aa.]

This seems to be a dup of bug 11632: [RFE] Save Page With Images, Stylesheets,
Objects, Applets"

See comment from Sean Richardson there, where the comparision with how IE
handles the matter is sketched. In the 6 other duplicates there IE is also often
compared to.

Comment 11

[Denis]

I would like a simple way to convert an absolute URL to a relative URL while 
browsing documents. It would be useful specially when browsing a set of 
documents saved in the same directory on my disk.
I was thinking of a kind of contextual menu to do the change.

Enhanced options would include "convert double-click URL to standard URL" or so

Comment 12

[neeti]

Cache bugs to Gordon

Comment 13

[Alex Bishop]

Try the following with 0.9.1:

1. Go to http://www.mozilla.org/
2. Follow the 'At a Glance' link
3. Go to http://bugzilla.mozilla.org/
4. Click on 'View Bugs Already Reported Today' and wait for the page to appear
5. Close all Mozilla windows
6. Disconnect from the Internet if possible
7. Restart Mozilla
8. Go into offline mode

Here comes the good bit...

9. Type http://www.mozilla.org/ into the location bar and hit Enter
10. Be amazed as the page appears!
11. Click on 'At a Glance'
12. Be similarly amazed as the page appears!
13. Open the History tab in My Sidebar, expand Today > bugzilla.mozilla.org and
open 'mozilla.org Bugzilla -- the Mozilla bug database'
14. If the novelty still hasn't worn off, be amazed again!
15. Follow the 'View Bugs Already Reported Today' link
16. Be amazed that even a cached CGI page appears!
17. Take an aspirin and lie down for a while to recover from all that amazement

It seems that this bug has been at least partially fixed, probably by a new
cache checkin. I haven't tested this extensively, but it seems to work for most
pages.

Comment 14

[gordon]

Can we mark this bug fixed?  The "engine" seems to work. Though there may still 
be bugs preventing full use of this feature, we could handle those in separate 
reports.

Comment 15

[Keyser Sose]

*** Bug 87719 has been marked as a duplicate of this bug. ***

Comment 16

[basic]

Reporter: Do you consider this fixed?

Comment 17

[Malx]

Yes. It is fixed.

Comment 18

[Alex Bishop]

In that case I'll mark it FIXED. In the next few days, I'll create a list of
problems with the offline browsing engine in its current state and post it here.
Then the problems can be filed as new bugs and a metabug created to track them.

Comment 19

[benc]

Alex:
sounds great!

Comment 20

[Alex Bishop]

I'm still in the process of compiling a list of problems with the current
implementation. I'm also examining IE's behaviour for comparison (its offline
browsing is very impressive). I hope to complete all this by about Thursday.

I came across bug 88575 which suggests that offline browsing may only work from
the disk cache, not the memory cache. Which is a bit of a shame.

BTW - what does "Conn:" in the summary mean?

Comment 21

[Malx]

Look at this URL
http://www.saslib.ru/ref/ekon/arc6/VDV-0322.txt
I browse it and leave browser for an hour.

Then I came back and press ALT+S

It starts do downloads!!!! (but I see this file on screen and even can't save it!)

I press "Offline" and got:

Error: "There was an error reading from source location"

// Even if there expiration header is set - I really want it to be in cache at
leas as long as it is on my screen :)

Comment 22

[Alex Bishop]

OK, here's a list of problems I have found with the offline browsing engine.
Bear in mind that I haven't done much extensive testing. All these tests were
conducted using Mozilla 0.9.2 on Windows 98SE. The version of IE used in the
comparative tests was 5.00.2614.3500 (also Windows 98SE).

1. Redirects don't seem to work

Go to http://uk.yahoo.com/ and click on "Sport". Note that the link points to
http://uk.yahoo.com/r/ei, but you are redirected to
http://uk.dir.yahoo.com/Recreation/Sport/. If you go offline and go back to
Yahoo!, clicking on the link doesn't work. This works fine in IE.
2. Google space strangeness

I decided to see how the offline browsing engine handles pages accessed through
forms. I went to http://www.google.co.uk/ (I didn't use Yahoo! because its
search form seems to use redirects) and searched for "noodles". I let the
results page load and went back to Google UK's home page. This time I searched
for "soy sauce". After the results page loaded I went back to Google UK and
changed to offline mode. I repeated the search for "noodles" and the results
page appeared fine. I then tried to search for "soy sauce" but this didn't work.
I went back online, tried searching for "chopstick suppliers", went offline and
tried to repeat this search. Again, it didn't work. I guess it must be something
to do with the spaces. This will probably require further investigation with
different sites. IE seems to handled cached Google searches with spaces fine.

3. Offline browsing does not work with memory cache

I was compelled to try this after seeing bug 88575. Go to Edit > Preferences >
Advanced > Cache. Clear your disk cache and set its size to 0. Go to
http://www.mozilla.org/. Go somewhere else and then go offline. Type in
http://www.mozilla.org/ into the location bar and hit Enter (or press Go).
Nothing happens. Which leads me onto the final problem...

4. No indication that a page isn't available offline.

If you try and access a non-cached page, nothing happens. This occurs whether an
address is typed into the location bar or a link is followed. IE displays a
dialogue (I'll attach a screenshot). Another thing IE does is use an alternative
cursor when you hover above a link to a non-cached resource. Rather than the
usual hand, you get a hand and circle with line through it (screenshot coming).
This is a very nice touch.

They're all the problems I found. Like I said, there was no extensive testing
done, so if anyone else has noticed any problems, post them here.

Here's the good news. These are things that I tested and expected not to work
but did:

1. Saving pages when in offline mode

Incidentally, I tried going to the page that [e-mail] said he
couldn't save. I didn't wait an hour (I'm a busy man!), but after going offline
the page seemed to save fine. If it's set to expire, it must be set by the
server because it's a text file and they can't be set to expire with meta tags
in the same way HTML files can.

2. Viewing source of cached pages

Works fine.

3. Files accessed with helper applications can be browsed when offline.

Go to http://www.damowmow.com/. Go to the Pictures section and click the video
link (second one down). Choose to open it with the default application as set by
the system (Windows Media Player in my case). The file will download (it's
1.6MB) and open in the media player. Watch it, it's funny. Go offline and repeat
the exercise. The video will load as if you were online (but with no waiting
while it downloads).

4. Still retrieves files if set to compare cached copies with network copies
every time

Go to Edit > Preferences > Advanced > Cache and choose "Every time I view the
page". Go to some pages and go offline. Try going to the pages again. Everythimg
works.

One thing I didn't check is if Mozilla will still access files from the cache
that are past their expiry dates.
Bugs need to be filed on the four problems identified (though number 2 needs
more investigation first). Am I correct in believing that all these bugs should
be filed with Product: "Browser" and Component: "Networking: Cache"? Number 4 is
really a UI issue, but I imagine it would involve some cache code as the cursor
change would require identifying if a resource is cached on mouseover rather
than when trying to access the page.

Questons? Comments? Other problems?

Comment 23

[Alex Bishop]

Attachment: Dialogue IE displays when trying to access a non-cached resource when offline

Comment 24

[Alex Bishop]

Attachment: IE - Oh no you don't!

Comment 25

[Malx]

thouse links, which are in cache. So you need not move mouse to see this (but pointer with 0 also good idea)

my url:
HTTP/1.0 200 OK
Date: Wed, 11 Jul 2001 22:26:57 GMT
Server: Apache/1.3.19 (Unix) PHP/4.0.2 rus/PL30.4
Last-Modified: Thu, 12 Apr 2001 06:13:27 GMT
ETag: "23826-3216d-3ad54787-koi8-r"
Content-Length: 205165
Content-Type: text/plain; charset=koi8-r
Expires: Thu, 01 Jan 1970 00:00:01 GMT
X-Cache: HIT from www-cache.citiustele.com
Connection: close

Note 1) Server with string "rus" in it.
2) Expires tag.

It is worst (IMHO) thing they do to russian pages :(
It dials with charsets (koi8-r || windows-1251 - both for Russian, but koi8-r - for unix and wind* - for MS win, also it's own for MAC). It looks at your "Agent" and gives you page translated to codepage acording to your OS. And it sets "Expire" so WWW-Proxy will not cache it, so IE will not get koi8 page from squid. :(
They call this - easy solution.
More than 50% of Russian and Ukrainian servers uses this technology :((((

But what we have - IE - works OK (it just ignores Expire) - so they say It's error of NN/M

You can't View_Source/Print/Back/Edit page/Save page without reloading page.
For Mozilla it's better - after an hour it ask you to reload page, but just after loading - all ok (in Netscape4 - no).

I do not know - could I ask for this Hack - to track for "Server: .* rus/.*" and specially for it ignore Expires: Thu, 01 Jan 1970 00:00:01 GMT (if Expires is less then current time).

For me - It is error. But in standart - you must through this page out befor you show it ;)

Comment 26

[timeless]

Instead of reopening bugs, could you please file new bugs?! reincarnating bugs 
is a good way to guarantee bugs that end up as zombies, forever unable to reach 
a satisfactory resolution.

Comment 27

[Alex Bishop]

Sorry about the delay.

I'm marking this bug RESOLVED FIXED again. [e-mail] please
file your problem as a new bug.

I've been experimenting with the problem where Google search results are not
cached if there's a space in the search query. I tried to reproduce this at some
other major search sites (AltaVista, Lycos etc.) but ran into problems caused by
the fact that the offline browsing engine can't handle redirects (I think). I
tried reproducing the problem at Teoma (a new search engine located at
http://www.teoma.com/ - try it, it's good!). I searched for "noodles" and
"chopstick suppliers" then went offline and repeated the searches. They both
worked leading me to believe it's probably a Google-specific problem.

I discovered the existence of bug 61689, a metabug for offline-specific bugs.
I'm guessing that all the new offline browsing engine bugs can be marked
dependent on that, rather than filing a new tracking bug.

To review:

1. Redirects don't seem to work

This needs to be filed as a new bug and marked as blocking bug 61689. I think
this needs to be filed in Browser/Networking: Cache. Can someone confirm this?

2. Google space strangeness

This appears to only affect Google and can be left alone for now.

3. Offline browsing does not work with memory cache

Bug 88575 can be expanded to include this problem (at the moment it just deals
with using back and forward buttons). The part of the bug dealing with the fact
that there is no notification that a resource is not available offline can be
moved to bug 45421 (see below).

4. No indication that a page isn't available offline.

This was actually filed over a year ago as bug 45421. MPT has suggested that an
error page (not a dialogue) appears in cases like this and I agree (see also bug
28586). No mention has been made of changing the cursor when hovering above a
link to a resource that is unavailable offline.

Therefore, my plans are:

* File a new bug about problem 1.
* Resummarise bug 88575 (for problem 3).
* Bring up the cursor-change issue for in bug 45421 (for problem 4).
* Mark the problem 1 bug and bug 88575 as blocking bug 61689 (and bug 61689 as
depending on them).

Does this all sound reasonable to everyone? If you've got anything else to add,
please do so now.

Comment 28

[benc]

Can someone write release notes for this bug?

Comment 29

[gordon]

Ben, what do you want in the release notes?  This is part of a request for a
future enhancement...

Comment 30

[Alex Bishop]

Not a future enhancement. A currently implemented enhancement (that's why this
bug is RESOLVED FIXED). It just needs a few tweaks (could someone please tell me
if the action plan for getting said tweaks filed in bugs sounds reasonable?).

Comment 31

[gordon]

Yes, I meant originally it was submitted as a future enhancement request.  This
part of the enhancement is complete (thus the bug is closed), but there are
still open bugs outstanding (tweaks) which prevent the complete manifestation of
the feature from a user's perspective.

re:action plan, yes, please file bugs on any aspects that you find are still
lacking.

Comment 32

[Alex Bishop]

Okay, I've done as described above. The redirect problem has been filed as bug
94004.

Comment 33

[benc]

gordon:
The idea was to get a paragraph that described the general implemented and
unimplemented behaviors. It seemed from the bug reports and the problems that
some behaviors should be described.