135239 - Option not to save URLs during SaveAs Text

Status: NEW

(Core :: DOM: Serializers, enhancement, P5)

Description

[Felix Miata]

In Netscape 4.x, choosing to save the displayed web page as text results in a
file that contains no words or numbers (readable "text" content) from the source
page that the browser does not display. Mozilla adds into the resulting file
source content not displayed by the browser, cluttering the result file with
unexpected content.

If the developers think this unexpected additional content is a necessary
feature, then there should be either a UI preference to provide behavior
equivalent to Netscape 4.x, or a new save as type added to the menu.

Comment 1

[Felix Miata]

This bug filed as a result of bug 131166, essentially a continuation.

Comment 2

[Felix Miata]

s/contains no words or numbers (readable "text" content)/, exept for image
markers, contains no words or numbers (readable "text" content)/

Comment 3

[Felix Miata]

Attachment: Screenshot of more litter than text content (extensive relative URL's)

Comment 4

[Boris Zbarsky [:bzbarsky]]

confirming,  ccing tanu, who's likely to end up with this one

As long as the default mode includes urls, preferably as footnotes, another mode
is fine.

Comment 5

[Felix Miata]

Attachment: Source for attachment 77512 http://us.imdb.com/Name?Theron,+Charlize

Comment 6

[Felix Miata]

Attachment: Same source as for 77512 but saved by Netscape 4

Netscape file size=5844 bytes (cr/lf format)
Mozilla file size=9080 bytes (cr/lf format; 55.4% bigger)

Comment 7

[Matthew T (active 1999-2002)]

This looks like a regression of (or unfixed aspect to) bug 66035.

Comment 8

[roberts]

Is this one being worked?  I'd hate to see 1.0 released with the "Save as text"
feature in its current poorly-implemented state.

Comment 9

[Boris Zbarsky [:bzbarsky]]

This is very unlikely to make 1.0 even if a patch were posted right this minute.
 The guy it's assigned to has a few dozen crash/hang bugs on his plate he's
working on at the moment.  So no, it's not being worked on.

Comment 10

[Tanu Mutreja]

This is a dupe of bug #131166, which we decided as a feature than bug. But 
on highlighting comment #6 here and comment #11 & #16 in 131166, it looks 
sensible to add another options which may indicate to retain the URI. Probably 
in the next milestone...

Comment 11

[Felix Miata]

This is not a dupe of bug 131166. I filed them both. This was filed after the
other was WAD wontfixed. This one, with a different focus in the summary, is
seeking a cleanup of the mess (litter) the current design leaves.

Unlike BenB, I maintain that Netscape 4 got it right, that only what is
literally displayed in the browser window is appropriate for saving in a text
file. If you need links preserved, save as HTML remains available. Since BenB is
apparently the one in control, the only way to have both is to provide another
menu choice so Netscape 4 upgraders can continue saving files that include no
more than what the browser displays.

Not saving relative links is not data loss when those relative links were not
literally displayed in the browser window.

Comment 12

[roberts]

The fact that Mozilla developers refuse to recognize that what was once a useful
feature which  users had come to reley upon has now been rendered useless by
arbitrary changes implemented by the Mozilla development team. This points out a
large deficiency in the Mozilla User Requirements development process.  Users
came to rely upon Netscape 4's File->Save As->Txt feature.  Mozilla has modified
that feature to the point where it can no longer be used.

Comment 13

[Tanu Mutreja]

I support the fact that text format has nothing to do with URI's i.e. we can't 
expect any text editor or reader to get some useful information out of it. It's 
only HTML format which has some meaning for URI but as I'm not aware with the 
usability history which made us to come to this decision, it would be great to 
have some more inputs here. 
But as BenB mentioned in his comments for bug #131166 (comment #16), there may 
be places where this is actually required. For that either we may go for another 
flag or compare the behavior with other products in market like IE or we may 
decide on the basis of usability probability.
CC'ing to few people who may help in having a consensus.

Comment 14

[roberts]

In which case the Mozilla team should not have buggered up a feature that users
depended on, but instead they should have added a new feature.

Comment 15

[Ben Bucksch (:BenB)]

[e-mail], [e-mail], I find it ultra-mean to file a
"continuation" bug of another bug that has been long discussed and marked
WONTFIX by a developer and VERIFIED by a relevant Netscape manager, without
noting the new bug in the old bug and with excluding the opposing parties in the
new bug, in a new attempt to get your will through nevertheless. I may also
remind people of bug 135200, where you also didn't cc me. I find *your*
behaviour absolutely *unacceptable*.

Thanks Tanu for ccing me.

Unlike MrMazda's claim, this is a mostly dup of bug 131166. The only difference
seems to be that MrMazda seems now be willing to accept that not saving the URLs
may be an additional option, something that I proposed in bug 131166, but which
he religiously rejected there.
(If I am missing some other difference between the bugs, please correct me.)

Please clarify the summary of this bug.

The major issue with providing such an option is the UI. I filed bug 138568
about that.

Comment 16

[roberts]

I suppose it's not too surprising that the user requirements process it badly
broken, at least with respect to this particular feature/bug report, given the
attitude evidenced by some of the developers.  I guess when it comes to the
Freeware software development cycle, user requirements tend get lost in the noise.

In the real world, products that don't meet user requirements lose in the market
place.  In the free browser software world, I expect people will move to Opera.

Comment 17

[Felix Miata]

The summary doesn't have room for a screenshot. That's what the attachments are
for, which explain better than words what the problem is. The attachments here,
plus http://bugzilla.mozilla.org/attachment.cgi?id=76876&action=view speak for
themselves, and confirm the summary itself.

Netscape 4.x text file saves are much smaller than Mozilla's. In my little bit
of comparison, Mozilla's were 50% to 100% bigger than Netscape's.

Please excuse me for being ignorant, but I assumed BenB was the driver of DOM to
Text conversion, and would automatically be included, regardless of anything I
did or did not do in creating the new bug. I didn't create bug 135200, so I
don't know how I should have known I was supposed to CC BenB about that.

Note that this bug was confirmed new a mere 33 minutes after submission by
[e-mail] (as normal severity BTW), someone I presume, based upon the two
parent bugs' comments (bug 70045 & bug 131166), is of similar relative position
in the overall Mozilla development scheme as [e-mail], the bug 131166
wontfix verifier.

This bug was created to differ from bug 131166 in subtle ways, which are a
result of the comments made mostly by BenB in that bug, but also in particular
by brade's http://bugzilla.mozilla.org/show_bug.cgi?id=131166#c31 comment. BenB
insisted that the actual behavior was WAD. BenB insisted that the actual
behavior was the result of policy decision discussions that I was never able to
find a record of, much less understand. BenB insisted that no markup was being
left in the text file result. BenB suggested room for yet another save as option
to get rid of more HTML remnants. I was opposed to the need to have a feature
that provides 4xp equivalence but is not a default behavior, not the feature itself.

So, instead of complaining about markup, this bug focuses on the result. The
screenshots for 131166 and this demonstrate results that the Mozilla development
team should have little reason to be proud of. Netscape 4.x provided text
pseudo-WYSIWYG. Mozilla currently does not.

I still believe that not saving what the browser does not display does not
constitute data loss. Those who believe it does have the option to save as HTML
and preserve undisplayed portions of the page. Since BenB, the apparent DOM to
Text converter driver, disagrees, the only apparent option to keep Netscape 4
users happy upon "upgrading" to Mozilla is to add a UI option to select the
meaning of "save as text". To this even BenB may agree, as only today he filed
bug 138568.

Comment 18

[Akkana Peck]

The summary of this bug is just plain wrong.  If you're requesting that url
information not be included in plaintext output, fine.  But there is no html
source shown in your screen shot.  </Sections/Gallery> is not html source (or do
you know of a version of html I'm not familiar with, where this is a valid html
tag?)  It is a plaintext representation of a url, as was pointed out in the
other bug.

If you are requesting that Save As Text (1) not include url information, or 2)
not include relative url information, that would be a reasonable request.  (Or
Ben's suggestion, (3) offer two types of save as text).  Any of those would be
valid requests, and quite easy to implement in the code.  But filing a bug with
a nonsense summary, and complaining at length about mozilla doing something that
it just plain doesn't do, just misleads everyone involved in evaluating the bug,
and makes it less likely that you will ever see a fix.

I suggest that you clarify exactly what you're asking for (I'm guessing that
it's either (1) or (2) -- (2) solves the problem illustrated in your example)
and fix the summary.  

And BTW, using a screenshot to illustrate ascii text output is pretty silly.

Comment 19

[roberts]

I don't get it.  What is so hard to understand about the following request?

Make Mozilla's File->Save As->Txt behavior the same as it was in Netscape 4.x.

Many of us view having all that partial url garbage spread thoughout the text as
buggy behavior.  It breaks a feature we were using.

Comment 20

[Felix Miata]

Attachment: Screenshot

This is the actual browser URL display of the bottom portion of screenshot
http://bugzilla.mozilla.org/attachment.cgi?id=77512&action=view.
Note that near the bottom of both are "sorted by ratings" and "sorted by
votes". These two lines in the HTML display show a radio button at the
beginning, followed by 15 or 17 text characters. The screenshot of the text
display referred to in http://bugzilla.mozilla.org/show_bug.cgi?id=135239#c18
is showing that the same two lines in the output of the DOM to Text converter
contain 73-75 characters, including the "[ ]" that represents each radio
button. Also on these two lines in the DOM to Text output are multiple
instances like </RName?Theron,+Charlize>. You can call this whatever you want
(I think it may be a relative URL fragment), but as it is not anything the
browser displayed in the original, and does not textually depict anything the
browser displayed, I call it litter, hence the use of the word in the summary.
It may or may not have anything to do with HTML or URL or URI, but that doesn't
matter. Whatever it is, it doesn't belong in the DOM to Text output.

I could do a s/Source Code/Unidentified Stuff/ in the summary, but I really
don't see how that would help anyone. If someone could provide a better
description of </RName?Theron,+Charlize> or
</tiger_redirect?NAME_TOP&http://www.amazon.com/exec/obidos/ts/dvd-cast-list/078322737X/imdb-button>Lucy

Liu or the extensive amount of similar stuff in attachment 77514 [details], I'd be happy
to change the summary to better reflect whatever others think it is.

Comment 21

[Ben Bucksch (:BenB)]

I am not "in control" of the DOM To Text Conversion.

[URIs]
> Whatever it is, it doesn't belong in the DOM to Text output.

For the nth time: It is a relative URI/URL. You are back at your old pattern in
bug 131166, claiming that the current behaviour were a bug, and your way is the
only way, when it is not. This makes it a duplicate of the religious other bug,
and I am sick of it.

I think everyone agrees that the request for an *option* not to save URLs is
very reasonable. But I think that it should not be the default or should be a
clear option during the saving process (e.g. bug 138568).
What are you arguing about? When this bug is fixed, you have what you need.

I will change the summary from "[4xp] FileSaveAs .TXT Litters Resulting File
with Source Code Not Displayed by the Browser" to "Option not to save URLs
during SaveAs Text" and severity from "normal" to "enhancement" (bz told me that
confirming it as "normal" was not his intend). Please leave it that way, stop
arguing and wait until somebody implements it.

Comment 22

[Heikki Toivonen (remove -bugzilla when emailing directly)]

I am actually thinking about implementing this, because I think it is an
unfortunate situation that you can not Save As Text the same way 4.x and IE do.
So to make everyone happy there would be two save as text modes. 

Akkana, any pointers to what I need to do? What are all the extra things that we
save compared to 4.x?

And just out of curiosity, could someone point me to discussion/bugs/whatever
about the decision to implement save as text the way we do now? If I knew the
reasons for certain things maybe I can avoid some pitfalls as well.

Have a nice weekend everyone!

Comment 23

[Felix Miata]

I would like to know what value anything in the form </FName?...> or similar in
the attachments to this bug or bug 131166 has to the average Netscape or IE
upgrader. I can see very few in a form such as <a href="http://. . .">. The
rest, which are most, are nothing but a big bunch of relative asp URL fragments,
which I can't fathom being of any use to anyone or anything, besides the page's
creator or maintainer, except the asp server that generated the original page
code. What am I missing here? Why are not those such as
</FName?Theron,+Charlize#vhs> considered to be DOM to Text error, thus making
the current behavior a normal bug, and the original summary appropriate? Please
cite at least one real world example, and please point me to the discussions you
refered to in the previous bugs that define data loss as not saving what the
browser never displayed before the save.

Comment 24

[Ben Bucksch (:BenB)]

See bug 134457.

Comment 25

[Akkana Peck]

Heikki, here's the code that outputs the url:
http://lxr.mozilla.org/seamonkey/source/content/base/src/nsPlainTextSerializer.cpp#943

The thing to do if you want a mode that includes no urls at all would be to add
a serializer flag (in nsIDocumentEncoder.h, for example, see
nsIDocumentEncoder::OutputFormatted) and then make the clause in
nsPlaintextSerializer conditional on the flag.

Alternately, rather than adding a flag, another option might be to insert code
into that serializer clause which checks the url string for a scheme (probably
make an nsIURI and ask it what the scheme is?) and if there's no scheme, don't
output the url.  It's hard to see URLs without a scheme as being useful in
plaintext output, since the plaintext output doesn't include the base url so
there's not enough information to use the relative link anyway.  Ben, would that
be okay?

Comment 26

[Ben Bucksch (:BenB)]

I agree that the flag is the right way to go.

Relative URLs are bug 134457. If we convert a HTML page to plaintext, the
converter does have a baseURL and can turn them into absolute ones using Necko.

I can see that *most* relative URLs are of little use (as the example
attachments demonstrate), but there are cases where they carry a significant
meaning. For example, a bigger news/magazine site links to older
stories/articles (examples are obvious, so I'll save us them). It's basically
the same discussion like for including absolute URLs, just with different
statistics.
I have no final opinion on that, but I am leaning towards: If we have 2 modes,
one with URLs and one without, why not do what we say and output all URLs (maybe
even image URLs?) in one case and none in the other?
In most cases, the "litter" comes from the top, left, right, bottom link bars
that commercial sites are so obsessed with, and that can easily be edited out of
the resulting plaintext document, leaving only the real text with the embedded,
meaningful links. But I am drifting....
I can see the argument for a "rule-of-thumb", though, and "do what makes sense"
in /most/ cases. ("Do what I mean", but with all the associated risks.)

Comment 27

[Ben Bucksch (:BenB)]

*** Bug 120889 has been marked as a duplicate of this bug. ***

Comment 28

[Ben Bucksch (:BenB)]

For entertainment (or not), see bug 146951, which is supposed to be a dup.

Comment 29

[greg]

I'd like to comment that the above quote is key:

Make Mozilla's File->Save As->Txt behavior the same as it was in Netscape 4.x.

Outputting link information is only part of the problem.  Netscape 4.x
has significantly better formatting (actually readable) as compared to
Mozilla.  This issue is now the *only* remaining reason I keep Netscape 
around.  For a really good example, save www.cnn.com in both browsers.
From Netscape it actually looks like the rendered page!  From Mozilla
you just get a mess.

Comment 30

[roberts]

I agree with you, [e-mail].  In fact, I'm one of the people who
reported this as a bug prior to the release of 1.0.  Unfortunately, one of the
Mozilla developers (I can't remember his name) stubbornly refuses to acknowledge
the garbaged-up output you get with Save As -> txt as a bug.  Instead, he
insists that the url remnants littering the resultant .txt saved page are essential.

--Doug

Comment 31

[Felix Miata]

I believe you are referring to Ben Bucksch, who thinks not saving partial URL's
amounts to data loss:  http://bugzilla.mozilla.org/show_bug.cgi?id=131166#c24

Comment 32

[greg]

That's fine, but if that is the case then this is actually a feature
request for a "Render as Text" option in Mozilla that essentially
does what the feature in Netscape called "Save as Text" does.

Greg

Comment 33

[roberts]

Yes, that "extra save as text" feature was requested along the way during the
pre 1.0 flurry of bug reports, but nothing ever happened.

[e-mail]:  Yeah, that's him.

--Doug

Comment 34

[Felix Miata]

Note the severity of this bug is "enhancement", with keyword "4xp". This bug is
because of WONTFIXED (WAD) non-enhancement bug 131166. The original summary line
was changed from my original rendition : "FileSaveAs .TXT Litters Resulting File
with Source Code Not Displayed by the Browser", apparently also by or by request
of Ben Bucksch at comment 15 and following.

Comment 35

[roberts]

Well, back when this bug was originally reported, we were firmly in the mode of
the developers telling us users that we didn't know what features we needed.  I
won't be surprised to see this attitude arise again, as our friend "Ben" is
still on distribution for this bug.

--Doug 

Comment 36

[Heikki Toivonen (remove -bugzilla when emailing directly)]

This horse has been beaten to death before. We know what to do. Nobody has had
the time to do it. Move the complaining to newsgroups or something, or come and
fix this yourself. Bugzilla is not the correct place for this kind of discussion.

Comment 37

[roberts]

We are complaining about the attitude of the developers, and their refusal to
acknowledge valid bug reports.  If you want us to go away, no problem:  I've got
plenty to do in the real world.

Comment 38

[Dan Tobias]

I pointed out the problems with Mozilla's "save as text" feature way 
back in Bug 120889 -- which has been marked as a duplicate of this 
one even though mine actually predated this.  There, I showed 
examples of the text output from Mozilla, Netscape 4, MSIE, and Lynx, 
and concluded that, although each had its pluses and minuses, NS4 
generally produced the cleanest output with the best rendition of the 
site's layout.  I thus wished that Mozilla would try to pick up the 
best features of each of these and put them together into a high-
quality text rendering that would be mostly, but not exactly, similar 
to that of NS4.  If link URLs are to be shown (and that might be 
useful sometimes), they should be added to the end as references as 
Lynx does.

Comment 39

[Ben Bucksch (:BenB)]

> link URLs [...] should be added to the end as references as Lynx does.

Bug 46990

Comment 40

[Ben Bucksch (:BenB)]

*** Bug 193406 has been marked as a duplicate of this bug. ***

Comment 41

[Felix Miata]

*** Bug 212547 has been marked as a duplicate of this bug. ***

Comment 42

[Ben Bucksch (:BenB)]

smontagu just reminded me that I implemented a hidden pref
converter.html2txt.structs to turn *bold* etc. off. This bug could be
implemented similarily, but then wouldn't have UI access and couldn't be chosen
per document, only per profile.

Comment 43

[rsx11m]

Note that this also affects HTML/plain-text conversions in e-mails, leading to an unexpected display, e.g., of links in plain-text replies to an HTML message (bug 565949). Thus, I think tying this to converter.html2txt.structs or another pref should be helpful. The following may be a good point to start:

http://hg.mozilla.org/mozilla-central/file/3054ada929a2/content/base/src/nsPlainTextSerializer.cpp#l1092

>   else if (type == eHTMLTag_text) {
>     /* Check, if we are in a link (symbolized with mURL containing the URL)
>        and the text is equal to the URL. In that case we don't want to output
>        the URL twice so we scrap the text in mURL. */
>     if (!mURL.IsEmpty() && mURL.Equals(aText)) {
>       mURL.Truncate();
>     }
>     Write(aText);
>   }

Adding a check on the pref into the second if() statement, thus applying the mURL.Truncate() if it is not set, would omit adding the URL in all cases.

Comment 44

[Jens Stutte [:jstutte]]

Bulk-downgrade of unassigned, >=5 years untouched DOM/Storage bugs' priority.

If you have reason to believe this is wrong (especially for the severity), please write a comment and ni :jstutte.