Closed Bug 388195 Opened 17 years ago Closed 14 years ago

Remove gopher protocol support for Firefox

Categories

(Core :: Networking, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla2.0b1

People

(Reporter: robert.strong.bugs, Assigned: joe)

References

()

Details

(Keywords: dev-doc-complete, relnote, user-doc-needed)

Attachments

(1 file, 8 obsolete files)

This is to lessen attack vectors by removing gopher protocol support.

btw: Bug 388192 is to remove gopher OS integration support
Component: General → Networking
Product: Firefox → Core
QA Contact: general → networking
Summary: Consider removing gopher protocol support → Consider removing gopher protocol support for Firefox
Nominating for 1.9 since cutting code is good for security, size, and other things.
Flags: blocking1.9?
I wouldn't block on this, but I do want it.
Flags: blocking1.9? → blocking1.9-
Whiteboard: [wanted-1.9]
Attached patch Base patch, rev. 1 (obsolete) — Splinter Review
Attached patch mail/ and mailnews/ (obsolete) — Splinter Review
Attached patch suite/ (obsolete) — Splinter Review
Attached patch camino/ (obsolete) — Splinter Review
Lessen attack vendors by dropping a protocol. How dare ye! As I said in bug #351748: why remove a working protocol that's still widely used and offering tons of data, and by the way also is indexed into Google?!

For Bob's sake, don't remove the Gopher I say!
Sorry for bothering you with that but as a simple end user, I am strongly against the removal of the Gopher protocol. There was a related debate in Bug 351748 recently, you can read several arguments in favour of keeping this protocol in Mozilla based browser there. Let me quote some of my remarks as this issue seems rather important to me:

Here are some comments from current gopher sites, written by experimented
gophermasters: gopher://quix.us/0/firefox : 
(...) Firefox 2.0.x still works just as 1.5.x did! Hurray for the Open Source
movement! Interesting, Microsoft has totally pulled ancient support for Gopher
out of Internet Explorer 7. It was only disabled in IE 6, but now it appears to
be gone completely.  Another good reason to dump IE.
== The Newest and Best Gopher Client: Mozilla Firefox! == (...)

gopher://gopher.floodgap.com/0/gopher/wbgopher :
(...) Most of you, alas, will find text to be a stifling environment even for
gopher. Fortunately, the Mozilla derivatives have significantly improved their
gopher support to the point where they are now THE CLIENT OF CHOICE if you want
a graphical system(...) Again, let me stress that with their current support,
for most users, Firefox 1.5.0.1 (and up), or Camino 1.0 (and up) for Mac OS X,
ARE THE CLIENT(S) OF CHOICE (...)

http://www.dmoz.org/Computers/Internet/Gopher/faq.html#1 :
1 Q:    I just can't see some pages of this section. Why?
A:      You're probably using a browser that doesn't support the gopher
protocol. For instance, Internet Explorer, Safari and Opera won't work for
Gopher sites. Get Mozilla Firefox or SeaMonkey and see the difference.
(disclaimer: I am the current editor of the Gopher category of the Open
Directory Project (dmoz.org), this one was written by me and reflects
only my personal views).

To put it bluntly, today Firefox and other Mozilla derivatives are praised as
very good Gopher clients in the Gopher community. Moreover, Firefox is the only widespread, cross-platform, open source graphical therefore easy to use client available today. The Removal of the Gopher protocol would be a big loss for us, and IMHO, Mozilla would also lose something. I think fixing Bug 388192 would be a good compromise, enhancing security without removing a feature... I'd really love to see this bug WONTFIX'ed.
Changing the title to one that's less wishy-washy...

25, or even 100, sites is far from enough to continue supporting it.

Do note that it would be fairly simple to write an extension to provide support for the gopher protocol.  You would simply need to implement nsIProtocolHandler and have its newChannel handle the details of making the gopher connections and providing readable data to display.  For further prettiness you could install a stream converter to convert raw gopher information into something vaguely pretty and formatted, as currently occurs with gopher directory listings.

For further details on implementing an extension to support gopher, see the following URL:

http://www.nexgenmedia.net/docs/protocol/
Summary: Consider removing gopher protocol support for Firefox → Remove gopher protocol support for Firefox
> 25, or even 100, sites is far from enough to continue supporting it.

Of course it is, if those sites are being used. There are considerably fewer FTP sites (and fewer still anonymous FTP sites) than there used to be, but those sites are still trafficked and I don't see anyone here seriously proposing killing FTP. Gopher, on the other hand, continues to be regularly updated by its users, has an increasing number of search facilities, and a large amount of historical content available that is in some cases not easily available any other way.

Gopher integrates well within the Core and is handled well by it, and represents a subset of file and resources for which Mozilla Core is currently the best and widest implemented solution. The asserted rationale for killing gopher to prevent it from being a potential attack vector is silly and throws the baby out with the bathwater. Fix whatever asserted problem with the protocol handler there is, and move on.

As far as, "well, write your own extension," that would be equally silly if the code is still in the browser and still works.
(In reply to comment #10)
> Of course it is, if those sites are being used. There are considerably fewer
> FTP sites (and fewer still anonymous FTP sites) than there used to be, but
> those sites are still trafficked and I don't see anyone here seriously
> proposing killing FTP. Gopher, on the other hand, continues to be regularly
> updated by its users, has an increasing number of search facilities, and a
> large amount of historical content available that is in some cases not easily
> available any other way.
It was seriously proposed by someone several months ago and there are plenty of people that would advocate the removal of ftp if the number of sites using ftp were any where near the number of sites supporting gopher.

> Gopher integrates well within the Core and is handled well by it, and
> represents a subset of file and resources for which Mozilla Core is currently
> the best and widest implemented solution. The asserted rationale for killing
> gopher to prevent it from being a potential attack vector is silly and throws
> the baby out with the bathwater. Fix whatever asserted problem with the
> protocol handler there is, and move on.
The attacks that aren't known are the problem. Such as
http://larholm.com/2007/06/12/safari-for-windows-0day-exploit-in-2-hours/

> As far as, "well, write your own extension," that would be equally silly if the
> code is still in the browser and still works.
Not when the vast majority of users don't use it and leaving it leaves open an attack vector for those users.
(In reply to comment #11)
> It was seriously proposed by someone several months ago and there are plenty of
> people that would advocate the removal of ftp if the number of sites using ftp
> were any where near the number of sites supporting gopher.

... the number of said FTP sites being 'you don't know.' Gopher, at least, has a number of trackers out there patrolling live servers and generating a count, which I might add is increasing. It also so happens I'm one of those doing the statistics, so I can actually give you figures. If you're planning to stake a numerical supremacy of FTP sites out there as your prime rationale for taking gopher out, you'd better be prepared to back it up.

> The attacks that aren't known are the problem. Such as
> http://larholm.com/2007/06/12/safari-for-windows-0day-exploit-in-2-hours/

The argument 'we're more afraid of the unknown attacks' holds little or no water. You could say that about any of the other protocol handlers, or any other bug.

Gopher may be a niche environment, but I suspect most of the people here stumping for its removal from the Core have not thoroughly investigated exactly what modern Gopherspace continues to offer, and the number of people who do in fact use it. Being the Floodgap Gopher Proxy maintainer, I think I'm in a much better position to judge relative interest.
(In reply to comment #12)
> The argument 'we're more afraid of the unknown attacks' holds little or no
> water. You could say that about any of the other protocol handlers, or any
> other bug.
And if those protocols were not widely used, we'd consider to remove them as well for the same reasons.  Do you have any suggestions?

> Gopher may be a niche environment, but I suspect most of the people here
> stumping for its removal from the Core have not thoroughly investigated exactly
> what modern Gopherspace continues to offer, and the number of people who do in
> fact use it. Being the Floodgap Gopher Proxy maintainer, I think I'm in a much
> better position to judge relative interest.
It also makes you a biased judge, which isn't a good judge.
(In reply to comment #11)
> (In reply to comment #10)
> > Of course it is, if those sites are being used. There are considerably fewer
> > FTP sites (and fewer still anonymous FTP sites) than there used to be, but
> > those sites are still trafficked and I don't see anyone here seriously
> > proposing killing FTP (...)
> It was seriously proposed by someone several months ago and there are plenty of
> people that would advocate the removal of ftp if the number of sites using ftp
> were any where near the number of sites supporting gopher.

And when I think that I installed the FireFTP extension because I find the FTP
client of SeaMonkey a little weak... :-P

> The attacks that aren't known are the problem. Such as
> http://larholm.com/2007/06/12/safari-for-windows-0day-exploit-in-2-hours/

Wait a minute, in my understanding this is a problem with Safari rather than
with Firefox. I don't think it's a good idea to correct Safari's problems by
completely removing a working protocol in Mozilla. Moreover, note that the
author of the exploit says "When Apple released Safari for the Windows platform
they neglected to implement a proper level of input validation for these
arguments, which means that you can break out of the intended confines and
wreak havoc (...) It is important to know that, even though this PoC exploit
uses Firefox, the actual vulnerability is within the lack of input validation
for the command line arguments handed to the various URL protocol handlers on
your machine. As such, there are a lot of different attack vectors for this
vulnerability, I simply chose Firefox and the Gopher URL protocol because I was
familiar with these". Hence removing Gopher protocol support only addresses a
very little part of the issue (and does it badly, IMHO). It's up to Apple to
correct their security breaches, isn't it?
 
> > As far as, "well, write your own extension," that would be equally silly if the
> > code is still in the browser and still works.
> Not when the vast majority of users don't use it and leaving it leaves open an
> attack vector for those users.

Is this the only way to correct the problem? There must be another way around
that leaves a working and appreciated feature intact. Also, IMHO it would be a
pity if the recently corrected support was removed (Bug 118438); and frankly,
although I'm anything but a code wizard, in my understanding added complexity
because of the Gopher protocol is minimal (the protocol itself is pretty simple
& stable).
> And if those protocols were not widely used, we'd consider to remove them as
> well for the same reasons.  Do you have any suggestions?

See below.
 
> It also makes you a biased judge, which isn't a good judge.

I'll plead guilty to 'biased' but not to the latter. So far there has been a lot of handwaving of 'this protocol is unused/little-used' and not much information to back that up, and what I'm trying to point out is that there are those of us in the community who actually do some attempt at auditing use of the protocol and can actually provide numbers.

What greatly disturbs me is that in an effort to correct a -potential- bug, an entire protocol is simply going to be blotted out because of an unproven assertion that it is 'little used.' A good judge may not be biased, but they had certainly better be informed.
(In reply to comment #13)
> (In reply to comment #12)
> > The argument 'we're more afraid of the unknown attacks' holds little or no
> > water. You could say that about any of the other protocol handlers, or any
> > other bug.
> And if those protocols were not widely used, we'd consider to remove them as
> well for the same reasons.  Do you have any suggestions?

I'd say "leave them as they are as long as you don't encounter a serious problem since they're still useful for those who use them". As I tried to explain in my previous message, the problem we're talking about appears because of Safari, not because of Firefox and removing Gopher support doesn't really address the issue (if my understanding is correct). So, we have a protocol with some die hards still using it and it doesn't add much complexity -- how often does this stuff change? So, why remove it?
 
> > Gopher may be a niche environment, but I suspect most of the people here
> > stumping for its removal from the Core have not thoroughly investigated exactly
> > what modern Gopherspace continues to offer, and the number of people who do in
> > fact use it. Being the Floodgap Gopher Proxy maintainer, I think I'm in a much
> > better position to judge relative interest.
> It also makes you a biased judge, which isn't a good judge.

Well, either you use the protocol, either you don't. I mean, it's hard to be an unbiased judge in this case; but I'd consider the participation of a skilled gophermaster as a chance. Microsoft removed Gopher support without asking any advice and they were being mocked in the gopherspace for their "laziness". Here, we have a chance to discuss the issue and to try to find a compromise. Let's not miss this possibility.
(In reply to comment #12)
> (In reply to comment #11)
> > It was seriously proposed by someone several months ago and there are plenty of
> > people that would advocate the removal of ftp if the number of sites using ftp
> > were any where near the number of sites supporting gopher.
> 
> ... the number of said FTP sites being 'you don't know.' Gopher, at least, has
> a number of trackers out there patrolling live servers and generating a count,
> which I might add is increasing. It also so happens I'm one of those doing the
> statistics, so I can actually give you figures. If you're planning to stake a
> numerical supremacy of FTP sites out there as your prime rationale for taking
> gopher out, you'd better be prepared to back it up.
Just checking the forums after we removed OS integration on Windows and a headcount around the office provides a couple of data points of how often Firefox users use gopher. The fact that other browsers have removed gopher and not ftp is another data point. Also, reading the forums will provide data points on how often ftp is used in that people do post about ftp functionality regularly though admittedly no where near as often as about http(s). I never tried to get into a hard numbers discussion and I don't have the time to track ftp usage as you have tracked gopher usage. I also didn't state that numerical supremacy of ftp sites has anything to do with removing gopher support much less it being my prime rationale... I did state "there are plenty of people that would advocate the removal of ftp if the number of sites using ftp were any where near the number of sites supporting gopher." I also stated that removing gopher support removes another attack vector.

> > The attacks that aren't known are the problem. Such as
> > http://larholm.com/2007/06/12/safari-for-windows-0day-exploit-in-2-hours/
> 
> The argument 'we're more afraid of the unknown attacks' holds little or no
> water. You could say that about any of the other protocol handlers, or any
> other bug.
> 
> Gopher may be a niche environment, but I suspect most of the people here
> stumping for its removal from the Core have not thoroughly investigated exactly
> what modern Gopherspace continues to offer, and the number of people who do in
> fact use it. Being the Floodgap Gopher Proxy maintainer, I think I'm in a much
> better position to judge relative interest.
As for "Gopher may be a niche environment" I personally believe that statement is disingenuous... it is a niche environment. Also, I suspect the relative interest is in relation to the gopher sites you track.

(In reply to comment #14)
> > The attacks that aren't known are the problem. Such as
> > http://larholm.com/2007/06/12/safari-for-windows-0day-exploit-in-2-hours/
> 
> Wait a minute, in my understanding this is a problem with Safari rather than
> with Firefox. I don't think it's a good idea to correct Safari's problems by
> completely removing a working protocol in Mozilla.
What is better for the average Firefox user that has never used much less even knows about gopher?

btw: The desire to remove gopher to lessen attack vectors was made by several of the security people at Mozilla before 2.0.
(In reply to comment #17)
> Just checking the forums after we removed OS integration on Windows and a
> headcount around the office provides a couple of data points of how often
> Firefox users use gopher. The fact that other browsers have removed gopher and
> not ftp is another data point. Also, reading the forums will provide data
> points on how often ftp is used in that people do post about ftp functionality
> regularly though admittedly no where near as often as about http(s). I never
> tried to get into a hard numbers discussion and I don't have the time to track
> ftp usage as you have tracked gopher usage. I also didn't state that numerical
> supremacy of ftp sites has anything to do with removing gopher support much
> less it being my prime rationale... I did state "there are plenty of people
> that would advocate the removal of ftp if the number of sites using ftp were
> any where near the number of sites supporting gopher." I also stated that
> removing gopher support removes another attack vector.

But you *are* making the assertion that numbers count. You said that there would be plenty of support for FTP's removal if the numbers were anywhere near Gopher's. Obviously numbers and usage levels must mean something in the final computation.

Furthermore, you're simply citing anecdotes rather than trying to do some reasonable assessment. A straw poll of a specific population isn't a usage count, it's a straw poll of a specific population. Over here on this side of the fence is a similar poll, and none of us in the pro-Gopher camp are going to admit we're disinterested, but we're actually trying to get some sort of numerical usage to back up our claim. If you want to write that off as a niche population, fine, but that doesn't make the protocol 'little-used' a priori. FTP is a niche by this sort of analysis too.

The issue with other browsers reflects the state of Gopher support in other browsers, which rotted for some time and in many cases was broken from the beginning. No one used Gopher in those browsers because it sucked, and I've said as much in my own analyses of those browsers' support environments. Mozilla, on the other hand, continuously improved Gopher support over several iterations to the point where anyone with any casual interest or greater in the protocol was using it simply because it was the best generally-available option. So I don't think it's a foregone conclusion that numbers wholly explained other browsers abandoning it.
 
> As for "Gopher may be a niche environment" I personally believe that statement
> is disingenuous... it is a niche environment.

Fine. And?

> Also, I suspect the relative interest is in relation to the gopher sites you track.

No, it's not. The Public Proxy can access any gopher site, and I track statistics for the gopher sites people access through it. I think that's a more representative sample of general interest in gopher.
(In reply to comment #18)
> But you *are* making the assertion that numbers count.
Like I said, "I never tried to get into a hard numbers discussion and I don't have the time to track ftp usage as you have tracked gopher usage." Perhaps someone else will take this path.

> > As for "Gopher may be a niche environment" I personally believe that statement
> > is disingenuous... it is a niche environment.
> 
> Fine. And?
Nothing... just glad we can agree on this without having to come up with hard numbers to support it one way or the other.

> > Also, I suspect the relative interest is in relation to the gopher sites you track.
> 
> No, it's not. The Public Proxy can access any gopher site, and I track
> statistics for the gopher sites people access through it. I think that's a more
> representative sample of general interest in gopher.
All I am saying is that if it is gopher-centric it will be gopher-skewed.
(In reply to comment #19)
> All I am saying is that if it is gopher-centric it will be gopher-skewed.

Which means what, that people interested in gopher access gopher sites? That's not exactly a dazzling conclusion.

If you're saying the population is self-selecting, there's no way around that -- it can't track the *absence* of interest, and I don't think anyone's contesting that the relative usage is a small proportion of the user base, so complaining that it's gopher-centric accomplishes little. The point is to demonstrate that there *is* continued interest and use of the protocol.

(In reply to comment #12)
> The argument 'we're more afraid of the unknown attacks' holds little or no
> water. You could say that about any of the other protocol handlers, or any
> other bug.

No, you're confusing TCB minimization with random bug fear. Security depends on reducing the size of the code that runs at high privilege, which must be correct for the system to uphold security properties. In this light, we have a lot to get rid of still. Gopher has to be considered along with other code for protocols and content types that are rarely to hardly ever used on the web.

Having written this, I'll repeat one of my mantras: the web is too big to make easy generalizations about what is rare. One person's, or locale's, hardly ever used protocol or content format may be another's must-have. So we won't do this (remove gopher) lightly, if we do it. More in a bit.

/be
One possible approach: rewrite gopher's protocol handler in JS. Anyone?

/be
(In reply to comment #22)
> One possible approach: rewrite gopher's protocol handler in JS. Anyone?
at that point, wouldn't it just be best to package it as an add-on and let those who want it install it (heck, it could even be a mozilla.org add-on!)?
(In reply to comment #17)
> (In reply to comment #12)
> > (In reply to comment #11)

> (...) I did state "there are plenty of people
> that would advocate the removal of ftp if the number of sites using ftp were
> any where near the number of sites supporting gopher." I also stated that
> removing gopher support removes another attack vector.

Just let me remind that there are several arguments in favour of keeping Gopher intact in Bug 351748, and also that I tried to explain my point of view above, in comment #16 for example.

> > Gopher may be a niche environment, but I suspect most of the people here
> > stumping for its removal from the Core have not thoroughly investigated exactly
> > what modern Gopherspace continues to offer, and the number of people who do in
> > fact use it. Being the Floodgap Gopher Proxy maintainer, I think I'm in a much
> > better position to judge relative interest.
> As for "Gopher may be a niche environment" I personally believe that statement
> is disingenuous... it is a niche environment. 

Please... Nobody is telling that Gopher is as widely used as FTP. I don't think anyone is trying to play with words here.

> (In reply to comment #14)
> > > The attacks that aren't known are the problem. Such as
> > > http://larholm.com/2007/06/12/safari-for-windows-0day-exploit-in-2-hours/
> > 
> > Wait a minute, in my understanding this is a problem with Safari rather than
> > with Firefox. I don't think it's a good idea to correct Safari's problems by
> > completely removing a working protocol in Mozilla.
> What is better for the average Firefox user that has never used much less even
> knows about gopher?

It's probably better for the ordinary Firefox user not to install untested, insecure software (when I'm a newbie on something, I wait a little bit before jumping in). And it's certainly unfair for Mozilla maniacs to suffer from problems of other software that they didn't even bother to check out. 
Put differently, I definitely appreciate your efforts to protect Firefox users (and I'm certainly not the only one), but you can't do it by fighting issues of other software and certainly not by removing unique, working features (which make, for a few people, a difference between Mozilla and other browsers). It would be like fighting wind mills. Instead, focusing on Mozilla itself would be more efficient IMHO.

> btw: The desire to remove gopher to lessen attack vectors was made by several
> of the security people at Mozilla before 2.0.

And were their fears justified? I mean, have there been any serious security threats because of the Gopher support in *Firefox* or other Mozilla based software? Anyway, even if Gopher was removed, plenty of other vectors will remain, the example cited above wasn't specific to Gopher in any way. I understand the will to increase security, but in our case it seems like an attempt to kill a fly that is probably not out there with a hammer. 

PS: Sorry for my poor English, as you certainly noticed it's not my native tongue.
(In reply to comment #23)
> (In reply to comment #22)
> > One possible approach: rewrite gopher's protocol handler in JS. Anyone?
> at that point, wouldn't it just be best to package it as an add-on and let
> those who want it install it (heck, it could even be a mozilla.org add-on!)?

Not necessarily -- removing it and implementing it in a memory-safe language are independent. If we had the JS implementation today, we might not even be bothering to argue in this bug.

/be
(In reply to comment #22)
> One possible approach: rewrite gopher's protocol handler in JS. Anyone?

I'm not opposed to this in principle (as I presume this would address many, if not most, of the concerns over security), but my desire would still be to keep gopher "in Core." If this would be an adequate compromise ...
(In reply to comment #25)

> Not necessarily -- removing it and implementing it in a memory-safe language
> are independent. If we had the JS implementation today, we might not even be
> bothering to argue in this bug.

The idea seems more than interesting to me. May I dare a proposal: 
Let's file an enhancement bug for a JavaScript implementation which -crossing fingers- shouldn't be that hard to code (and maybe some people in the Gopher community would volunteer for it, there are people who wrote Gopher servers from scratch), and until this bug is fixed let's keep the Gopher protocol support untouched. Does this sound reasonable?
(In reply to comment #25)
> Not necessarily -- removing it and implementing it in a memory-safe language
> are independent. If we had the JS implementation today, we might not even be
> bothering to argue in this bug.
Removing gopher came up again due to Bug 384384 so implementing in a memory safe language wouldn't have helped. Bug 388192 removed gopher OS integration for Win32 (we never set ourselves as the default handler for gopher on Mac OS X and I still have to get more info regarding removal on Linux) which does prevent gopher being used as an attack vector in the way it was used in Bug 384384. Beyond that I defer to Window and Schrep since they requested the removal of gopher support.
(In reply to comment #27)
> ..... and until this bug is fixed let's keep the Gopher protocol
> support untouched. Does this sound reasonable?
Just to clarify: at this point this bug is for the version of the Mozilla code which will become Mozilla 1.9 (Firefox 3.0) and gopher support is very unlikely to be removed from 1.8.1.x (Firefox 2.0.0.x)

I presume that you don't care so much about what happens in the pre-release versions; are you suggesting that this be put on hold until Mozilla 2.0?  (Which is quite a ways off)
(In reply to comment #29)
> (In reply to comment #27)
> > ..... and until this bug is fixed let's keep the Gopher protocol
> > support untouched. Does this sound reasonable?
> Just to clarify: at this point this bug is for the version of the Mozilla code
> which will become Mozilla 1.9 (Firefox 3.0) and gopher support is very unlikely
> to be removed from 1.8.1.x (Firefox 2.0.0.x)

Indeed, this is what I thought when I read this "bug" file, especially the following line: Whiteboard: [wanted-1.9].

> I presume that you don't care so much about what happens in the pre-release
> versions; are you suggesting that this be put on hold until Mozilla 2.0? 
> (Which is quite a ways off)

If someone is able to write the JS implementation before Firefox 3 is released, very well, but this possibility seems unlikely to me (IIRC the beta is scheduled for September). So I think we'll need more time, probably the next milestone release of Firefox given the fact that minor versions only include security and stability fixes. 3.5 or 4, I don't know how it will be called and on which version of Gecko it will be based, but that's the idea. Since Bug 388192 is currently being handled and as we lived fine with at least partial Gopher support since years, I dare to hope that this delay won't be a problem in order to find a decent compromise.
First, I'll note a JS implementation is exactly what I proposed in comment 9, for what it's worth.

(In reply to comment #30)
> If someone is able to write the JS implementation before Firefox 3 is released,
> very well, but this possibility seems unlikely to me (IIRC the beta is
> scheduled for September).

The current amount of code for gopher is ~25K of C++.  If you convert that to JS, that size is going to go down due to reduced error-handling bloat (errors being converted to JS exceptions that can be handled in fewer places, versus error-checks of every return value in C++).  I'm told gopher uses a base channel implementation that shares code a bit, and adding in that code would make it a bit bigger (17K more for the code, 10K for the header for it), but I'd honestly be surprised if gopher-in-JS came to more than 50K total.  That little code is easily doable in even a month for someone who cares enough to make it happen, particularly since there's already an implementation from which you can cheat.  :-)
(In reply to comment #31)
> The current amount of code for gopher is ~25K of C++.  If you convert that to
> JS, that size is going to go down due to reduced error-handling bloat (errors
> being converted to JS exceptions that can be handled in fewer places, versus
> error-checks of every return value in C++).  I'm told gopher uses a base
> channel implementation that shares code a bit, and adding in that code would
> make it a bit bigger (17K more for the code, 10K for the header for it), but
> I'd honestly be surprised if gopher-in-JS came to more than 50K total.  That
> little code is easily doable in even a month for someone who cares enough to
> make it happen, particularly since there's already an implementation from which
> you can cheat.  :-)
> 

I don't know about Steg, but I'd be willing to put my keystrokes where my mouth is. However, I'm an admitted XPCOM novice and I can guarantee I wouldn't be able to get something release-worthy that fast since I'd be trying to learn in parallel. I'll step up for this but I'd need a little more time than what's currently being discussed.
(In reply to comment #32)
> I don't know about Steg, but I'd be willing to put my keystrokes where my mouth
> is. However, I'm an admitted XPCOM novice and I can guarantee I wouldn't be
> able to get something release-worthy that fast since I'd be trying to learn in
> parallel. I'll step up for this but I'd need a little more time than what's
> currently being discussed.
Writing it in JS should be really straightforward - especially with a C++ implementation sitting there for you.  At that point it would even be trivial to pull it out of core and make it an extension in which you can add more to it if you want.

As for help, #extdev on irc would probably be enough, and there are docs on devmo on how to write JS components.  The best part is that we have XPCOMUtils.jsm now that makes some of the nuisances go away.
(In reply to comment #33)
> At that point it would even be trivial
> to pull it out of core

I think the objections to that have already been made clear from this side.

In the meantime, I'm looking at the source.
I should note, the rewrite in JS won't be quite as trivial as people make it sound because the current gopher code uses helper classes that can only be used inside of libnecko. Starting from 1.8 code may be a better approach.
I use this protocol regularly. Most people using gopher these days are using firefox. I think this shouldn't be removed, even if its only for historical reasons. Preemptively removing features for fear of unknown security issues is nonsense. Whats 20k these days anyway?
FF is open source, and it needs to fight planed corporate obsolescence. Keep gopher.
Keep gopher!
Nothing more to say. It is probably more a political than a technical reason, thus we have to harass or beg. So please, keep gopher! It is sad to see open source people following the (bad) lead of microsoft.
Keep gopher. Don't be lazy about security like Microsoft. Keep Firefox the premiere gopher client.
Are we still doing this for 1.9?  It's wanted-1.9, has patches, but hasn't been reviewed...
I certainly hope not. Christian's problem as described was what initially hit me, and I haven't had a chance to start over from 1.8.
The idea of removing support for the gopher protocol due to some vague notion of the protocol being used as an "attack vector" is about as silly as saying the http protocol can be used as an attack vector, and suggesting the http protocol should no longer be supported.  I've been waiting for someone to demonstrate a proof-of-concept for such an attack, but have not seen one.

I'm not a Mozilla dev, but I am a dev on another OSS project (http://www.wikakwiki.org). The idea that a WikkaWiki feature be discontinued simply because it *might* be compromised is not a criteria that is used in our dev process.  Instead, we do our best to make the codebase as secure as possible to forestall such compromises, and do not lay awake at night fearing that we might have missed something. Because in the end, there's never going to be code that is 100% secure, so we strive for "as secure as possible."  Removing gopher support to prevent a future (and unknown attack) makes about as much sense as removing a car engine to prevent a future accident.

This seems like a non-issue.  It's certainly not a bug, and I'm not too sure why the issue continues to languish after 6 months.  
(In reply to comment #43)
> The idea of removing support for the gopher protocol due to some vague notion
> of the protocol being used as an "attack vector" is about as silly as saying
> the http protocol can be used as an attack vector, and suggesting the http
> protocol should no longer be supported.  I've been waiting for someone to
> demonstrate a proof-of-concept for such an attack, but have not seen one.
> 

There have been attacks:

http://www.microsoft.com/technet/security/Bulletin/MS02-027.mspx

Any place were content can be downloaded from the net is an attack source.  

The question is how many users are there of the protocol - according to wikipedia (http://en.wikipedia.org/wiki/Gopher_(protocol)):

"As of 2007, there are fewer than 100 gopher servers indexed by Veronica-2.[7] Many of them are owned by universities in various parts of the world. Most of them are neglected and rarely updated except for the ones run by enthusiasts of the protocol. A handful of new servers are set up every year by hobbyists - 30 have been set up and added to Floodgap's list since 1999[8] and possibly some more that haven't been added. Today Gopher exists as an almost forgotten corner of the internet - one can publish email addresses in plaintext without having to worry about spam, and publish large amounts of data without the risk of the server's bandwidth becoming saturated, while at the same time people do still browse the gopher servers regularly."

http://news.netcraft.com/archives/web_server_survey.html

Says there are at least 155 Million web servers and growing strong.

Besides security implications we are continually trying to keep code size down to prevent feature bloat.  Do we have to hit absolute 0 users of a feature before it is considered a candidate for removal for the next release? If a feature is used by .01% of users should we keep it?  .1%?  1%?  How many gopher users are there on firefox?

If a JS impl is difficult can we build a binary extension that implements it for those folks who want it.  

> There have been attacks: (IE link omitted)

We are talking about FF here, correct?  I don't see what a buffer overflow in IE has to do with FF...apples and oranges.

> The question is how many users are there of the protocol

Why is this the question?  Who deemed this question as being a valid criteria? And why does it bother you (and others) so?

> Do we have to hit absolute 0 users of a feature
before it is considered a candidate for removal for the next release?

That sounds like a good policy to me.

I've seen that the gopher implementation takes about 25K lines of C++ code.  Assuming this is true (and I have not verified this independently), then yes, I agree:  There is a severe case of code bloat.  Maybe the solution is some prudent refactoring of said code.  What if someone were to step forward to assist with this?  Would that have any bearing on the issue?
If there were a way to add a negative vote on bugzilla....
[stops daydream]
Keep gopher. Mozilla-based browsers are practically the only choice for a graphical gopher client that can also do the web.
I think since we have not landed the patches for Firefox 3 and we're approaching beta 3, we should not remove gopher from Firefox 3 at this point. Going by the (mostly unwritten) book, we should have removed it before beta 1 if we meant to.

But: in Mozilla 2, gopher *is* going to be removed early and only re-added via an addon. So fresh versions of this bug's patches, made against the hg.mozilla.org mozilla-central repository, should be attached, reviewed, and landed there. Who volunteers to take this bug? Benjamin, will you do it?

The ~212 people who care about gopher can then develop, own, and use that addon.

In the mean time, gopher fans: spare us more soap-boxing and special-pleading here, and be happy today. Gopher will be an option (not a mandatory part of the C/C++ "TCB") as long as people are motivated to own its implementation. But as we draw closer to Mozilla 2, if no one steps up and delivers the gopher addon, you have only yourselves to hold accountable. No more free lunches!

/be

P.S. I was going to attach an appropriate Simpsons clip, but youtube has taken down all the good ones -- see bug 332174 and read comments for context.
Target Milestone: --- → mozilla2.0
Flags: wanted1.9+
Whiteboard: [wanted-1.9]
Should I read that as saying that Gopher will remain in 1.9, but removed from core in 2.0?
Clearing wanted-1.9+ per comment 47.

Cameron: that's exactly what he's saying.  "in Mozilla 2, gopher *is* going to be removed early and only re-added via an addon".
Flags: wanted1.9+
Okay, then with the writing on the wall, we'll get cracking on the next step. This gives us a little time to work on an addon, and that *is* appreciated.
A pitty you decided on removing it. It will make me change browser from Firefox.
Note that if gopher is to be made an add-on, then you can't quite remove all of the core support for it, such as (for example) in nsHTTPIndex.
Brendan,

Thank you for making the right call. As I recall, long ago, I was the original person in Netscape that proposed: remove gopher if it was not going to be maintained, (mostly for the reasons that were discussed above).

(Before everyone else reposts what they already said, please read what I said very carefully. Also, in case anyone is really angry at me, this was many years ago. I've made repeated efforts to get someone to understand the point, and nobody would buy in, until now.)

On a separate note, I found the entire: gopher is growing argument interesting, because it seems to me that the important threshold is: is it growing faster than the overall growth of the internet?
Whatever. Taking gopher support out of firefox is a big shame. Gopher is a well written protocol. I've an idea, let's take out FTP and https, too!
If that is an important threshold for you, benc, then how about this:  Is the web growing faster than the overall growth of the Internet?  Is FTP growing faster?  I'm curious how responses to those questions would compare to a response about Gopher.  Arguments could be twisted in favor of any answer to that question.  I'm not sure any of them are relevant.

Agreed that if the Gopher code isn't going to be maintained enough to make it useful, that it should be removed.  I fail to see anything to suggest that the code isn't being maintained.  It's gotten better than in previous versions.  FF3's support is a fair improvement on FF2.

In any case, it takes very little effort to maintain code for a Gopher client.  Gopher clients from many years ago function perfectly well after years of ZERO maintenance; fancy that!  The Gopher protocol is very simple.  It is simpler than HTTP, HTTPS, FTP, SOCKS -- all of which are supported very well in Firefox.  If one finds it difficult to maintain Gopher code as compared to that for HTTP, HTTPS, FTP, SOCKS, ... I think something is seriously awry.

And regarding the lessening of attack vectors:  Has anyone bothered to enumerate even a preliminary list of potential attacks on Firefox' Gopher support?

I guess the real question is...  Is Firefox merely a "Web" browser, or a much more feature-rich Internet/browsing application?  If it's just a Web browser, why bother with protocols other than HTTP?
(In reply to comment #55)
> I guess the real question is...  Is Firefox merely a "Web" browser, or a much
> more feature-rich Internet/browsing application?  If it's just a Web browser,
> why bother with protocols other than HTTP?

In my opinion, Firefox should focus on being a browser to web sites, which includes downloading data from a web server and handling the (X)HTML/CSS/JavaScript/etc contents. HTTP and HTTPS is clearly needed for this.

Then the question is, should Firefox also have a support for Gopher and FTP. Sure, there can be links to Gopher and FTP sites in a web site. But so are for email, smb, sftp, etc. protocols, too. There are also many Gopher and FTP client apps that do their job much better than Firefox (like file uploads in FTP). So why not using these better apps for the other protocols instead of having the support in Firefox?

So in my opinion, Firefox should focus on being a superb web browser. Of course FTP and Gopher support is also nice to have, but currently they are blocking the use of better clients for these protocols.
In my experience, Firefox is simply the best easily usable gopher client right now, and I and other people I know use it a lot for that purpose.

There aren't many gopher clients at all. The original University of Minnesota client (in Debian as package "gopher") insists on gopher+ and is buggy for HTML links. Lynx works, but the display is slightly buggy and it caches everything by default.

There are simply no GUI clients on Windows (the last one is some Windows 3.1 dinosaur that doesn't really work in modern environments).

If gopher support is removed from Firefox, imho the protocol will just die off (as the last chance for any kind of "wide" adoption is removed). Please don't :)
(In reply to comment #1)
> Nominating for 1.9 since cutting code is good for security, size, and other
> things.
> 

Well we might as well cut out support for HTTP while we're at it.
That'll make it EVEN SMALLER. Now wouldn't that be great!
Forgive my sarcasm but seriously cutting out support for Gopher on the logic that it's not used by a lot of people is like saying we should all just use IE because that's what everyone else uses.
Firefox is my Gopher client of choice and I'd feel betrayed if it cut out support for Gopher especially when it's so well implemented (for a web browser that is).
The decision has already been made by Brendan Eich in comment 47: gopher support *will not* be removed from Firefox 3/Mozilla 1.9, but *will* be removed from Mozilla 2.

Please read the Bugzilla etiquette page (especially the 'No pointless comments' section): https://bugzilla.mozilla.org/page.cgi?id=etiquette.html
These are not pointless comments. They reflect the opinions of the userbase this software has. Please search for a document on "how to not be a dictator on bug trackers".

Furthermore, the decision that has been made can be revoked at any time which seems to be stressed more and more.
(In reply to comment #60)
> These are not pointless comments. They reflect the opinions of the userbase
> this software has. Please search for a document on "how to not be a dictator on
> bug trackers".
Clearly you didn't *read* the section and just took the title at face value.  Allow me to quote the relevant section for you so you are more likely to read it:
In bugs where there is a heated debate going on, you should be even more inclined not to add a comment. Unless you have something new to contribute, then the bug owner is aware of all the issues, and will make a judgement as to what to do. If you agree the bug should be fixed, vote for it. [snip]  Constructive and helpful thoughts unrelated to the topic of the bug should go in the appropriate newsgroup[1].

[1] http://www.mozilla.org/community/developer-forums.html
I read it, I just didn't really care for it.
With the upcoming Firefox 3 Gopher support is really beautiful now. On Linux the native folder widgets are used. Just beautiful: gopher://gopher.floodgap.com/ or gopher://gopher.floodgap.com/1/gopher/keepcore
I strongly protest the removal of Gopher support from Mozilla 2 and shoving it as an addon.
Now that Firefox 3 is out (congratulations), I would like to revisit what was discussed earlier w/r/t creating a JavaScript-based implementation.

While waiting for FF3 to crystallize, myself and several others in the community created an add-on as a migration path. I am planning to upload it to the official add-ons site, but those who are interested can get it from

http://gopher.floodgap.com/overbite/
gopher://gopher.floodgap.com/1/overbite/

This is more or less a JavaScript port of Darin's/Bradley's original code, with changes as appropriate for both Mozilla 1.8 and 1.9, and some additional enhancements requested by the community.

It was mentioned above that a JavaScript version would potentially solve (many of) the concerns about the security of the Gopher portion of Core. This is not ready for Core (it isn't house-style, and has some community-specific features that may not be appropriate for Fx or Mozilla), but I would be willing to create and maintain a JavaScript port of Gopher in Core if permission is granted to write one for Mozilla 2.0. As you can see, the add-on means most of the work is done already.

If the decision is still that Gopher's final presence in Core will be through EOL of Mozilla 1.9/Firefox 3, and removed from Core by 2.0/Firefox 4, I would like to have the add-on put into the release notes for Fx4 so that users who rely on the functionality and are not aware of #388195 will be able to go get it.
This Javascript port should definitely go into the core. Please let him port it and keep gopher support onboard. If Lynx can do it, Firefox should be able to do it as well. 

And many, many thanks to Cameron Kaiser for his offer to maintain this code!!!
Don't look for premature decisions, do the JS implementation.

I'm already in favor (comments above) of supporting gopher if it's implemented in a memory-safe programming language, has otherwise light-to-zero overhead, etc.

/be
Okay. I will start adapting the add-on back into something Core-suitable and will attach a first draft when finished. 
So, can someone familiar with Gopher in FF3 summarize what is in and what is out?

If I'm reading this right, in #47, Brendan said this it out for the milestone "mozilla 2.0", which is FF4? 

I can update the summary so the status is a bit clearer.
This bug is open so it can't have been resolved during the Firefox 3 development. I don't think anything is unclear here. Please stop adding noise comments (and do cite comments with "comment 47" not "#47").

/be
Blocks: 351748
Attached file First draft, Gopher in JavaScript (obsolete) —
First draft, Gopher in JavaScript (see revised necko.properties under separate cover). This contains the protocol object, channel object and menu->index object. It relies on the current version of netwerk/streamconv/converters/nsIndexedToHTML.cpp with the exceptions for gopher in order to properly format itself.
Attachment #333063 - Flags: review?
Attachment #333063 - Flags: review?
Attached file Revised necko.properties (obsolete) —
Revised necko.properties for the additional interface strings in the first draft.

I apologize if I have made any obvious mistakes; I'm still a relative novice to XPCOM and ask your patience.
Keep gopher. I hit these keyboards for 30 years. I want it to be there when I retire. Simple as that.
(In reply to comment #73)
> Keep gopher. I hit these keyboards for 30 years. I want it to be there when I
> retire. Simple as that.
> 

Gopher was created in 1991, so I seriously doubt that you're using it for 30 years.
(In reply to comment #74)
> Gopher was created in 1991, so I seriously doubt that you're using it for 30
> years.

Why do you doubt this? This is no comparison. I was created in 1984 and I'm still being used, too! :D 

I didn´t say I use it often. In fact I use it very rarely. But: it is a very lightweight protocol, and I like to have a fallback. Its code overhead for browsers is neglectable, and everybody who uses it knows about the pros and cons.
Keep it. I've just been getting back into gopher after many years absence, and having quite a lot of fun with it. Please keep the gopher support.
The importance of retaining the gopher protocol was highlighted to me today when I needed to get something from gopherspace from a Windows box where installation of . As Firefox was installed, the process was no more difficult than getting a file from http. Were Firefox not to support gopher, I would probably have had to logged into a remote box, used lynx to browser the gopherspace to get the file, and then
Opps. I realised as I was typing the above that I was repeating something mentioned earlier. However, I hit “commit” instead of cancelling. Apparently there is no trivial way to delete a comment on Bugzilla. I therefore apologise for the noise and the spam.
Flags: wanted1.9.2?
"There have been attacks:
http://www.microsoft.com/technet/security/Bulletin/MS02-027.mspx"
In fact, if you didn't notice, this was exactly why MS disabled Gopher support by default in the next IE cumulative security update, which made into IE 6 SP1 and IE 5.01 SP4:
http://www.microsoft.com/technet/security/bulletin/MS02-047.mspx
Later Gopher support was completely removed from IE 7.
We're not discussing IE.
Warning to gopher fans:

Bill Murray is on a bender again--this time it's the electrolysis project:

   https://wiki.mozilla.org/Content_Processes

We're splitting Firefox into a multiple process architectures, so that tabs live in separate processes, and all network traffic is funneled through the main ("chrome") process.   This means modifying all of our network protocols.

We do not have in-house resources to devote to making gopher multi-process.  So, if someone doesn't emerge from a hole and present us with an implementation, gopher will definitely be removed from standard builds of firefox (it could conceivably live on in single-process builds, and/or seamonkey, etc, but I wouldn't bet on it).

As discussed above, it looks like we'd prefer a pure JS implementation of this if it's going to happen.  If someone is interested, please contact me (best email: jduell aaat mozilla doot com).  

The timeline we're looking at here is less than a year from now when this is supposed to ship, so good luck, ya fuzzy varmints.

"Hey, that kangaroo just took my ball..."
Jason, I sent you E-mail.
Attached patch remove gopher support (obsolete) — Splinter Review
Here's an initial rev of removing gopher support from mozilla-central. I presume that followup patches will be needed for comm-central.
Assignee: nobody → joe
Attachment #439616 - Flags: superreview?(bzbarsky)
Attachment #439616 - Flags: review?(jduell.mcbugs)
I spoke to Jason in E-mail and per our last communication he isn't yet ready for review of the JavaScript Gopher in Electrolysis. I would appreciate that consideration being completed before the older support code is yanked, when Jason is ready to do so.
Comment on attachment 439616 [details] [diff] [review]
remove gopher support

># HG changeset patch
># Parent 935bb616b8a699537f2bbc2ef6392be1a263f4d3

>diff --git a/browser/components/migration/src/nsOperaProfileMigrator.cpp b/browser/components/migration/src/nsOperaProfileMigrator.cpp
>--- a/browser/components/migration/src/nsOperaProfileMigrator.cpp
>+++ b/browser/components/migration/src/nsOperaProfileMigrator.cpp
>@@ -481,18 +481,18 @@ nsOperaProfileMigrator::CopyPreferences(
> nsresult
> nsOperaProfileMigrator::CopyProxySettings(nsINIParser &aParser, 
>                                           nsIPrefBranch* aBranch)
> {
>   nsresult rv;
> 
>   PRInt32 networkProxyType = 0;
> 
>-  const char* protocols[4] = { "HTTP", "HTTPS", "FTP", "GOPHER" };
>-  const char* protocols_l[4] = { "http", "https", "ftp", "gopher" };
>+  const char* protocols[4] = { "HTTP", "HTTPS", "FTP"  };
>+  const char* protocols_l[4] = { "http", "https", "ftp" };

[3]?
(In reply to comment #86)
> >-  const char* protocols[4] = { "HTTP", "HTTPS", "FTP", "GOPHER" };
> >-  const char* protocols_l[4] = { "http", "https", "ftp", "gopher" };
> >+  const char* protocols[4] = { "HTTP", "HTTPS", "FTP"  };
> >+  const char* protocols_l[4] = { "http", "https", "ftp" };
> 
> [3]?

ugggh r- on me.
I agree that it would be best to try to switch to a JS implementation of gopher as soon as possible.   The last time I touched base with Cameron we didn't have cross-process object wrappers (CPOW) working yet, so a JS gopher for electrolysis seemed premature.  But if we can knock one off now, that would be good.

Cameron,  I don't actually know CPOW at all, so I'm going to point you at Ben Newman (bnewman@mozilla.com) for starters.  

Joe, I'll review when I get a chance.  I may hold off on landing it right away, because if we handle FTP for e10s as I've been planning (see bug 536289), we could probably have the current gopher code work, too, as the socket transport thread will still exist on the child process.  I'd rather not go there, but it's an option that might be worth keeping open for now.
The plan is now to move FTP into the chrome process, and turn off the socket transport thread in content.  So the current gopher code will not work out of the box in e10s.  It doesn't sound like we have a story yet for a JS implementation (would need reverse CPOWs?).  So gopher will not be in the next release of fennec (2.0).  We are more likely to have the groundwork (maybe even a whole golf course?) in place for a JS gopher add-on by the time we ship multi-process firefox.
Blocks: fennecko
Add-on or core?

Is there an omnibus bug I can follow for either/or reverse CPOW or MP Fx?
Attached patch Remove gopher v2 (obsolete) — Splinter Review
This patch removes one accidentally-included hunk, and fixes the indexing problem that Daniel Cater found (the previous patch passed all tests, proving that we don't have tests for the Opera importer).
Attachment #439616 - Attachment is obsolete: true
Attachment #441387 - Flags: superreview?(bzbarsky)
Attachment #441387 - Flags: review?(jduell.mcbugs)
Attachment #441387 - Flags: review?(gavin.sharp)
Attachment #439616 - Flags: superreview?(bzbarsky)
Attachment #439616 - Flags: review?(jduell.mcbugs)
Comment on attachment 441387 [details] [diff] [review]
Remove gopher v2

sr=me, but that pageinfo protocol list makes me very sad.  File a bug on that?
Attachment #441387 - Flags: superreview?(bzbarsky) → superreview+
Comment on attachment 441387 [details] [diff] [review]
Remove gopher v2

>diff --git a/browser/components/migration/src/nsIEProfileMigrator.cpp b/browser/components/migration/src/nsIEProfileMigrator.cpp

>       ProxyData data[] = {

>-        { "gopher=",  7, PR_FALSE, "network.proxy.gopher",
>-          "network.proxy.gopher_port" },

>       for (PRUint32 i = 0; i < 5; ++i) {

need to update this!       ------^

>         for (PRUint32 i = 0; i < 5; ++i)

and this! Just have them use NS_ARRAY_LENGTH(data).

>diff --git a/browser/components/migration/src/nsOperaProfileMigrator.cpp b/browser/components/migration/src/nsOperaProfileMigrator.cpp

Could use NS_ARRAY_LENGTH here too.

(this code is all stupid)

with those changes, r=me on the browser/, toolkit/, and extensions/reporter parts.
Attachment #441387 - Flags: review?(gavin.sharp) → review+
At least Mozilla is able to implement Gopher using JS, while IE/WinInet can't.
An interdiff!!

diff --git a/browser/components/migration/src/nsIEProfileMigrator.cpp b/browser/components/migration/src/nsIEProfileMigrator.cpp
--- a/browser/components/migration/src/nsIEProfileMigrator.cpp
+++ b/browser/components/migration/src/nsIEProfileMigrator.cpp
@@ -2156,17 +2156,17 @@ nsIEProfileMigrator::CopyProxyPreference
         { "https=",   6, PR_FALSE, "network.proxy.ssl",
           "network.proxy.ssl_port"    },
         { "socks=",   6, PR_FALSE, "network.proxy.socks",
           "network.proxy.socks_port"  },
       };
 
       PRInt32 startIndex = 0, count = 0;
       PRBool foundSpecificProxy = PR_FALSE;
-      for (PRUint32 i = 0; i < 5; ++i) {
+      for (PRUint32 i = 0; i < NS_ARRAY_LENGTH(data); ++i) {
         PRInt32 offset = buf.Find(NS_ConvertASCIItoUTF16(data[i].prefix));
         if (offset >= 0) {
           foundSpecificProxy = PR_TRUE;
 
           data[i].proxyConfigured = PR_TRUE;
 
           startIndex = offset + data[i].prefixLength;
 
@@ -2179,17 +2179,17 @@ nsIEProfileMigrator::CopyProxyPreference
                        data[i].portPref, aPrefs);
         }
       }
 
       if (!foundSpecificProxy) {
         // No proxy config for any specific type was found, assume 
         // the ProxyServer value is of the form host:port and that 
         // it applies to all protocols.
-        for (PRUint32 i = 0; i < 5; ++i)
+        for (PRUint32 i = 0; i < NS_ARRAY_LENGTH(data); ++i)
           SetProxyPref(buf, data[i].hostPref, data[i].portPref, aPrefs);
         aPrefs->SetBoolPref("network.proxy.share_proxy_settings", PR_TRUE);
       }
     }
 
   }
 
   return NS_OK;
diff --git a/browser/components/migration/src/nsOperaProfileMigrator.cpp b/browser/components/migration/src/nsOperaProfileMigrator.cpp
--- a/browser/components/migration/src/nsOperaProfileMigrator.cpp
+++ b/browser/components/migration/src/nsOperaProfileMigrator.cpp
@@ -481,22 +481,22 @@ nsOperaProfileMigrator::CopyPreferences(
 nsresult
 nsOperaProfileMigrator::CopyProxySettings(nsINIParser &aParser, 
                                           nsIPrefBranch* aBranch)
 {
   nsresult rv;
 
   PRInt32 networkProxyType = 0;
 
-  const char* protocols[3] = { "HTTP", "HTTPS", "FTP"  };
-  const char* protocols_l[3] = { "http", "https", "ftp" };
+  const char* protocols[] = { "HTTP", "HTTPS", "FTP"  };
+  const char* protocols_l[] = { "http", "https", "ftp" };
   char toggleBuf[15], serverBuf[20], serverPrefBuf[20], 
        serverPortPrefBuf[25];
   PRInt32 enabled;
-  for (PRUint32 i = 0; i < 3; ++i) {
+  for (PRUint32 i = 0; i < NS_ARRAY_LENGTH(protocols); ++i) {
     sprintf(toggleBuf, "Use %s", protocols[i]);
     GetInteger(aParser, "Proxy", toggleBuf, &enabled);
     if (enabled) {
       // Enable the "manual configuration" setting if we have at least
       // one protocol using a Proxy. 
       networkProxyType = 1;
     }
I'm still not clear on your statement, Jason. Are you referring to a reimplementation that would still be part of core?
At this point I'd lean towards making a future JS gopher impl an add-on, given the market share of gopher these days, and the resulting utility/security tradeoff (i.e the benefit of having it on by default for the 500 people or so who'd be saved the time to install the add-on, versus the downside to 200 million users if there wind up being security holes and it's installed by default).
One thing that bugs me is it sounds like that there is no way at all even for an Gopher add-on to function with the way CPOWs are set up now.

More importantly, however, with usage arguments aside (since that got beaten to death during the early part of this bug), I want to make sure that I didn't misinterpret Brendan's comment 67, i.e., that it could persist in core if a memory-safe lightweight implementation were developed. I wrote up the original JavaScript gopher draft implementation in good faith (and tested it) to be part of core, so I'd like to make sure the whole deal still applies after e10s reaches release.
And I should also add that I will gladly rewrite it once a 'reverse CPOW' solution reaches daylight, and commit to maintaining that code.
Mozilla has obvious interests in dealing in good faith, and in JS as safer implementation (including netwerk protocol implementation) language than C++. Therefore I continue to conclude that we should keep gopher in the core if it is implemented safely in JS.

/be
Cameron, given Brendan's position on the inclusion of gopher, and the current status of reverse CPOW, your time is probably best spent on creating a gopher test suite and making sure your current code passes it so it's possible for us to ship it - and then when reverse CPOWs become a real thing, you can port to them and ensure that it keeps working. :)
Oh, and I'd create another bug for reimplementing gopher in JS, attach your current patches, and make it dependent on both this bug and whatever bug encompasses reverse CPOWs. That way we can keep this bug focused and more understandable.
Comment on attachment 441387 [details] [diff] [review]
Remove gopher v2

+1 on Joe's suggestions for the new bug and test suite.
Attachment #441387 - Flags: review?(jduell.mcbugs) → review+
Okay, new bug 562320. I'll start working on that portion. Thanks for the suggestions.
Alright. We've got all the reviews we need here. I'd like to land this soon, before it bitrots, and then let Cameron loose in bug 562320. Do people have objections or alternative plans?
Why remove the C++ implementation of gopher before the JS implementation is even started? Wouldn't the JS implementation need the configuration table entries in the patch?

The point here isn't gopher _per se_. It's orderly work and square dealing.

Did a bug on nsOperaProfileMigrator not being tested, or whatever's wrong with it ("this code is all stupid") get filed?

/be
The JS version I attached originally to this bug (attachment 333063 [details]) does need the current 'magic' in netwerk/streamconv/converters/nsIndexedToHTML.cpp to work. I'm perfectly fine with marking it for review now, or transferring it to bug 562320 and asking for review there. It's still essentially the same code right now.
> Why remove the C++ implementation of gopher before the JS implementation is
even started?

Because the existing C++ gopher isn't going to work in electrolysis, so we'd need to block fennec, or land this patch on a fennec-only tree (and I believe the goal is to share trees?  Correct me if I'm wrong here).
Blocks: 559714
Comment on attachment 441387 [details] [diff] [review]
Remove gopher v2

Clearing my review for now.  We should hold off on removing C++ gopher for now, as in the absence of reverse-CPOW support (and/or whatever else it takes to make JS gopher work under e10s), I suspect that the path of least difficulty in the short term may be to e10s-ify the gopher C++ code.  This has turned out not to be so hard for FTP (see bug 536289), and gopher ought to similar, i.e. a few days' work.  It's a fair amount of code, but mostly cut-and-paste scaffolding for IPDL/e10s.

Cameron:  how's your C++? (email me if you're up for trying this).
Attachment #441387 - Flags: review+
Enough to be dangerous :) E-mail sent.

I'm about a quarter of the way through converting the HTTP test suite to Gopher, btw.
(In reply to comment #100)
> Mozilla has obvious interests in dealing in good faith, and in JS as safer
> implementation (including netwerk protocol implementation) language than C++.
> Therefore I continue to conclude that we should keep gopher in the core if it
> is implemented safely in JS.

Shaver pointed out that I changed my position from comment 47. I did not mean to say Gopher has to stay in the core, forever. I was arguing for an orderly series of steps from Gopher in C++ in the core in Firefox 3, to Gopher in JS probably in the core, to Gopher in JS as add-on. But my comment 100 did not restate the last step there.

I think gopher should be an add-on unless there's some massive gopher revival, which I do not foresee.

Hosting protocols in JS where we can is worthwhile, but we can do it with gopher or not, add-on or not, as a separate exercise. My comment 47 threat to remove gopher "by Mozilla 2" -- meaning only the release where we break frozen API compatibility -- was intended to get people thinking about building gopher as an add-on, not into the core.

I bet we are close to a virtual "Mozilla 2" with 1.9.3. We should think about it.

/be
First, I think it's *great* that Cameron is working on this; I'm always glad to see someone with energy (and great manners, I must say) coming to work with the project, even if I don't think that I'll ever run their code. :-)

Right now, we face a choice:

a) remove C++ gopher support
b) block Fennec
c) fork the Fennec necko to remove it
d) wait for e10s-izing of gopher

I think that of those, given my understanding of our constraints and resources, we should pursue a) right now.

Then we face another choice:

1) reintegrate an e10s-ized gopher in C++
2) block Firefox 4 and Fennec 2 on sufficient JS-accessible infrastructure for Gopher to be implemented in JS
3) provide the infrastructure needed for gopher to be written in JS, as best we're able, but without a promise or blocking commitment

Here I prefer 3, quite strongly: we have many miles to go with E10S and networking, and wrappers, and we might not have cycles for extra gopher-only requirements.  The energy of Cameron and others isn't something that I want to squander, so I want to be clear about this: it's still possible that we might hit a roadblock of functionality or bug that keeps Gopher-in-JS from working in a post-E10S Fennec, and in Firefox 4.  I hope it's not the case, because I'd like our platform to support that sort of extension in JS, but I honestly don't think that I would be in favour of holding a release for it.

So, tl;dr: I think we need to remove Gopher now, and have interested parties work on a Gopher add-on using JS as the various necessary capabilities are available.  Thanks especially to Cameron for being so pleasant about this somewhat disorderly progress!
Attachment #441387 - Flags: review?(jduell.mcbugs)
Comment on attachment 441387 [details] [diff] [review]
Remove gopher v2

This patch still applies.
Just so I can make some estimations for the community, is the current C++ code (and I assume the non-e10s code) remaining for the remainder of Firefox 3's life? At least that gives us a chance at a transition. There is the Overbite extension which already exists, but from what I'm reading here it won't work with an e10s-based Mozilla 2/Fx 4 as written.

I'll wait for a definite decision before I stop work on the e10s conversion I have started on, although the bugs I have read imply that whatever I'm doing right now would not work in any case (it would just be getting the groundwork down).
(In reply to comment #114)
> Just so I can make some estimations for the community, is the current C++ code
> (and I assume the non-e10s code) remaining for the remainder of Firefox 3's
> life? At least that gives us a chance at a transition. There is the Overbite
> extension which already exists, but from what I'm reading here it won't work
> with an e10s-based Mozilla 2/Fx 4 as written.
We would not back-port the removal, no.
Firefox 4 is also not going to be based on Electrolysis, at least not in the "separate chrome and content" sense.
Correct, we will not be removing it from Firefox 3.6.x, but it would not be present in the next major release (Firefox 4, or Firefox 2 for Maemo/Android).
Okay, then let me ask bluntly whether it is worth me continuing on the e10sified Gopher I'm working on now. Based on Jason's suggestion, I am presently converting the C++ Gopher to a C++ e10s version, and writing a simple test suite to go with it, modeled on the FTP patches. This is not ready yet.

If you would be willing to consider including the Electrolysis C++ gopher if I could get it done within the timeframe you specify, I would be willing to rise to the challenge. I certainly don't want to block any release by being tardy.

However, what bothers me most is that it seems like a JS gopher is not possible now, and might not ever be (comment 112 #3), even as an add-on and let alone in core. If that is true *and* an electrolysis C++ gopher is not desirable (comment 112), then that sort of leaves me with no options whatsoever for implementation now or in the future.
(In reply to comment #118)
> However, what bothers me most is that it seems like a JS gopher is not possible
> now, and might not ever be (comment 112 #3), even as an add-on and let alone in
> core. If that is true *and* an electrolysis C++ gopher is not desirable
> (comment 112), then that sort of leaves me with no options whatsoever for
> implementation now or in the future.
Is there any reason why you couldn't ship an add-on with C++?
Is it really the case that we require C++ as protocol handler host language? That would suck no matter what happens with gopher. It's not urgent, but I would be interested to hear the details of why this is so (if it is so).

/be
Attachment #441387 - Flags: review?(jduell.mcbugs) → review+
While this is being determined, I have changed bug 562320 to refer to the changes to the C++ gopher for e10s and attached a WIP patch with test case. The patch doesn't fully work yet; I'm not sure what I'm doing wrong (the unit test *does* work with 1.9.2). I'll follow up there for Jason's comments while watching this bug.
Comment on attachment 441387 [details] [diff] [review]
Remove gopher v2

> Shaver said:
> d) wait for e10s-izing of gopher

So bug 562320 is progressing nicely;  we've almost got full e10s C++ gopher support (we're downloading pages in xpcshell already).  I doubt completing it will be the long pole in the e10s necko timeline, and Cameron is doing all the work for us, so far with very little hand holding.

Given that this will provide continuous gopher support in Firefox, and is less likely to consume our developer time going forward than trying to figure out and supply the needs of a JS e10s gopher, should we just take it?  That's my vote at this point, unless someone convinces me otherwise.
Attachment #441387 - Flags: review+
(In reply to comment #122)
> Given that this will provide continuous gopher support in Firefox, and is less
> likely to consume our developer time going forward than trying to figure out
> and supply the needs of a JS e10s gopher, should we just take it?  That's my
> vote at this point, unless someone convinces me otherwise.
We'd need those anyway for other potential protocols, wouldn't we?
I do not believe that an in-core gopher implementation is desirable, whether it supports e10s or not -- certainly, the intent to remove was not motivated by electrolysis, since it long predates it.  It is code size, attack surface, test suite time and future maintenance burden that simply does not pay for itself, even by our generous standards.  Making the C++ implementation available in an extension seems like a good option there, as raised by sdwilsh in comment 119.

I agree with brendan's sentiment in comment 120: we want to have JS protocol handler support in the longer term anyway, though it may not need to be present in FF-for-Maemo-and-Android 2.0.
As background, an "enhanced" gopher already exists as the OverbiteFF add-on, which people are using now. It is written in JavaScript.

OverbiteFF in C++ would be, with all due respect to Shawn and Mike, a really rotten idea. I would lose all the special features in the JavaScript add-on I've already written and have to port them to another implementation, and would need to set up (or have the community set up) a system for building the binaries for the environments we already support and people already depend on with the current JS implementation. It would basically require a rewrite for the third time, and would significantly reduce its reach in the process for want of binaries. If e10s were going to be the future, take it or leave it, then I'd grit my teeth and do it, but this thread already makes clear that it is not.

I realize this isn't Mozilla's problem, but FTR, I (and the others who wanted Gopher to remain) haven't asked for a free lunch more than any other core feature would require, and I hope I've demonstrated that with the work I've done. Essentially in 1.9.3 it seems there is no way for the OverbiteFF add-on as written to function, nor any way to make it function until the JS protocol handlers are written (please correct me if I misunderstand). Switching gears completely for a C++ implementation as a temporary holdover would be a huge burden just to switch back.

I've personally already accepted that Gopher support in Mozilla Core will be removed in the near future, which is why I wrote the OverbiteFF add-on in the first place as a good-faith attempt at a transition for the users including myself who still use its functionality. At least with the code I've written for bug 562320 there is *some* support for those users, even though they will lose the special features of OverbiteFF until the JS protocol handler support comes back. I'm just asking to allow this transition to occur more smoothly, and I'm still just as willing to do whatever work necessary to make that happen. This isn't really a big deal for Fennec to me, but it would be for a desktop Fx.

If JS protocol handler support is further along than I have surmised, I would appreciate some pointers where to watch.
(In reply to comment #125)
> As background, an "enhanced" gopher already exists as the OverbiteFF add-on,
> which people are using now. It is written in JavaScript.

I've looked for the source of OverbiteFF before. Is it available?

Here's what I think we should do:

 1.) Make sure OverbiteFF-in-JS can run on 1.9.3
 2.) remove our current gopher support
 3.) ship a gopher protocol handler that shows an about:gopher page which links to Overbite's AMO page (bonus points for no-restart installation)
(In reply to comment #126)
>  1.) Make sure OverbiteFF-in-JS can run on 1.9.3
>  2.) remove our current gopher support
>  3.) ship a gopher protocol handler that shows an about:gopher page which links
> to Overbite's AMO page (bonus points for no-restart installation)

But would this work with other Gecko-based browsers which have no extension mechanism, or which have one not compatible with firefox add-ons?
OverbiteFF literally is just JavaScript. Unzip the .xpi and the protocol is largely housed in components/protocol.js (or I can post it somewhere). The JavaScript implementation I attached to this bug is largely based on it, just stripped down and congruent with Mozilla house style.
(In reply to comment #127)
> 
> But would this work with other Gecko-based browsers which have no extension
> mechanism, or which have one not compatible with firefox add-ons?

I think other Gecko browsers could bundle the Overbite code with very small modifications.
>  1.) Make sure OverbiteFF-in-JS can run on 1.9.3

This should definitely happen. What's involved, which bug(s)?

Cameron broke it down more than I would have: we should not make platform changes that require rewrites from JS to C++ without some serious discussion. Add-on or core is secondary. Gopher is tertiary, but thanks to Cameron for being a good citizen here on the primary and secondary fronts.

/be
Attached patch V2, unbitrottedSplinter Review
Unbitrotted. Carrying forward review flags.
Attachment #275284 - Attachment is obsolete: true
Attachment #275285 - Attachment is obsolete: true
Attachment #275286 - Attachment is obsolete: true
Attachment #275287 - Attachment is obsolete: true
Attachment #333063 - Attachment is obsolete: true
Attachment #333064 - Attachment is obsolete: true
Attachment #441387 - Attachment is obsolete: true
Attachment #451120 - Flags: superreview+
Attachment #451120 - Flags: review+
Dolske pushed this. http://hg.mozilla.org/mozilla-central/rev/b1d4b2d36902
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
(In reply to comment #132)
> Dolske pushed this. http://hg.mozilla.org/mozilla-central/rev/b1d4b2d36902

"Bug 388195 - Remove gopher support. r=jduell,gavin sr=bz"

For the record, jduell revoked his r+ of this patch, so the commit comment is wrong.
So what happens now, especially with regard to bug 562320?

Also, I would like to work on Robert Sayre's suggestion #3 (I'll file a separate bug and assign it to myself if necessary).
Flags: wanted1.9.2?
Target Milestone: mozilla2.0 → mozilla1.9.3a6
(In reply to comment #134)
> So what happens now, especially with regard to bug 562320?

Not sure about that, but does

>> 1.) Make sure OverbiteFF-in-JS can run on 1.9.3

need its own bug? Please cc: me.

> Also, I would like to work on Robert Sayre's suggestion #3 (I'll file a
> separate bug and assign it to myself if necessary).

Cool, thanks.

/be
I have filed bug 572000 for comment 126. I have a couple questions about how it should be implemented; see bug summary.

I remember from a discussion earlier with Jason Duell and Ben Newman that it needed a 'reverse CPOW' structure, which was going to be part of the handle design, but I don't know what bug is tracking that, if any.
Actually, I think the one specifically that I will need is probably bug 557259.
Keywords: dev-doc-needed
One final request while I wait for review on bug 572000; since this will undoubtedly appear in the release notes for Fx 4, would it be okay to add a link to the OverbiteFF AMO page there as well? Should that also be filed as a bug?
https://addons.mozilla.org/addon/7685/
Blocks: 572389
(In reply to comment #139)
> One final request while I wait for review on bug 572000; since this will
> undoubtedly appear in the release notes for Fx 4, would it be okay to add a
> link to the OverbiteFF AMO page there as well? Should that also be filed as a
> bug?
> https://addons.mozilla.org/addon/7685/

While I believe that the gopher community is small, I also think this is likely to show up in both relnotes and userdocs for FF4 - tagging with the appropriate keywords. I don't make this call, but I'm personally all for linking to the addon.
Blocks: 572000
Depends on: 700253
Depends on: 593352
You need to log in before you can comment on or make changes to this bug.