73964 - can't login to compuserve webmail (stream converter problem)

Status: RESOLVED FIXED

(SeaMonkey :: General, defect)

Description

[Judson Valeski]

1- goto www.compuserve.com
2- click on computing
3- provide compuserve user name and password. 
4- user is presented w/ the following

We were able to validate your login ...
your browser did not accept the login session cookie.
Hence you can't access this content area at this time.

Comment 1

[Phil Peterson]

severity -> major. need to know why this doesn't work.

Comment 2

[Stephen P. Morse]

I'm on it.

Comment 3

[Stephen P. Morse]

Problem appears to be that a cookie which is received is never sent back.  
Specifically the cookie named SESSAUTH which has a value that is 240 characters 
long.  Don't know why yet.

Here are the sequence of events from traffic captured using a 4.x browser 
(success case) and a mozilla browser (failure case).  The cookies followed by a 
question mark are sent in the failure case only.

<< bunch of binary traffic >>

send:
  GET /both_no_qc_wam?
  no cookies sent

receive:
   Location: http://login.compuserve.com/login/LoginSuccess.asp?
   no cookies set

send:
  GET /login/LoginSuccess.asp?
  Cookie: WAPref, WAUrl, (ASPSESSIONIDGQQQQHXC, ASPSESSIONIDGGGQQILN)?

receive:
  set-cookie: WAUrl, WAFlag, SESSAUTH
  location: http://login.compuserve.com/login/LoginSetCheck.asp

send:
  GET /login/LoginSetCheck.asp
  cookie: ASPSESSIONIDGQQQQHXC, WAPref, WAFlag, (ASPSESSIONIDGGGQQILN)?
           missing SESSAUTH

receive:
  bad case: <TITLE>No Login Cookie!</TITLE> followed by error message
  good case: <title>Object moved</title>
            and redirects you to http://www.compuserve.com/login?

Comment 4

[Stephen P. Morse]

OK, there's more than one problem here.

First there's a bug in the cookie module as follows.  When cookies are being 
sent out to the server, we loop through the cookie list looking for cookies that 
belong to that server and send them out.  While looping we also check each 
cookie to see if it expired and we delete it from the list if it is.  But we do 
not alter the index which we are using to loop through the list.  So, for 
example if we determine cookie at position 3 has expired, we delete it and then 
go and test cookie at position 4.  But the act of deletion has moved all cookies 
higher than #3 down one position.  So the cookie at position 4 is really the 
cookie that used to be at position 5.  Note that we completely skipped over the 
cookie that was originally at position 4 and never tested it.  And if that 
cookie belongs to the server in question, it will not get sent although it is 
suposed to.

Note that many things have to all come together in order to hit this problem.  
Specifically the list contains a cookie that has expired followed immediately by 
a cookie that is supposed to get sent to the server.  It's not a likely event 
and normally not a very reproducible one (the expiring cookie) if it does occur.  
But if the host itself expires its own cookies (for the purpose of deleting 
them), then this will be reproducible.  And this is exactly what compuserve was 
doing.

Well the fix for that is easy.  I'll post it right here, inline, instead of 
making an attachment.  In my next comment I'll discuss the next problem.

Index: nsCookies.cpp
===================================================================
RCS file: /cvsroot/mozilla/extensions/cookie/nsCookies.cpp,v
retrieving revision 1.1
diff -u -r1.1 nsCookies.cpp
--- nsCookies.cpp       2001/03/29 01:53:21     1.1
+++ nsCookies.cpp       2001/03/30 04:33:03
@@ -496,7 +496,7 @@
       /* check for expired cookies */
       if( cookie_s->expires && (cookie_s->expires < cur_time) ) {
         /* expire and remove the cookie */
-        cookie_list->RemoveElementAt(i);
+        cookie_list->RemoveElementAt(i--); /* decr i so next cookie isn't skipp
ed */
         deleteCookie((void*)cookie_s, nsnull);
         cookie_changed = PR_TRUE;
         continue;

Comment 5

[Stephen P. Morse]

With the above patch made, we get much further.  The error message does not 
occur and the browser requests and receives several more pages.  It 
eventually gets the following response from the server.  It is supposed to 
display a message (as indicated in the html below) and in a few seconds go to 
the next page.  But instead it simply displays a blank page and nothing more 
happens.

HTTP/1.0 403 Forbidden
WWW-Authenticate: Remote-Passphrase Realm="nonsense", State="Failed"
Expires: Tue, 01 Jan 1980 00:00:00 GMT
Pragma: no-cache
Content-Type: text/html
Content-Encoding: 8bit
Content-Length: 1357

<HTML>

<HEAD>
<TITLE>RPA Error Page - Access Denied</TITLE>
<META HTTP-EQUIV="Refresh" CONTENT="0;url=http://www.compuserve.com/guestsignup/
RPA_AccessDenied.asp?refer=https%3a%2f%2flogin.compuserve.com%2flogin%2flogin.as
p%3fOrigUrl%3dhttp%253a%252f%252fwww.compuserve.com%252fcomputing%252f%26RefUrl%
3dhttp%253a%252f%252fwww.compuserve.com%252fcompuserve%252fdefault.asp%26AuthUrl
%3dhttp%253a%252f%252fwww.compuserve.com%252flogin&dest=http%3a%2f%2fwww.compuse
rve.com%2fcomputing%2f&site=www.compuserve.com&type=im&GIDs=">
</HEAD>

<BODY BGCOLOR="FFFFFF">
<FONT FACE="Arial,Helvetica,Geneva" SIZE=-1><B>
<P>
CompuServe is in the process of sending you to another page... please wait.
<P>
Certain older browsers do not support this feature.  If this message is displaye
d for more than a few seconds, please <A HREF=http://www.compuserve.com/guestsig
nup/RPA_AccessDenied.asp?refer=https%3a%2f%2flogin.compuserve.com%2flogin%2flogi
n.asp%3fOrigUrl%3dhttp%253a%252f%252fwww.compuserve.com%252fcomputing%252f%26Ref
Url%3dhttp%253a%252f%252fwww.compuserve.com%252fcompuserve%252fdefault.asp%26Aut
hUrl%3dhttp%253a%252f%252fwww.compuserve.com%252flogin&dest=http%3a%2f%2fwww.com
puserve.com%2fcomputing%2f&site=www.compuserve.com&type=im&GIDs=>
click here</A> to go on, or contact Customer Service for more information.
</B></FONT>
</BODY>

</HTML>

Comment 6

[Stephen P. Morse]

cc'ing some network folks because I now believe that with the cookie bug patch 
(given above), the remaining impediment to a successful compuserve login has to 
do with the server response (perhaps the 403 code).

Incidentally, the lines in the response that read

   HTTP/1.0 403 Forbidden
   WWW-Authenticate: Remote-Passphrase Realm="nonsense", State="Failed"

are normal.  The same response is received when running the 4.x browser and you 
do get a successful login in that case.

Comment 7

[Judson Valeski]

r=valeski on the RemoveElementAt(i--); patch. Let's not start mixing bugs here,
and open another one for the problem w/ the server response.

Comment 8

[Stephen P. Morse]

Could also be a layout problem because the META HTTP-EQUIV="Refresh" is never 
getting acted on (I'm not getting to a breakpoint set in ProcessHeaderData for 
the refresh).  cc'ing harish.

Comment 9

[Stephen P. Morse]

Jud, it's not a case of "mixing" bugs.  This bug report says we can't get to 
compuserve mail and even with the patch you just reviewed we still can't get to 
compuserve mail.  We don't yet know why and it's only speculation on my part 
that it might be due to the server response.

Comment 10

[Stephen P. Morse]

cc'ing alecf for a superreview of the cookie patch presented above.  But that 
still doesn't completely solve the problem.

Comment 11

[Judson Valeski]

we found a bug in the cookie code, let's fix that. independent of everythign
else. mixing checkins to address a problem is, IMO, bad... it leads to "which
checkin is causing more problems". I can make a bug that says "The browser
should be faster" and we certainly wouldn't just lop all bugs into that.

Comment 12

[Stephen P. Morse]

OK, I broke out the cookie bug into bug 74112.  This bug is still focused on why 
we can't do a successful compuserve login.

Alecf, please put your sr in bug 74112 and not here.  Thanks.

Comment 13

[Stephen P. Morse]

Component is unknown at this time but it definitely is not cookies.  Updating 
component to browser-general.

Comment 14

[timeless]

reassign :(

Comment 15

[Stephen P. Morse]

Oops, by reassigning this you completely took me off the list of people who get 
notified about this bug.  Even though you've reassigned it, I'm still looking 
into this and trying to figure out how to fix it.

Comment 16

[Asa Dotzler [:asa]]

I'm not looking into this and trying to figure out how to fix it  :) 

Comment 17

[hirata masakazu]

This seems to cause login errors at www.email.com and www.scmp.com.
2001040309 mac-trunk.

Comment 18

[Stephen P. Morse]

OK, I've returned to actively investigating this.

With the fix to the cookie problem (bug 74112), we no longer get the message 
about session cookies not being accepted.  Instead we get further and arrive at 
a blank screen.  This is what I stated in my comment of 2001-03-29 20:42.

I now have clues as to why we are getting the blank screen.  It's due to the 
failure to find a stream converter to convert from "8-bit" to "uncompressed".  
Specifically, the following call in nsStreamConverterService.cpp on line 558 is 
failing:

      rv = FindConverter(cContractID, &converterChain);

at this point the value of cContractID is

   "mozilla.org/streamconv;1?from=8bit&to=uncompressed"

And, indeed, if you read the sourcecode in nsStreamConverterService.cpp you will 
see the comment

 // XXX curently we only handle a single from and to combo, we should repeat the
 // XXX registration process for any series of from-to combos.

So it looks like the stream converter that we want has not yet been implemented.

Not knowing how to implement a stream converter (or even what a stream converter 
is), I can at least work around this problem by commenting out the following 
lines in nsHTTPResponseListener.cpp:

            rv = mResponse->GetHeader(nsHTTPAtoms::Content_Encoding, 
                    getter_Copies(compressHeader));

This will cause us to ignore the header that says

      Content-Encoding: 8bit

and consequently we never even try to use a stream converter.

Comment 19

[Stephen P. Morse]

So, continuing with my comment above, the good news is that we now know that the 
next problem causing compuserve login to fail is an unimplemented stream 
converter.  And I have a work-around for that problem although we will need a 
real fix.

The bad news is that although this allows us to get further along, we still 
don't get to do a successful compuserve login.  Instead I now get to a page that 
says:

   This area can not be accessed by your account.

   This usually happens because:

    * Special permissions are required.
    * Availability is limited by geographic region.

   If you believe that your account should be able to access this area, please
   contact Customer Service at GO FEEDBACK to report the problem.

   If you would like to make another selection, click on your browser's back
   button to return to the previous page.

Comment 20

[Stephen P. Morse]

Got it!  The last problem is due to our user-agent string.  If I modify the 
string so that we look like a 4.x browser, then compuserve lets us in.  So our 
own company, compuserve, needs to be evangelized to accepting the N6/Mozilla 
browser.

To demonstrate this I made the following change to nsHTTPRequest.cpp

-        SetHeader(nsHTTPAtoms::User_Agent, uaString.get());
+        SetHeader(nsHTTPAtoms::User_Agent, "Mozilla/4.73 [en] (WinNT; U)");

And with that change I was able to successfully log on to compuserve.

So let me summarize.  There were three problems that prevented the compuserve 
login from succeeding:

1. Cookie problem which has since been fixed (bug 74112)
2. Unimplemented stream converter for "8-bit"
3. Compuserve is blocking the mozilla browser (needs evangelism help here)

I just filed a separate bug on the evangelism issue.  It's bug 75058.

Comment 21

[Stephen P. Morse]

With the cookie problem broken out and already fixed (bug 74112), and the 
evangelism broken out (bug 75058), the only issue left to be addressed in this 
bug report is the stream converter issue.  Modifying the summary line to 
indicate that.

Comment 22

[Phil Peterson]

> So our own company, compuserve, needs to be evangelized to accepting the 
> N6/Mozilla browser.

I contacted someone at CS. Thanks for the analysis, Steve.

Comment 23

[Stephen P. Morse]

Continuing my comment about stream converters, it appears that the following are 
the stream converters that we currently recognize:

--- FROM ---                       --- TO ---
text/ftp-dir-unix                  application/http-index-format"
text/ftp-dir-nt                    application/http-index-format"
text/gopher-dir                    application/http-index-format"
multipart/x-mixed-replace          */*"
multipart/mixed                    */*"
application/x-unknown-content-type */*"
chunked                            unchunked"
unchunked                          chunked"
gzip                               uncompressed"
x-gzip                             uncompressed"
compress                           uncompressed"
x-compress                         uncompressed"
deflate                            uncompressed"
text/plain                         text/html"
application/http-index-format      text/html"

Note that the one needed for the compuserve page is from "8-bit" and that is not 
in the list

Comment 24

[Stephen P. Morse]

I'd like to propose the following patch to the stream converter problem and see 
if it will fly.  Instead of returning an error message when we can't find a 
converter, simply don't do a conversion.  In that way we can at least make an 
attempt to support any unrecognized conversions that might be encountered in the 
future.  And it will solve the problem that we are having with the compuserve 
login.

So the patch is simply the following.  Any comments?

Index: nsHTTPResponseListener.cpp
===================================================================
RCS file: 
/cvsroot/mozilla/netwerk/protocol/http/src/nsHTTPResponseListener.cpp,v
retrieving revision 1.149
diff -u -r1.149 nsHTTPResponseListener.cpp
--- nsHTTPResponseListener.cpp	2001/03/20 22:42:03	1.149
+++ nsHTTPResponseListener.cpp	2001/04/08 18:10:35
@@ -478,8 +478,9 @@
                         mResponseDataListener,
                         request,
                         getter_AddRefs(converterListener));
-                if (NS_FAILED(rv)) return rv;
-                mResponseDataListener = converterListener;
+                if (NS_SUCCEEDED(rv)) {
+                  mResponseDataListener = converterListener;
+                }
             }
         }

Comment 25

[Stephen P. Morse]

I was mistaken above when I said that there was an evangelism issue with 
compuserve.  I just ran some more tests, both with the real mozilla user-agent 
string and with the masqueraded 4.x string.  I observed that in each case 
sometimes it worked and sometimes it didn't.

I now realize that the success or failure is caused by the fact that I sometimes 
simplified my testing by going directly to www.compuserve.com/computing rather 
than going to www.compuserve.com and then clicking on the link for computing.  
In the latter case you are missing some cookies at the point at which you log in 
and that is why you get to the page saying "This area can not be accessed by 
your account".

I must have changed my method of testing when I switched the user-agent string 
which caused me to erroneously concluded that it was an evengelism issue.  I 
will post this comment in the evangelism bug (bug 75058) and close out that 
report.

So the only issue blocking us from doing a successful compuserve login is the 
stream-converter problem for which I posted a proposed patch above.

Comment 26

[Judson Valeski]

The patch looks good to me. It's up to the consumers of the stream converter
service to determine what action to take if a converter can't be found. For this
8bit conversion, it makes sense to just move along as though nothing happend.
However, I suspect this will get nasty when other converters come down the pike
that we don't handle. To get compuserve mail to work, the patch makes sense.
However, I wonder if we should be throwing a dialog when we run into conversion
we can't handle in HTTP.? That's another bug though. r=valeski on the HTTP patch.

BTW. sending 8bit encoding as a content-encoding seems redundant to me; the
content will be treated as 8bit anyway. sounds like a server misconfiguration.

Comment 27

[Stephen P. Morse]

cc'ing alecf for sr

Comment 28

[Darin Fisher]

steve: nice find... the code in nsHTTPServerListener::OnDataAvailable is
definitely far from forgiving ;-)

r/sr=darin

Comment 29

[Stephen P. Morse]

Fix checked in.