15111 - [DOGFOOD][PP] 5.0 fails to read in cookies.txt file on mac

Status: VERIFIED FIXED

(Core Graveyard :: Profile: Migration, defect, P3)

Description

[pmock]

Build Date & Platform Bug Found:
 MacOS seamonkey build 1999-09-28-08 installed on G3/400 MacOS 8.5.1

Overview Description:
 Migration fails to port over my 4.7 cookie file properly.  In 4.7, I had saved
my password to log into the Wall Street Journal.  After migrating to 5.0, it
does not use my cookie file.  It insists on prompting me for my username and
password.

Steps to Reproduce:
1) Edit your MagicCookie file
2) Copy the cookie information for the WSJ site
3) Save the file
4) Launch 4.7
5) Go to http://interactive.wsj.com/edition/current/summaries/front.htm
   It will log you in.
6) Exit 4.7
7) Migrate your 4.7 pref to 5.0
8) Launch 5.0
9) Go to http://interactive.wsj.com/edition/current/summaries/front.htm
   It prompts you for your password.

Actual Results:
  It prompts you for your user id and password.

Expected Results:
  It should log you in automatically.

Additional Builds and Platforms Tested On:
 I have not tried migrating on win32 or linux.

Additional Information:
 If you look in the cookies.txt that was migrated.  It does contain your
migrated info.

Comment 1

[lchiang]

change QA contact to Grace since this is migration of cookies.

Comment 2

[(not reading, please use [e-mail] instead)]

accepting.  marking all milestone 11.

Comment 3

[Phil Peterson]

Nominate for dogfood

Comment 4

[Phil Peterson]

Would like to have this for M11, but wouldn't hold for it. Definitely a dogfood
stopper.

Comment 5

[selmer (gone)]

Bhuvan, Seth has a lot of PDT+ bugs.  Let's try to help with this one.  Don, is
this anything you've seen?

Comment 6

[dbragg]

No I haven't seen anything like this but then I don't run regularly on the Mac.
But this doesn't seem to be a migration issue.  I see a lot of these, "I
migrated and it doesn't work" types of bugs.  The problem isn't the migration
but how the client uses the migrated files.  The statement from the original
bug report: "If you look in the cookies.txt that was migrated.  It does contain
your migrated info." is what makes me think this isn't a migration bug.

Comment 7

[lchiang]

cc: paulmac. Cookies area.

Comment 8

[Paul MacQuiddy]

I agree with don, the file is copied over correctly, but 5.0 must have issues
with the formatting of the file. cc'ing neeti
p.s. tever is the cookies guy

Comment 9

[racham]

tever & neeti,

Lately, I have noticed the sentence 'Opening file cookperm.txt failed' on the
console as I ran the app with a migrated profile. Does CookieManager create this
file (cookperm.txt) by default...? Is this failure stopping us from reading
cookies.txt ?

Comment 10

[Paul MacQuiddy]

Steve Morse is the keeper of the cookieperm.txt file, I believe

Comment 11

[Stephen P. Morse]

That message about cookperm.txt is normal.  It doesn't exist initially and so
you fail to open it.  But that shouldn't bother anything else.

A couple of points.

1. The Wall Street Journal site uses cookies with high-bits set.  In particular,
the username and password cookies end in the ascii character FD (hex).  Also the
encoding they use for the characters in username and password maps the ascii
characters into other ascii characters; for the most part the mapped characters
are printable and don't have the high bit set but the mapping for 'c' does have
the high bit set (very surprising).  So could it be that the migration program
is not preserving the setting of the high bits?  That would explain why it
appears on visual inspection that the migrated file is correct but in reality it
is not.

2. Please try this on a win32 platform.  If it fails there as well, then send me
the pre and post migrated files and I'll investigate further.

Comment 12

[Stephen P. Morse]

Also try just copying the 4.7 cookies file into the 5.0 profile (i.e., don't
migrate but do a manual copy) and see if that makes any difference.

Comment 13

[racham]

Steve Morse can give us the insight on failure to use cookies info.
For now, it sounds more like a cookie service issue.
Reassiging the bug to Steve Morse.

However, If the migration is proved to be wrong, don might have to use a
different way to copy the cookies file to maintain the integrity of the file.

Adding myself to the cc list.

Comment 14

[Stephen P. Morse]

I'll accept this bug only if you answer my question -- does the failure occur on
win32 platform as well or is this a mac-only problem?

Comment 15

[Grace Bush]

Steve,

I don't have a wsj cookie but I can access my NY Times account and get through
to downloading a subscription service using a migrated cookie. Does that help?

Grace

Comment 16

[Stephen P. Morse]

Yes, it helped more than you realize.  I was confusing Wall Street Journal with
New York Times.  Everything I said above about the use of high-order bits on the
login cookie applies to New York Times, not Wall Street Journal.  Reading
your comment made me realize my confusion.

But will somebody please answer my question about whether this is reproducible
on win32 or is it a mac-only bug.

Comment 17

[racham]

Grace,

Do you have latest builds on Mac and Windows ?
Can you provide the data Steve is asking for ?

Thanks.

Comment 18

[Grace Bush]

I was using build 1999111608 - M12 for win32 - waiting for installer build for
today.

I have installed today's build for Mac and will try to setup NY times account
there and test

Comment 19

[Grace Bush]

Attachment: 5.0 migrated cookie file

Comment 20

[Grace Bush]

Attachment: 4.7 cookie file

Comment 21

[Stephen P. Morse]

Hmm.  So your New York Times username is gbush2 and your password is abbysa260.
;-)

Comment 22

[Grace Bush]

nope,  wrong password, which file did you look in to get abbysa260.....?
hopefully it is the 5.0 file and it did not get copied correctly....

Comment 23

[Stephen P. Morse]

OK, how about abbysam60?  There was a number "1" in the cookie file and I
mistakenly read it as the letter "l".

I got this from you 4.7 cookie file.  The 5.0 one has no relation at all to the
4.7 one.

Comment 24

[Stephen P. Morse]

Here are some questions that I need to get answered before I can make any
progress on this bug.

1. Is this bug reproducible on a win32 platform or is it only on a mac?

2. Is there a Wall Street Journal login that someone is willing to let me use?
I don't want to pay for a subscription (nor do I want to sign up for the two
week free trial since that requires using my credit card as well).

3. In 5.0, after you are prompted (which you shouldn't have been) for your
username and password, can you then have the login saved in a cookie and will
you not be prompted on future visits to the site?

Awaiting Peter's return from the offsite for answers to these questions.

Comment 25

[pmock]

Answer to question:
1) Using today Win32 commercial seamonkey build 1999-11-17-09-m12 build, I can
not reproduce this problem. Using today MacOS commercial seamonkey build
1999-11-17-08-m12 build, this problem is still reproduciable.

2) I don't have a Wall Street Journal account.  I'm was using the saved cookie
file for Chris Saito.  The cookie file allows me to login in 4.7.  After
migration to 5.0, I am prompted for the user id and password.

3) Can't test this scenario because I don't know Chris's password or login id.
You might contact Chris Saito.  If you would like a copy of his cookie file, I
can email it to you.  Please note, Chris asked me NOT to attach his cookie file
to the bug.

* I have not tried Linux.

Comment 26

[Stephen P. Morse]

By coincidence, before I read your comment above, I also got Chris to save some
wsj cookies on my machine.  I guess he's the only one around with a wsj account.

Scenerio three is important.  It will allow us to isolate this as a cookie
problem in 5.0 or a migration problem between 4.x and 5.0.  Could you get Chris
to install a 5.0 cookie on your mac and try this out?

Since you can't put this in the bug report, can you e-mail to me the 4.x mac
cookie file and the 5.0 one.

Comment 27

[pmock]

Update:
 - I emailed Chris's cookie to engineer.
 - I emailed request to Chris Saito asking him for his help.

Comment 28

[pmock]

I believe there is a problem with Mac seamonkey throwing away the contents of the
migrated cookie file.  I noticed that when I first migrate my profile from 4.7 to
5.0, it did indeed transfer the contents of the cookie file.  I did a file
compare between the 4.7 Magic cookie file and the 5.0 migrated cookies.txt file
using the BBedit 5.0 compare feature.  The files were identical.

Once I try to go to the Wall Street Journal site, it appears to throws away or
reset the contents to be empty except for the basic file header information,
because it prompts me the id and password. When I exit apprunner to re-examine
the cookies.txt file, the file contains only the following:

# Netscape HTTP Cookie File
# http://www.netscape.com/newsref/std/cookie_spec.html
# This is a generated file!  Do not edit.


Steve, while I'm waiting for Chris to respond to my request for help, could you
test what happens when you try to access the WSJ site with the migrated profile
in a debug build.  I have sent you a third cookie file that has been migrated
without accessing the wsj site.

Peter

Comment 29

[Stephen P. Morse]

Using the migrated cookie file that you sent me, and running a win32 build, I
had no trouble getting to the wall street journal site.  And my cookies.txt file
was intact when I exited the browser.

We really need to run scenerio three (on a mac) to understand more what is going
on.  If Chris doesn't have the time, then maybe you need to ask him for his
username/password so you can run the test yourself.

Comment 30

[pmock]

I haven't heard from Chris Saito yet.

We already know that this problem does not occur.  Could you try it on a Mac
debug build to see what is happening to the cookies.txt file to cause it to
throw away the contents?   Even when I try your suggestion, it does not explain
what the migrated cookies.txt information is being lost.

This does not appears to be a migration issue because the cookies.txt file is
being migrated correctly as far as I can tell.  It once I try to use the
cookies.txt file to access the WSJ web site that the problem occurs.

Comment 31

[pmock]

Excuse me, I meant to say "We already know that this problem does not occur on
Windows".

Comment 32

[pmock]

Chris Saito just stopped by.
In 5.0, after you are prompted for his username and password, we have the login
saved in a cookie.  The WSJ page did appear.  I checked the cookies.txt file
while apprunner was running and the file was NOT empty.  Once I exit apprunner,
it did save the changes.  I email you the cookies.txt file.  Problem: once I
restart apprunner and try to access the WSJ page, it re-sets or wipes the
cookies.txt file again.  This is the same behavior as using a migrated
cookies.txt file from 4.7 to 5.0.

Note: I do NOT have Chirs's password and he does not want to give it out.  He
suggested to expense a WSJ account?  If you still need a login account Steve,
could you expense one?  Chris indicated that he does not want to be the
bottleneck on this problem.  He said the expense could come out of his budget.

Comment 33

[Stephen P. Morse]

Is this problem unique to wall street journal.  In other words, can you visit
some other site that sets cookies (home.netscape.com is a good example) and see
if the cookies file gets wiped out when you exit the browser.

Comment 34

[Paul MacQuiddy]

It happens on netscape.com cookies also. The first time you write out to the
cookies file (set a cookie) the contents that were migrated get wiped out. If
you don't write a cookie out, the contents stay there, so the file looks ok in
an editor, but it still looks like 5.0 can't read it. I guess there is some
problem with the formatting of 4.x mac cookies files that 5.0 can't handle.

Steve, let me know if you need any more info.

Comment 35

[Stephen P. Morse]

I just ran some tests with Peter and here is what we learned.  This bug has
absolutely nothing to do with migrated cookies, nor does it have anything to do
with wsj.com.  What is happening (for any site) is that the mac browser is not
reading in the cookie file on startup and so it always starts with an empty
cookie list.

It should be pointed out that the browser updates the cookies.txt file whenever
it encounters a site that sets a cookie.  So at that point the browser will
write out a cookie list consisting of the cookie that it just set but not
consisting of any other cookies that were originally in the cookies.txt file
(since the browser failed to read them in).

I'm surprised there is no other bug filed on this (or did this just start
happening?).

Comment 36

[Paul MacQuiddy]

I think people are just starting to try migrating their profiles, and even if
they have tried, they haven't noticed (or maybe cared) that their cookies
weren't transferring over properly. I imagine this has been around since cookies
starting working on the mac, which was 2 or 3 months ago.

Comment 37

[Stephen P. Morse]

Paul, you are missing a key point here.  This has nothing to do with cookie
migration.  The browser fails to read in the cookie file whether it was
generated by migration or from a previous seamonkey run.  Certainly people who
run mac seamonkey do that every day.

Comment 38

[Paul MacQuiddy]

Okay, sorry, I didn't read your last comment carefully enough. Sounds like the
summary of the bug should be changed.

Comment 39

[Robert John Churchill]

So, here's what I am seeing on my Mac:

Some cookies, if they specify an expiration date far in the future, are passing
the signed/unsigned boundary for a "long" value (note: its a signed long).  This
causes cookies to be expired (i.e. discarded) before they are ever used.

Here's a one-liner attempt at fixing the problem:

in nsCookie.cpp, around line # 1985, change
from    new_cookie->expires = atol(expiresCString);
to      new_cookie->expires = strtoul(expiresCString, nsnull, 10);

I tried this on my Mac and cookies now appear to work much better for me. Steve,
you should review this change for other platforms.

You might want to consider changing to using PRTime values (which are 64 bit
values) and doing date/time comparisons with that level of precision.

While looking over this code, I also noticed that COOKIE_GetCookie() is called
for *every* HTTP request, whether its a HTML file, an image, whatever. A linked
list (i.e. a nsVoidArray) is being used to maintain the list of available
cookies. I'd recommend thinking about changing this to using a hash table, it
would be a big performance win when the user has more than a couple of cookies.

It would be nice if we could bypass sending cookies altogether for some things
(such as images.) Is that possible, or is it a protocol violation? I heard a
rumor of some sort of similar optimization in 4.x

Comment 40

[Stephen P. Morse]

Changing summary from:

   After migration, 5.0 fails to use saved cookies information

to:

   5.0 fails to read in cookies.txt file on mac

Comment 41

[Stephen P. Morse]

Thanks Robert, I owe you one.  I just tried your fix and it doesn't adversely
affect the win32 platform -- the cookies file is still being read in
successfully.  I will check this in as soon as the tree goes green.

Regarding images, yes they can and do set cookies.  That's how all these nasty
marketing sites track you -- they put their images up on other companies sites.

The use of PRTime is a good idea and so is the hash.  I'll consider them
for future enhancements.

Comment 42

[Stephen P. Morse]

Fix to nsCookie.cpp checked in.  This should fix the mac problem.

Comment 43

[Grace Bush]

Steve,

I tried to verify this bug, but saw the same behavior on the 1999112108 build. I
was unable to access the NYTimes puzzles as I can in 4.7.  I retested after
setting more cookies in 4.7, migrated a profile to 5.0, the 5.0 cookie file
looked just like the 4.7 cookie file.  I then browsed some sites-amazon and
yahoo maps to be specific and now my cookie file only contains the amazon
cookies. The rest of the cookies were removed.

Comment 44

[Suresh Duddi (gone)]

Robert, steve is on vacation. Could you help out with this one.

Comment 45

[Robert John Churchill]

I'll try and take a look at it.

Comment 46

[Robert John Churchill]

This is working for me... I can use my.yahoo.com & www.amazon.com via my
cookies. The cookie file is read in correctly on my Mac, I can add new cookies,
and they work they next time I run mozilla.

gbush, can you try again with the latest build?  If it does NOT work for you,
please do this:

run mozilla, go to the site where cookies don't seem to be working for you, log
in (so that a cookie is set), quit mozilla, and then attach your cookie file to
this bug.

Comment 47

[Grace Bush]

Attachment: cookies.txt for build 1999112308

Comment 48

[Grace Bush]

see attachment for  cookies tested today-not working, I setup 4.7 cookies for my
amazon and mynetscape accounts as well as nytimes and when visiting these sites
in 5.0 I had to sign in again, they did not recognize me

Comment 49

[Robert John Churchill]

After you ran 5.0 once, signed into a site, and quit 5.0, could you re-run it
and go back to the site without having to resign in again?

Comment 50

[Grace Bush]

attempted to sign on to two cookie sites and relaunch to see if cookies were
maintained by 5.0.  Unable to signon at either amazon or mynetscape (see bugs
20055 and 20056)

Comment 51

[Tom Everingham]

Verified working on Mac build 1999112908.  Loaded http://scopus/bugsplat
and logged in.  Exited application.  Verified cookie file contained
login name and password.  Re-logged on to site without being prompted
for name & password.

Comment 52

[pmock]

Oh no...the original scenario that I reported still occurs. The 4.7 MagicCookie
migrated fine to 5.0.  Once I tried to access the Wall Street Journal site
(http://interactive.wsj.com/edition/current/summaries/front.htm), it failed
again. It re-set or wiped out the cookies.txt file.

I tried it on two different commercial seamonkey builds.  1999-12-01-13-m12 and
1999-12-03-08-m12.

Re-openning bug.

Comment 53

[pmock]

Clearing resolution.

Comment 54

[Stephen P. Morse]

Could you please separate out the issue of cookie migration from cookie
persistence.  That seems to be confusing this bug report throughout.

Specifically, start with an empty cookie file, visit wsj, login and allow the
cookie to be set, and then exit the browser.  Then reenter the browser and
return to wsj.  Do you need to log in again or can you get in with your old
cookie.  If you need to log in again, then the problem is cookie persistence and
not cookie migration and we should stop talking about migration in this report.

Comment 55

[Grace Bush]

Steve,

I think this bug started with cookie migration and ended up with cookie
persistence.  I believe the cookie persistence works ie you can set a cookie in
5.0 and use it on subsequent launches.  What Peter and I cannot do (I test the
Profile Migration feature) is migrate a 4.7 cookie file and log into sites where
the cookie is required in 5.0 - but only on the Mac.  I do not have this problem
on Win32 or Linux. I see the problem is that after the cookie file is migrated
that wsj or nytimes does not does not allow access on Macs- for ???
Hope that helps,
Grace

Comment 56

[pmock]

Uh Steve,

I originally logged this bug against migration.  On 11/18/99 20:12 you changed
the summary from:
   After migration, 5.0 fails to use saved cookies information
to:
   5.0 fails to read in cookies.txt file on mac

My original steps to reproduce this bug involved migration.  I fail to see how
having me copy the original text and original summary description to a new bug
will help speed up resolving this bug.  It will only delay resolving this bug
since it will need to be re-approved for PDT+ again.

Comment 57

[Stephen P. Morse]

I'm not asking you to coy this to a new bug report.  I'm trying to understand if
we have a migration problem or a persistence problem.  And from the comments
that you and grace just made, I'm still not sure of the answer to that question.

Comment 58

[pmock]

Sorry, cookies is not an area that I normally test in mail/news.  I can tell you
this much.  If I manually copy the wsj cookie information from 4.7 to my 5.0
cookie file, the same results occur as when I migrate the profile my 4.7 to 5.0.

This behavior is bad because it not only removing the wsj cookie information but
throwing alway all the other contents in the cookie file.

So maybe it's not a migration issue but more an issue of 5.0 reading the cookie
information that was brought over.  Truly, I do not know if this is a migration
problem or a persistence problem.  I can't answer the question.

Changing the qa assigned from gbush to tever since he verify the bug before.

Comment 59

[Robert John Churchill]

Steve, I'm bouncing this bug back to you.  (This works on my Mac... so perhaps
you should take a look at it on gbush's machine.)

Comment 60

[Grace Bush]

results of latest tests per Steve
Test1. in 4.x visit a site, set a cookie, save a backup of cookie file
   migrate to 5.0 and view cookie file (edit/wallet/display cookies)
 RESULT: there is nothing to see in cookie viewer
                  -------
(4.7 and 5.0 cookie files look  the same outside seamonkey)

Test2. Start up 5.0 with new profile
       visit site and set cookie (same site as in 4.7-->5.0 migration)
       make backup of cookie file
       exit, relaunch and revisit cookie site/view with cookie viewer
   RESULT: cookie is viewable, site is accessible

Steve, backups of cookie files are on my test machine should you need, but they
are same as discussed.

Comment 61

[Stephen P. Morse]

Thanks Grace.  Now can you attach those two cookies files (the migrated 4.7->5.0
one and the one that was created with 5.0) to this report so I can examine
them.

Comment 62

[Grace Bush]

Attachment: 4.7->5.0 migrated cookie file

Comment 63

[Grace Bush]

Attachment: 5.0 cookie file - created in 5.0

Comment 64

[Stephen P. Morse]

I don't understand these attachments.  Neither one is a valid cookie file.  They
don't have any line feeds at the end of each cookie.  Nor do they have tabs
separating the fields.  You must have done some unintentional reformatting on
them before attaching them.

Could you please e-mail me the raw cookie files exactly as they exist on your
machine.  Thanks.

Comment 65

[Suresh Duddi (gone)]

That attachments are alright steve. When we click on them, we see it as HTML as
opposed to txt. Do view source on the attachment to see a nice pretty cookie
file.

Comment 66

[Stephen P. Morse]

Even after doing a view source, the file is still not right.  Granted I get the
line feeds this time.  But in both files the fields within each cookie are
separated by spaces whereas they need to be separated by tabs to be valid cookie
files.

To prove the files are invalid, I copied the attached 5.0 cookies file (the one
that worked on the mac) and put it into my profile (on my win32 machine).  Then
I started the browser and went to the cookie viewer.  None of the cookies were
displayed.

So that's why I'm asking to be e-mailed the original cookies file for the two
cases.  Hopefully they will still have the original formatting so I can examine
them and even run with them.

Comment 67

[Suresh Duddi (gone)]

I see. Alright. Work with testing on narrowing down the problem. Enlist Robert's
help if you need. Robert, I am ccing you on this bug. Let us talk to testing
more folks.

Comment 68

[Stephen P. Morse]

Robert, can you duplicate the test that q/a is doing and see if you get the
same results.  That is, collect some cookies with 4.x, migrate the cookie file,
and then use it in 5.0.  When you do, are the cookies indeed read in by 5.0?
You can use the cookie viewer (edit/wallet/display-cookies) to determine this.
Thanks.

Comment 69

[Robert John Churchill]

I moved a large cookie file from 4.7 over to 5.0 and it is read in just fine. I

can see them all.



However, it looks like most, if not all, of the cookies are still being expired

prematurely.  Thus, it gives the appearance after you quit out of Mozilla that

the cookie file has been substantially trimmed, as well as giving the appearance

that you have to re-login in to various sites.



I'm guessing that the problem here is with the usage of  time(null)  in various

spots of the cookie code to get the current time.  Why am I guessing that?  Well,

because on non-Mac platforms, the epoch is on Jan 1, 1970.  However, on Mac, the

epoch is on Jan 1, 1904.  So, using  time(null)  can give a much larger value

(depending on how CodeWarrior's support libraries have implemented  time(null)

and since the cookie code basically expires the cookie if



cookie_expire_time  <  time(null)



you can see how this could be the case if  time(null)  is significantly larger.



Steve, I strongly recommend that you drop all usage of  "time_t"  and  time(null)

in favor of using  "PRTime",  PR_Now(),  and the various LL_  comparison

operators...  by doing so, you get 64-bit precision, and the PRTime stuff is

extensively used throughout the browser, so you'd be much more likely to get what

you expect on every platform.

Comment 70

[Stephen P. Morse]

Robert,

I'm a bit confused by your comment.  On the one hand you said that the migrated
cookie file "read in just fine.  I can see them all."  But then you added that
"most if not all of the cookies are being expired prematurely."   It sounds like
those two statements are contridictory.  If you could see them after they
were read in, they weren't being expired.  And, similarly, if they were being
expired, you wouldn't see them after they were read in.  Could you please
clarify.  Thanks.

Comment 71

[Robert John Churchill]

Once you go to a site that requests the cookie, it is expired (prematurely)
instead of being used.

Comment 72

[Stephen P. Morse]

OK, I see what's going on.  Robert's comment gave me the clue.  Take a look at
the cookies in the two files that Grace Bush uploaded.  The files are *not*
identical.  In fact they differ considerably in the expiration times.
Specifically, the nytimes cookies expire as follows:

created in 4.x: expire = 1,133,831,367 seconds = 36 years
created in 5.0: expire = 3,342,791,376 seconds = 106 years

The only way this would make sense is if the 4.x cookies were counting seconds
from 1-1-1970 and the 5.0 cookies were counting seconds from 1-1-1900 -- in that
case both are saying that the cookie is to expire in the year 2006.

So what's happening is the following.  In 4.x the mac cookie file used
expiration times starting from 1900 whereas all the other platforms started from
1970.  In 5.0 it was made cross platform so that all platforms use expiration
times starting from 1970.  That means that mac cookies generated in 4.x cannot
be migrated to 5.0 as is -- instead the expiration time must first be reduced by
the number of seconds between 1-1-1900 and 1-1-1970.  That number can be
computed as follows:

  70 years * 365 days/year * 86,400 secs/day =     2,207,520,000 seconds
+ 17 leap years * 86,400 additional sec/leapyear =     1,468,800 seconds
=                                                  2,208,988,800 seconds

So the correct thing to do is to modify the mac migration program to subtract
this number from the cookie expiration times.  Is this an easy thing to do?
Where is the source of the migration code?

Comment 73

[Stephen P. Morse]

By the way, my analysis indicates a difference of 70 years which indicates
that the start of epoch began 1-1-1900.  But Robert's comment said that the
start was 1-1-1904.  Are you sure of that.  Looking at
mozilla/prpub/pr/src/misc/prtime.c I find the following comment:

 * The mktime() routine in MetroWerks MSL C
 * Runtime library returns seconds since midnight,
 * 1 Jan. 1900, not 1970.  So we need to adjust
 * its return value to the NSPR epoch.

That agrees with my analysis of 70 years.

Comment 74

[Stephen P. Morse]

Ignore my question -- I found the migration code.  It's in
mozilla/profile/pref-migrator/src/nsPrefMigration.cpp.  I'm looking into it
right now and will come up with a fix.

Comment 75

[Stephen P. Morse]

OK, here's an outline of the coding change that should be made to
nsPrefMigration.cpp.  In DoSpecialUpdates there currently exists the
following code to rename the cookie file.

  rv = ename4xFileAfterMigration(...);
  if (NS_FAILED(rv)) return rv;

That should be changed to the following:

#if defined(XP_MAC)
  open COOKIES_FILE_NAME_IN_4x as input file
  open COOKIES_FILE_NAME_IN_5x as output file
  loop until end of input file {
    read next line from input file
    if column 1 = "#" continue
    get integer between fourth and fifth tab on the line
    subtract 2,208,988,800 from that integer
    write line to output file
  }
  close input file
  close output file
#elif
  rv = ename4xFileAfterMigration(...);
  if (NS_FAILED(rv)) return rv;
#endif

Seth, I see that you are the one who has written this area of the file.  Do you
want to make and test this change?

Comment 76

[Robert John Churchill]

Mac ROM's definitely have an epoch of Jan 1, 1904.  For the longest time, until
Apple started using 64-bit dats, the Mac only understood dates from 1904 until
2040.

However, MetroWerks MSL C can certainly be written to do something different.

Anyway, glad to see the problem being hammered out.  :^)

Comment 77

[Robert John Churchill]

Note:  If/when you use PRTime et.al. you'll want to see what epoch it uses and
take that into account.

Comment 78

[Stephen P. Morse]

Seth, don't bother writing the new code for the migration tool -- I'm already
doing it.  But I'll probably be giving to you to code review and test for me
later today when it's done.

Comment 79

[Stephen P. Morse]

I have the fix completed and ready to be checked in.  Waiting for a code review
from Seth.

Comment 80

[(not reading, please use [e-mail] instead)]

morse's code is checked in, but not turned on yet until I finish testing it.

so far, it seems to be working.

I expect to turn it on before the end of the day, and we can mark this bug
fixed.

Comment 81

[(not reading, please use [e-mail] instead)]

ok, steve's fix is now turned on.

marking this fixed.

Comment 82

[(not reading, please use [e-mail] instead)]

fyi, your logic is backwards.

if 4.x cookie times are relative to 1970
and 5.0 cookie times are relative to 1900

you need to add 70 years of seconds to a 4.x cookie time, not subtracted.

there was another problem as well.  the code in nsCookie.cpp expects the line
endings in the cookies.txt file to be '\n' or '\r\n'.  this works fine for
migrate linux and windows cookie files, but not for mac, which would be '\r'.

I've fixed the migration code in nsPrefMigration.cpp to fix the expire times (by
adding 70 years of seconds) and by fixing the line endings.

Comment 83

[Stephen P. Morse]

No, my logic was not backwards.  The 4.x mac cookies are relative to 1900 and
the 5.0 cookies are relative to 1970.  That's the opposite of what you just
said.  So the subtraction (rather than addition) was the correct thing to do.

Comment 84

[(not reading, please use [e-mail] instead)]

yes, you are right.  I'll switch it back to -= instead of +=

thanks.

Comment 85

[(not reading, please use [e-mail] instead)]

ok, it's switched back now, and 4.x prefs migrate properly on the mac.

tested with the wsj cookies.

Comment 86

[Stephen P. Morse]

Seth,

Thanks for all your help on this.

Comment 87

[Grace Bush]

this looked good on both 19991207 and 19991208 builds on the Mac....
tever, do you want me to mark verified?

Comment 88

[Grace Bush]

verified with 1999121508 builds Wall Street Journal cookie!