74603 - Append isn't working from script

Status: RESOLVED FIXED

(Core :: XPCOM, defect, P3)

Description

[Pete Collins (MDG)]

To reproduce on unix:

$ echo this is line \#1 > /tmp/foo

then run the attached script in xpcshell

$ js test.js
Type Manifest File: /usr/src/MOZILLA/mozilla/dist/bin/components/xpti.dat
nsNativeComponentLoader: autoregistering begins.
nsNativeComponentLoader: autoregistering succeeded
nNCL: registering deferred (0)

/tmp/foo
exists=true

size=16

Comment 1

[Pete Collins (MDG)]

Attachment: Test case

Comment 2

[Pete Collins (MDG)]

Attachment: Cleaned up the test case a bit more to make it easier to read.

Comment 3

[Pete Collins (MDG)]

Running it we have:

$ js test.js
Type Manifest File: /usr/src/MOZILLA/mozilla/dist/bin/components/xpti.dat
nsNativeComponentLoader: autoregistering begins.
nsNativeComponentLoader: autoregistering succeeded
nNCL: registering deferred (0)

/tmp/foo

exists=true


size=16


CanUnload_enumerate: skipping native
Mon Apr 16 06:37pm
root@ool-18bc22b1
 /usr/src/MOZILLA/mozilla/dist/bin  $ cat /tmp/foo
this is line #1
Mon Apr 16 06:38pm
root@ool-18bc22b1
 /usr/src/MOZILLA/mozilla/dist/bin  $

Comment 4

[Pete Collins (MDG)]

Doesn't work on windows either . . .

--pete

BASH.EXE-2.04$ ./xpcshell.exe test.js

c:\temp\foo

exists=true


BASH.EXE-2.04$

Comment 5

[Keyser Sose]

Marking NEW.

Comment 6

[Doug Turner (:dougt)]

mass setting milestone.  if this is something you believe is more urgent then 
the milestone that I just sent, please send me mail.

Comment 7

[Pete Collins (MDG)]

Doug, the actual writing takes place in nsFileOutputStream::Write method.

Doing some testing i found that in append mode PR_Write(mFD, buf, count); fails
returning -1.

The file descriptor mFD is a valid. I see it is initialized in
nsLocalFile::OpenNSPRFileDesc to a PR_Open(mPath, flags, mode); call.

mPath is valid, 
flags are correct (16)
mode doesn't matter

I can't figure out why PR_Write is failing . . .

PR_GetError=673160688

But how do i inturpret this nspr error code?

Any ideas??

--pete


 

Comment 8

[Pete Collins (MDG)]

Ok, i figured this out the hard way. 
I have to create the transport like so.

JSFILE_NS_CREATE_FILE | JSFILE_NS_WRONLY | JSFILE_NS_APPEND

This certainly doesn't seem to adhere to any conventional logic that i know of.
But what do i know . . .

This is more of an inspiration for me to finish with jslib so a developer like
myself doesn't have to spend days upon days just to open a local file in append
mode.

It really seems that mozilla goes out of it's way to making the most simple of
tasks complex and convoluted. 

Most every language i know of uses an fopen type of interface.

fopen("file/path", "a");

Why one earth nspr totally breaks this convention type i don't know.

PR_Open("some/path", PR_CREATE_FILE | PR_WRONLY | PR_APPEND);

What the fuck is that?? 

--pete

Comment 9

[Pete Collins (MDG)]

Ok, like i said this works fine *on unix*.
Doesn't work on windows. Haven't tested mac yet.

It just keeps overwriting all of the content.

--pete

Comment 10

[Pete Collins (MDG)]

Attachment: another test case

Comment 11

[Pete Collins (MDG)]

Ok, if i run attachment id=38340 i get entirely different behavior on windows
and unix.

Windows:

administrator@LAPTOP /usr/src/package
$ cat //c/tmp/foo.dat
this is an original line

administrator@LAPTOP /usr/src/package
$ js test.js

c:\tmp\foo.dat

exists=true


size=25



administrator@LAPTOP /usr/src/package
$ cat //c/tmp/foo.dat
this is a new line #1
this is a new line #2
this is a new line #3



Unix:

Wed Jun 13 09:19pm
root@seals
/usr/src/mozilla/dist/bin $ cat /tmp/foo.dat
this is an original line
Wed Jun 13 09:19pm
root@seals
/usr/src/mozilla/dist/bin $ js test.js
Type Manifest File: /usr/src/mozilla/dist/bin/components/xpti.dat
nsNativeComponentLoader: autoregistering begins.
nsNativeComponentLoader: autoregistering succeeded
nNCL: registering deferred (0)

/tmp/foo.dat

exists=true


size=25


CanUnload_enumerate: skipping native
Wed Jun 13 09:19pm
root@seals
/usr/src/mozilla/dist/bin $vi /tmp/foo.dat

this is an original line
this is a new line #3
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@

this is running the same exact script on two different platforms.

Is this an NSPR problem? BTW, a few months ago, append worked fine. So something
has changed from then to now. I beleive it is NSPR changes.

--pete

Comment 12

[Wan-Teh Chang]

PR_Open() is modeled after the Unix open() system call.
(See http://www.opengroup.org/onlinepubs/007908799/xsh/open.html.)
The Windows CreateFile() interface is not any simpler.
(See http://msdn.microsoft.com/library/psdk/winbase/filesio_7wmd.htm.)
fopen() is a C library function, which is why it has a
simpler interface than system calls.

The problem you reported is a known bug in the version
of NSPR (4.0.2 Beta) that Mozilla client is using.  Prior
to NSPR 4.1, PR_APPEND only works on Unix.  See bug #4090.
This bug was fixed in NSPR 4.1.  See Section 2.2 "Bug Fixes"
in the NSPR 4.1 Release Notes
(http://www.mozilla.org/projects/nspr/release-notes/nspr41.html).

Assuming you don't move the file pointer between writes,
your workaround is to open the file without PR_APPEND,
immediately seek to the end of file, and then do your writes.
If you want your code to automatically switch to using
PR_APPEND when Mozilla client upgrades to NSPR 4.1, you
can do this:

#include "prinit.h"  /* for NSPR version info */

#if PR_VMAJOR > 4 || (PR_VMAJOR == 4 && PR_VMINOR >= 1)
    /* NSPR 4.1 or newer.  PR_APPEND works. */

    PRFileDesc *fd = PR_Open(some/path, ... | PR_APPEND);
    /* do your writes */
#else
    /*
     * PR_APPEND does not work.  Emulate it by seeking to
     * the end of file after opening.
     */
    PRFileDesc *fd = PR_Open(some/path, ...);  /* no PR_APPEND */
    PR_Seek(fd, 0, PR_SEEK_END);
    /* do your writes, assuming you don't move the file pointer */
#endif

This is yet another reason Mozilla client should upgrade
to the latest NSPR release.  See bug #78471.  You can
experiment with NSPR 4.2 Beta by editing mozilla/client.{mk,mak}
and changing NSPRPUB_CLIENT_BRANCH to NSPRPUB_CLIENT_TAG.

Comment 13

[Wan-Teh Chang]

This is a better way to describe the workaround:

#include "prinit.h"  /* for NSPR version info */

#if PR_VMAJOR > 4 || (PR_VMAJOR == 4 && PR_VMINOR >= 1)
    /* NSPR 4.1 or newer.  PR_APPEND works. */

    PRFileDesc *fd = PR_Open(some/path, ... | PR_APPEND);
#else
    /*
     * PR_APPEND does not work.  Emulate it by seeking to
     * the end of file after opening.
     */
    PRFileDesc *fd = PR_Open(some/path, ...);  /* no PR_APPEND */
    PR_Seek(fd, 0, PR_SEEK_END);
#endif

     /* do your writes, assuming you don't move the file pointer */

The writes don't need to be inside the ifdef.

Comment 14

[Doug Turner (:dougt)]

john, I think that you are looking at this already.

Comment 15

[John Taylor]

retargeting. I may just wait for the new NSPR to be landed. Any objections?

Comment 16

[John Taylor]

cleaning bug list

Comment 17

[Pete Collins (MDG)]

This *should* theoretically be fixed now right? 

I beleive NSPR 4.1 landed sometime last week.

I don't have a windows box so i can't confirm.

--pete

Comment 18

[Pete Collins (MDG)]

Actually i'll take this and will confirm fixed when i'm finished w/ the nsIFile
reorg.

--pete

Comment 19

[Wan-Teh Chang]

Yes, the version of NSPR that Mozilla is using implements
PR_APPEND on all platforms.

Comment 20

[Pete Collins (MDG)]

Marking fixed as to last NSPR landing. I will reopen if there is a problem.

--pete