232877 - Static build doesn't move sec/xpins *.js files->./defaults/pref

Status: RESOLVED WORKSFORME

(Core :: Preferences: Backend, defect)

Description

[Mickey Stein]

User-Agent:       
Build Identifier: Mozilla Thunderbird 0.5a (20040201)

This is true of linux static thunderbird builds (Jan-31, Feb-01). I've been
tracking my progress (or lack of it ) in this mozillazine thread:

http://forums.mozillazine.org/viewtopic.php?t=47848

Scott M. suggested I bug this in this category, although it really doesn't occur
on linux static firebird builds, but figuring out the categories is always kind
of puzzling.



And here's a recap. If I build dynamic thunderbird (say today's (02Feb) ) then
it works fine and uname/pwd are recalled automatically for usenet groups. If I
build the same CVS statically, then uname/pwd are forgotten each time for usenet
groups. This has actually been addressed by Scott before by adding some prefs
into thunderbird.js. 

Here's what appears to be happening: 

1) build dynamic thunderbird, which works, and you get these files in:
./{tb-path)/defaults/pref:
all.js       editor.js    mdn.js             smime.js        unix.js
composer.js  mailnews.js  security-prefs.js  thunderbird.js  xpinstall.js

2) build static thunderbird, which forgets uname/pwd, and you get these files in
./{tb-path}/defaults/pref:
all.js       editor.js    mdn.js             smime.js        unix.js
composer.js  mailnews.js  thunderbird.js  

So static is missing security-prefs.js and xpinstall.js.

3) adding them in manually will cause static tbird to work fine and remember
passwords.

In xpinstall & netwerk subdirs, are xpinstall.js & security-prefs.js
respectively that're both not installed during a static build. They're told to
be included in *.pkg files in each tree under a subheading of [gecko-support].
[gecko-support] appears to be directed from this file
"modules/libpref/src/libpref.pkg" which *does* have some #ifdef static logic.
Now the trail grows colder.

the libpref.pkg file is only referred to from the CVS (I really mean
CVS-related) files, so this seems like a deadend for me. I'm wondering what'll
happen if they're not a part of [necko-support] but rather [necko] in the *.pkg
files. This seems worth a try since for whatever reason, they're not being
pushed up into dist/bin/defaults/pref like the [necko] files are during a static
build. Ok, I tested pushing them into [necko] and it then places them in the
correct place, but this isn't the way to do it. It's just happening at some
level (probably a static macro in ./mozilla/config) that I can't detect. 


Reproducible: Always
Steps to Reproduce:
1. Use this .mozconfig to build tbird static (any cvs from Feb is fine)
export MOZ_THUNDERBIRD=1
mk_add_options MOZ_THUNDERBIRD=1
ac_add_options --enable-crypto
ac_add_options --disable-mathml
ac_add_options --disable-installer
ac_add_options --disable-activex
ac_add_options --disable-activex-scripting
ac_add_options --disable-tests
ac_add_options --disable-oji
ac_add_options --disable-necko-disk-cache
ac_add_options --disable-debug
#ac_add_options --enable-debug
ac_add_options --disable-profilesharing
ac_add_options --enable-image-decoders=png,gif,jpeg,bmp
ac_add_options --enable-extensions=wallet,spellcheck,xmlextras
ac_add_options --enable-necko-protocols=http,file,jar,viewsource,res,data
ac_add_options "--enable-optimize=-O3 -march=athlon-xp -msse "
ac_add_options --enable-xft
ac_add_options --enable-freetype2
ad_add_options --enable-static
ad_add_options --disable-shared



2. cd dist/bin, run it and open newsgroups. It'll have forgotten any uname/pwd
entered previously, and will continually do so.

3. Look in ./dist/bin/defaults/pref and note the missing *.js files mentioned
before. 

Actual Results:  
Uname and pwd for usenet are never remembered past a single look at a usenet msg. 

Expected Results:  
uname and pwd should be rememebered regardless of invokations. (As it works in
dynamic builds).

OS: FC1 (latest development): Kernel: 2.6.2-mm3
glibc-2.3.3-7, gcc-3.3.2-8.

NF7 mobo, Athlon XP

Comment 1

[Scott MacGregor]

cc'ing bsmedberg, Ben, do you think the recent pref re-org could have caused a
bug in static builds which would keep xpinstall.js and security-prefs.js from
getting copied over to defaults/pref?

Comment 2

[Benjamin Smedberg]

Yeah, I probably caused this; I pretty much never build static. Though I have
trouble understanding how a static build would affect things. In any case, I'm
poking around.

BTW, the *.pkg files are not an issue... I've landed them in preparation for
landing the rest of the packaging branch (see bug 20640), but I'm not going to
be able to land the rest of that patch until after 1.7a.

Comment 3

[Benjamin Smedberg]

> 1) build dynamic thunderbird, which works, and you get these files in:
> ./{tb-path)/defaults/pref:
> all.js       editor.js    mdn.js             smime.js        unix.js
> composer.js  mailnews.js  security-prefs.js  thunderbird.js  xpinstall.js
> 
> 2) build static thunderbird, which forgets uname/pwd, and you get these files in
> ./{tb-path}/defaults/pref:
> all.js       editor.js    mdn.js             smime.js        unix.js
> composer.js  mailnews.js  thunderbird.js  

This cannot be correct. This is what the prefs dirs *should* look like:
{tb-path}/greprefs/xpinstall.js
{tb-path}/grepregs/security-prefs.js
{tb-path}/greprefs/all.js
{tb-path}/defaults/pref/thunderbird.js
{tb-path}/defaults/pref/smime.js
{tb-path}/defaults/pref/mdn.js
{tb-path}/defaults/pref/mailnews.js
{tb-path}/defaults/pref/composer.js

And they should be read in the above order (GRE prefs, reverse alphabetically,
then app prefs, reverse alphabetically).

Comment 4

[Mickey Stein]

#3: I don't doubt that's how the directories *should* appear, and in fact, those
missing pref files are where you say they are. However: On any dynamic build,
those 2 missing files from defaults/pref *are* there, and that's the only place
where they actually work from. That may be wrong or a mistake or a code error,
but that's how the symptoms play out.

To go from a nonworking static build of tb to a working build you can move those
2 files from where you said to defaults/pref and you have no more security
issues with usenet. If you leave them in greprefs (for static), they're never
accessed or used correctly.

Comment 5

[Mickey Stein]

Sorry -- I entered that last note before I was ready :) -- Anyway: From what Ben
said, it sounds as if greprefs isn't actually read, so the fact that the files
wind up on a dynamic build in defaults/pref appears to be why the dynamic build
works. 

I'll search through the cvs and see where greprefs/*.js is read from (or
possibly is supposed to be, or at least where defaults/pref/*.js is read from.

Comment 6

[Benjamin Smedberg]

> On any dynamic build,
> those 2 missing files from defaults/pref *are* there, and that's the only place

Let's call this "problem 1". They should not be there. Can you help me find out
where in the build process they are getting installed to this incorrect location?

> If you leave them in greprefs (for static), they're never
> accessed or used correctly.

Let's call this "problem 2". If you have a debug static build, do you get spew
from the pref loader (NS_ENSURE_* failed, or assertions)?  I'm gonna try
building a static build sometime today/tomorrow to debug whether NS_GRE_DIR is
not being picked up correctly (I'm almost positive it's being set, because
otherwise all sorts of interesting things would stop working, like system DTDs
and ua.css).

Comment 7

[Benjamin Smedberg]

> I'll search through the cvs and see where greprefs/*.js is read from (or
> possibly is supposed to be, or at least where defaults/pref/*.js is read from.

see http://lxr.mozilla.org/mozilla/source/modules/libpref/src/nsPrefService.cpp#719

Comment 8

[Mickey Stein]

 On any dynamic build,
>> those 2 missing files from defaults/pref *are* there, and that's the only >>place

>Let's call this "problem 1". They should not be there. Can you help me find out
>where in the build process they are getting installed to this incorrect >location?

I can sure try. I spent an hour or two trying to figure that out yesterday and
the closest I could get was the two .pkg files under the [gecko-support] clause
before the trail appeared to end. I'll continue to look.

>> If you leave them in greprefs (for static), they're never
>> accessed or used correctly.

>Let's call this "problem 2". If you have a debug static build, do you get spew
>from the pref loader (NS_ENSURE_* failed, or assertions)?  I'm gonna try
>building a static build sometime today/tomorrow to debug whether NS_GRE_DIR is
>not being picked up correctly (I'm almost positive it's being set, because
>otherwise all sorts of interesting things would stop working, like system DTDs
>and ua.css).

I'm just now building a debug static tb. On 512mb, building + running that
particular build becomes a swapfest challenge match :) I'll reply back here
after I build, reboot to purge virtual and run the thing.

thanks for the pointer to nsPrefService

Comment 9

[Mickey Stein]

WARNING: malformed pref file, file prefread.cpp, line 243
[New Thread 32769 (LWP 32445)]
[New Thread 16386 (LWP 32446)]
************************************************************
* Call to xpconnect wrapped JSObject produced this error:  *
[Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED)
[nsIPrefBranch.getIntPref]"  nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" 
location: "JS frame ::
file:///d2/src/firebird/mozilla/dist/bin/components/offlineStartup.js ::
anonymous :: line 61"  data: no]
**************************************************

--- 

This is what the debug session starts with, but my experience is embyonic, so if
you could give me some help, it'd be greatly appreciated. 

I know how to set a basic breakpoint for the initially loaded modules (Like
"break main") but I don't understand the sequence of commands to load a forward
reference (or something that isn't yet loaded, like nsPrefService.cpp) and how
to set the breakpoint inside of that at pref_InitInitialObjects. 

It seems easy if I just hit <ctrl-c> from within the running gdb session, and do
"break nsPrefService", then "break pref_InitInitialObjects", but that only sets
them because they're been loaded and passed by. 

Anyway: If you could tell me what is probably really old news to you, but
something I'm having a bad time getting to work (because of error messages like
'can't do that in a multi-threaded program), it'd help.

Thanks.

Comment 10

[Benjamin Smedberg]

(In reply to comment #9)
> WARNING: malformed pref file, file prefread.cpp, line 243

OK, wanna look through your pref files and see if there are malformed files (for
example, preprocessor directives not stripped out)?

> I know how to set a basic breakpoint for the initially loaded modules (Like
> "break main") but I don't understand the sequence of commands to load a forward
> reference (or something that isn't yet loaded, like nsPrefService.cpp) and how
> to set the breakpoint inside of that at pref_InitInitialObjects. 

well, since it's a static build you ought to be able to breakpoint it
immediately. For a dynamic build see
http://www.mozilla.org/unix/debugging-faq.html you want
export XPCOM_BREAK_ON_LOAD=pref

Comment 11

[Mickey Stein]

That page on mozilla.org for debugging was just what I needed (well, part of
it), so thanks again.

Here's what I've come up with so far, and I've got to do this in bits and pieces
as the debugger chews up way too much virtual mem to actually do anything.
Typing in this message is downright glacial. 

1) The path NSGREDIR is ok and points to ./greprefs.
2) the prefs files xpinstall.js and security-prefs.js read from greprefs are ok.
3) the file greprefs/all.js is incorrect in some way I don't yet see so I'm
going to attach it here. 

After I made it and decided that was the deal, I double-checked this by copying
the all.js from today's dynamic version over it which made it work fine. 

I see there's ~4-5 version of all.js scattered through the code, and I'm
guessing that ./mozilla/modules/libpref/src/init/all.js is the one being pushed
through into greprefs. Clearly there's a transformation of whatever is in it. 

The all.js that works from the dynamic build's greprefs is 11k, and the one that
doesn't work from the static builds greprefs is 51K. I'll attach them both
because your name pops up in all.js so you'll have a much better chance than I
of figuring out where/how it's generated and what happened. 

** I guess I've got to attach it in the next comment **

Comment 12

[Mickey Stein]

Attachment: This is the 'working' version of all.js in greprefs

This is the 'working' version of all.js from greprefs as generated by the
dynamic build. 

The next attachment will be the non-working version of greprefs as generated by
the static tb build.

Comment 13

[Mickey Stein]

Attachment: The non-working all.js

This is the non-working all.js from static bld.

Comment 14

[Benjamin Smedberg]

Can we close this bug? If it was anything, I doubt it is that any more...
tinderboxen and my builds look fine.