Closed Bug 81373 Opened 23 years ago Closed 22 years ago

make mac CFM static build

Categories

(SeaMonkey :: Build Config, defect)

Product:
SeaMonkey
Component:
Build Config
Platform:
PowerPC
All
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED WONTFIX
Milestone:
mozilla1.1alpha

People

(Reporter: cathleennscp, Assigned: sfraser_bugs)

References

Details

(Keywords: embed, topembed-)

Attachments

(2 files, 12 obsolete files)

nsStaticComponents.cpp gererated during unix build
8.37 KB, text/plain
Details
Current state of my build scripts. With some MakeALias hacking, this should almost work.
13.51 KB, patch
Details | Diff | Splinter Review
Blocks: 46775
Here is a break down list of libs required per install option (mac mozilla)

minimal xpcom lib (required by installer)
----------------------
viewer:Components:libjar.shlb 
viewer:Components:xpinstall.shlb 
viewer:Components:unicharutil.shlb 
viewer:Essential Files:JavaScript.shlb 
viewer:Essential Files:libreg.shlb 
viewer:Essential Files:LiveConnect.shlb 
viewer:Essential Files:NSPR20.shlb 
viewer:Essential Files:NSRuntime.shlb 
viewer:Essential Files:NSStdLib.shlb 
viewer:Essential Files:oji.shlb 
viewer:Essential Files:xpcom.shlb 
viewer:Essential Files:xpistub.shlb 
viewer:Essential Files:zlib.shlb 


browser lib
----------------------
viewer:Essential Files:gfx.shlb 
viewer:Essential Files:libimg.shlb 
viewer:Essential Files:libutil.shlb 
viewer:Essential Files:RaptorShell.shlb 
viewer:Essential Files:widget.shlb 
viewer:Essential Files:NSSckbi.shlb
viewer:Components:docshell.shlb 
viewer:Components:AppShell.shlb 
viewer:Components:Caps.shlb 
viewer:Components:chardet.shlb 
viewer:Components:ChomeRegistry.shlb 
viewer:Components:content.shlb
viewer:Components:Cookie.shlb 
viewer:Components:appcomps.shlb 
viewer:Components:dom.shlb 
viewer:Components:EditorCore.shlb 
viewer:Components:EditorTxmgr.shlb 
viewer:Components:EmbedComponents.shlb
viewer:Components:FindComponent.shlb 
viewer:Components:gifdecoder.shlb 
viewer:Components:htmlparser.shlb 
viewer:Components:jpgdecoder.shlb 
viewer:Components:JSLoader.shlb 
viewer:Components:jsurl.shlb 
viewer:Components:layout.shlb 
viewer:Components:libpref.shlb 
viewer:Components:lwbrk.shlb 
viewer:Components:mngdecoder.shlb
viewer:Components:Mork.shlb 
viewer:Components:mozBrowser.shlb 
viewer:Components:nslocale.shlb 
viewer:Components:PIPNSS.shlb
viewer:Components:PIPPKI.shlb
viewer:Components:plugin.shlb 
viewer:Components:pngdecoder.shlb 
viewer:Components:prefm.shlb 
viewer:Components:profile.shlb 
viewer:Components:RDFLibrary.shlb 
viewer:Components:RegViewer.shlb 
viewer:Components:shistory.shlb 
viewer:Components:strres.shlb 
viewer:Components:TextServices.shlb
viewer:Components:uconv.shlb 
viewer:Components:ucth.shlb 
viewer:Components:ucvcn.shlb 
viewer:Components:ucvibm.shlb 
viewer:Components:ucvja.shlb 
viewer:Components:ucvko.shlb 
viewer:Components:ucvlatin.shlb 
viewer:Components:ucvtw.shlb 
viewer:Components:ucvtw2.shlb 
viewer:Components:uriloader.shlb 
viewer:Components:view.shlb 
viewer:Components:Wallet.shlb 
viewer:Components:WalletViewers.shlb 
viewer:Components:xfer.shlb 
viewer:Components:XPConnect.shlb 
viewer:Components:xmlextras.shlb
viewer:Components:transformiix.shlb
viewer:Components:gfx2.shlb
viewer:Components:libimg2.shlb
viewer:Components:pngdecoder2.shlb
viewer:Components:gifdecoder2.shlb
viewer:Components:jpegdecoder2.shlb
viewer:Components:icondecoder.shlb
viewer:Components:Necko.shlb 
viewer:Components:Necko2.shlb
viewer:Components:Cache.shlb 


mail lib
----------------------
viewer:Components:MsgAddrbook.shlb 
viewer:Components:mimeEmitter.shlb 
viewer:Components:mailnews.shlb 
viewer:Components:Mime.shlb 
viewer:Components:MsgCompose.shlb 
viewer:Components:MsgDB.shlb 
viewer:Components:MsgImap.shlb 
viewer:Components:MsgLocal.shlb 
viewer:Components:MsgNews.shlb 
viewer:Components:vcard.shlb 
viewer:Components:smime.shlb
viewer:Components:signed.shlb
viewer:Components:msgImport.shlb
viewer:Components:msgImportText.shlb
viewer:Components:msgImportEudora.shlb
viewer:Components:AbSyncSvc.shlb 
viewer:Components:mozldap.shlb
viewer:Essential Files:LDAPClient.shlb


So the DOM Project doesn't build. Now go back, open the project, try to build it 
manually, and inspect the errors.

Does the static build work for people on other platforms? You should try doing 
the static build on Windows and Mac at the same time, because all you are seeing 
here are genuine build errors.

dprice is working off the same branch for windows, at the same time. kandrot had 
done the original static linking work, on LINUX, on an earlier branch. It'll be 
interesting to see if they have/had run into the same problem, which for the 
record is:
Per previous attachment:

The problem is not that ` NS_IMETHOD GetTimeStamp(DOMTimeStamp *aTimeStamp) = 0;` 
is a pure virtual function, because there are other pure viruals, earlier in this 
class (class nsIDOMEvent).

When using CW's "preprocess" option on nsJSEnvironment.cpp, I get the line:
virtual nsresult GetTimeStamp(DOMTimeStamp *aTimeStamp) = 0;

The key problem, I think, is that there is no previous "DOMTimeStamp" decl in the 
preprocessed file.


what are the errors? are they in XPCOM?  If that's true, I think waterson has 
some patches for XPCOM that we need.  :-)

never mind.  waterson had already checked in XPCOM changes. see bug 46775
responding to sfraser's 5/25 12:21 comment:
dprice says, on windows, he had particular problems getting the correct versions 
of "nspr, psm, ldap, accessible, imglib2 and gfx2", and had to massage his 
mozilla/client.mak to get the correct versions.

So, this leads me back to the question: do we really have the correct 
MozillaCheckoutList.txt? When cross-referencing dprice's list of problems with my 
current version of MozillaCheckoutList.txt, NSPRPUB_CLIENT_BRANCH and 
LDAPCSDK_40_BRANCH stand out as questionable.

mozilla/nsprpub,        			NSPRPUB_CLIENT_BRANCH
mozilla/security/nss,   			NSS_CLIENT_TAG
mozilla/security/psm,				STATIC_BUILD_20010523_BRANCH   
mozilla/security/manager,			STATIC_BUILD_20010523_BRANCH
mozilla/accessible,					STATIC_BUILD_20010523_BRANCH
DirectorySDKSourceC,    			LDAPCSDK_40_BRANCH
mozilla/lib/mac/Instrumentation,	STATIC_BUILD_20010523_BRANCH
mozilla/gfx2,						STATIC_BUILD_20010523_BRANCH
mozilla/modules/libpr0n,			STATIC_BUILD_20010523_BRANCH
SeaMonkeyAll,						STATIC_BUILD_20010523_BRANCH

If anybody can certify this file as correct, I'm open to suggestions of how to 
proceed from here, given that the build problem laid out in previous attachments, 
still exists.

Steve, I think you're tree is messed up.  all of them should have 
STATIC_BUILD_20010523_BRANCH tag.

On windows, I just did a check out by
> cvs co -r STATIC_BUILD_20010523_BRANCH mozilla/client.mak

than I did
> nmake /f client.mak 

that'll pull the right branch for you. this applies to all directories.

You probably need to wipe out your tree and pull a clean one.

If you've checked out the mac checkout list file from the branch and it contains
pointers to branches that are not your static build branch, you need to fix that
file and check it back in on the branch.  You should also confirm that those
files have been correctly branched (although from cathleen's comments it sounds
like win32 is pulling the files correctly).  Both mac and win32 check out files
and directories specific to that platform that are not checked out on Linux
which can cause problems if you're trying to branch the whole tree and only
branch a tree checked out on one platform via the build scripts.
STATUS:
The source I pulled yesterday has built.

The executable fails after about 10 seconds, with a type "12" error while
loading the first page (www.mozilla.org).

PLANS:
I'm not going to worry about the executable failure right now. My first goal is
to get a statically linked build. 
Current status here is that we can't find a way to script CodeWarrior to create 
new project targets for the static build.

I have managed to get the XML::DOM Perl module working on Mac, and I can parse an 
XML-format project, which should make DOM operations to clone targets pretty 
easy.

To install XML::DOM for MacPerl:

1. Get and install
     http://www.perl.com/CPAN/authors/id/A/AS/ASANDSTRM/XML-Parser-2.27-bin-1-
MacOS.tgz
2. Get and install XML::DOM
     http://www.cpan.org/authors/id/T/TJ/TJMATHER/XML-DOM-1.31.tar.gz

3. Hack MacPerl:site_perl:XML:DOM:DOM.pm so that it's happy with XML::Parser 
2.27:

-   my $needVersion = '2.28';
+   my $needVersion = '2.27';       # smfr hack. Was 2.28

(note: I've no idea what bad side-effects this might have).

To install MacPerl modules, drag the .tar.gz (or .tgz) file onto "installme.plx" 
in the MacPerl folder.

If you need some Perl to get all the projects into XML, it's checked into the
CW7_20010609_BRANCH. The routine of interest is export_project in
CodeWarriorLib.pm. You'll have to hack the build scripts to call this for each
project. Also, the export will fail on projects which contain bogus access
paths. In the build script hackery, I put in code that printed the success or
failure and then went through and removed bad paths from the ones that failed
(10 or so)
Response to Simon Fraser 2001-06-27 19:45:

Thanks for the XML::DOM for MacPerl install instructions.

I've failed on the last step, as my copy of mozilla tools
has no `DOM.pm`; and I can find none in the current MacPerl folder
on `4G Server Storage` , either.
(On the server, there isn't even an XML folder, under the site_perl folder)
Random notes:
* You can clone a target in the XML by cloning the subtree under the <TARGET>
  tag. The new target must have a unique <SETTING><TARGETNAME>, and there must
  be a <ORDEREDTARGET> entry under <TARGETORDER> for the new target.

* XML::DOM collapses empty elements into the <VALUE/> form when writing XML,
  and CW Pro 5 cannot handle these (requiring <VALUE></VALUE>. Pro 6 handles
  them fine.

So I think it's going to be do-able to use this to make all the static lib 
targets.

The attached perl module can be used to read in an XML project, clone a named 
target, change the target settings to be a static lib target, and write the XML 
out again. It works with CW Pro 6, currently. Usage is something like:

my $doc = ProjectXML::ParseXMLDocument("Test.mcp.xml");
ProjectXML::CloneTarget($doc, "Test.shlb", "Test.lib");
ProjectXML::SetAsStaticLibraryTarget($doc, "Test.lib", "TestOutput.lib");
ProjectXML::WriteXMLDocument($doc, "Test_out.xml");
ProjectXML::DisposeXMLDocument($doc);

More recent patch in bug 88340.

I currently have things working to the point where we can script the creation of 
static library targets for all shared lib targets in projects in the build. There 
is a little more work to do to "fix up" these created targets (e.g. removing 
linkage with other libraries), but that should be relatively easy.

What we will need is a Mac version of the Perl XML::Parser module vers. 2.28, 
since XML::DOM depends on that.

Branch is still being cut, sorry we had cvs errors.
Branch will be called: STATIC_BUILD_20010628_BRANCH
ETA to finish branch cut, 8pm Friday.

branch is ready now, both mozilla & ns trees.

the branch, STATIC_BUILD_20010628_BRANCH, builds. It runs -- I am adding this
comment from the app that was built(optimized) from it.

The only major source problem which I encountered, had to do with corruption of
the source during the pull, rather than any problem with the official source
itself. For the record:


Error   : undefined identifier 'yption'
nsInternetSearchService.cpp line 5760   yption

`yption` was an extra line falsely added, at EOF, during the pull.
> `yption` was an extra line falsely added, at EOF, during the pull.

That's a bug in the version of MacCVS Pro that you're using. Grab MacCVS Pro 
2.7d7.

> That's a bug in the version of MacCVS Pro that you're using. Grab MacCVS Pro
> 2.7d7.

WTF version of MacCVS Pro has such a flaw? (I hope not the one I'm using and
didn't know)


reply to conrad (2001-07-02 19:13):

I used version 2.7d6
2.7d6 is afflicted, fixed in 2.7d7. The bug only happens if a file has mixed 
linebreaks in the CVS repository.

I believe that the only preprocessor macro which the build process needs to
define, uniquely for the static build, is ENABLE_STATIC_COMPONENT_LOADER.
Status: I've landed on the branch, STATIC_BUILD_20010628_BRANCH. Paul Chen and I 
are in the process of ensuring that I landed correctly.

In order to do that, Paul:
1) get  XML::DOM for MacPerl (try "4G Server Storage:Tools:MacPerl XML.sit") If 
that doesn't work, talk to me -- the technique from "Comments From Simon Fraser 
2001-06-27 19:45" wasn't recreatable on my machine, although it certainly worked 
on his.

2) do a Debug build (optimized doesn't work yet). Suggested Prefs file to follow.
Depends on: 90793, 93566
Depends on: 90763
No longer depends on: 90793
Gotten preliminary comparing results on mac startup time, cold boot went from 23
down to 18 sec, 20+% gain.  warm boot stayed at 11 sec on both builds.

Need to:
1) figure out landing plan
   what are the issues/bugs need to be sorted out before we land on mozilla??
2) make installer works with static build
3) fixup/make sure commercial build works

Steve, please fill in all missing details we need in this bug, thanks!
cathleen - this performance improvement is with the static build already broken
up into components like the win32 build, right?  I just want to be sure we think
the gain will stay with us before we start hacking the build system.  Thanks!
No. The mac static build is currently monolithic, apart from JS, NSPR, XPCOM, 
XPInstall and some runtime libs.

Landing Plan/issues:

Known bugs on tip of branch:
1) NS_GET_MODULE implementation of nsLocaleModule.cpp is almost certainly wrong 
(it links , but logic is almost certainly wrong). We need to know what we're 
doing before landing (Bug #93556)
    suggestion: either current owner or us should eyball currrent changes; ask 
owner how we test it; get reviewed.
2) probable resource-related problems: cursor bits are sometimes mangled; "Save 
AS" isn't working; printing seems broken.

Landing issues:
1) what is the official list of libs that should remain shared? 
(JavaScript.shlb, xpcom.shlb, and NSPR20.shlb are good-to-go; what about 
libreg.shlb?, others? NSRuntime.shlb adn NSStdLib.shlb should probably be taken 
out.)
2) do we check in only 2 .mcp's (the new StaticMerg.mcp and the edited 
apprunner.mcp) and let scripts recreate new static targets); 
OR also overwrite most of the other .mcp's with new static targets.

    PRO for checking in only two .mcp files: it makes landing easier
    CON for checking in only two .mcp files: it makes more difficult, fixing 
cases where the static build needs different (fewer) files than the shared build 
(for the first of many possible examples this: see next issue, where we may have 
to resolve dupicate symbol diagnostics by removing some object files)
        Since for the static build, the build scripts clone the set of files 
from the shared build; if we want different set of files, then special knowledge 
must be put into the build scripts)

    
    CON for checking in many .mcp files with new targets: penalizes everybody 
doing old-style build (by size and speed)(on pull and opening of project)
   
    Action Item: raise w/ macdev on Monday Aug 6th.

3) do we check-in apprunner.mcp with (prefs: "ignore duplicate symbol linker 
warnings" (only) for static targets)?
    suggestion: OK to check-in , but for the longer term it should be "off"
4) do we check merge trunk into branch first, or not?
    Action Item : look at changes to apprunner
    suggestion: decision in part, depends on how complicated landing is, which 
depends greatly on the decision for Landing Issue#2.

5) make sure installer works

6) make sure commerci
My Bad -- the NS_GET_MODULE bug for nsLocale is Bug#93566
Here's an overview of all the files which have been changed (or added), in order 
to make the static build work. For convenience, I have grouped them together in 
light of the gerneal reason why they were changed (or added). This grouping is 
only suggestive (some files were changed for more than one reason), but I 
believe that the list of files is complete.

build system stuff:
        BuildUtils.pm
        CodeWarriorLib.pm
        Moz.pm
        MozillaBuildList.pm
        ProjectXML.pm           // <---- new file
        MozillaBuildFlags.txt
        
projects changed to fix access paths, to fix build errors
        expat.mcp
        gifdecoder.mcp
        jpgdecoder.mcp
        libutil.mcp
        LiveConnect.mcp
        mngdecoder.mcp
        png.mcp
        txtsvc.mcp
        ucvja.mcp
        ucvko.mcp
        ucvtw2.mcp

high-level Projects, and related headers (like Prefix files)
        apprunner.mcp
        StaticMerge.mcp         // <---- new file
        apprunnerDebug.Prefix   // <---- new file
        apprunner.Prefix        // <---- new file
        apprunnerConfig.h       // <---- new file
        DefinesMozilla.h

change XPCOM registration, and change NS_GET_FACTORY implementation to 
NS_GET_MODULE
        nsGfxModule.cpp         // <---- new file
        nsLocaleModule.cpp      // <---- new file
        nsWidgetModule.cpp      // <---- new file
        gfx.mcp
        locale.mcp
        widget.mcp
        nsSetupRegistry.cpp
        nsStaticComponents.cpp  // <---- new file
        xpcomPPC.mcp
        xpcomConf
Depends on: 94221, 94372, 94373
Verified security functionality by going to the `https://www.verisign.com' site.

On the branch, I checked in cryptoComponent.mcp, mailnewsComponent.mcp, and 
MozillaBuildList.pm changes which enable this functionality.

Comparison of nsStaticComponents.cpp: generated during unix build, vs. tip of 
mac static build branch.

1) there were references for mailnews and crypto in the unix version. Was I 
wrong to be surprised to see them still there?
	MODULE(NSS),                 // security: taken out of mac version
	MODULE(PKI),                 // security: taken out of mac version
	MODULE(nsMsgBaseModule),     // mailnews: taken out of mac version
	MODULE(nsMsgDBModule),       // mailnews: taken out of mac version
	MODULE(nsMsgNewsModule),     // mailnews: taken out of mac version
	MODULE(local_mail_services), // mailnews: taken out of mac version
	MODULE(nsMimeEmitterModule), // mailnews: taken out of mac version
	MODULE(nsVCardModule),       // mailnews: taken out of mac version
	MODULE(mime_services),       // mailnews: taken out of mac version
	MODULE(nsMsgComposeModule),  // mailnews: taken out of mac version
	MODULE(IMAP_factory),        // mailnews: taken out of mac version
	MODULE(nsAbModule),          // mailnews: taken out of mac version
	MODULE(nsImportServiceModule),// mailnews: taken out of mac version
	MODULE(nsTextImportModule),  // mailnews: taken out of mac version
	MODULE(nsAbSyncModule),      // mailnews: taken out of mac version
	
2) there was a reference in unix, which was never in the mac static build. (I 
suppose I should check that the build option `options transformiix' works) 
    MODULE(TransformiixModule),// <---- new to mac prototype
    
3) 5 references from unix, were in the base mac static source, but commented out 
during when engineering the static build, for various reasons (among them, I 
think, the the module boundaries might have been different on mac, and so some 
functionalty was in fewer modules on the mac).
Also, there are 2 names on the mac branch, which are not present on the unix 
branch.

	MODULE(nsGtkTimerModule), // commented out in mac prototype
	MODULE(nsGfxPSModule), // commented out in mac prototype
	MODULE(nsPPMDecoderModule), // commented out in mac prototype
	MODULE(nsWidgetGTKModule), // commented out in mac prototype
	MODULE(XRemoteClientModule), // commented out in mac prototype

// missing in unix, present on mac prototype: MODULE(nsJarModule),
// missing in unix, present on mac prototype: MODULE(nsWidgetModule),

In at least the widget case (nsWidgetModule vs. nsWidgetGTKModule), I think this 
is due to a difference in mac naming conventions.

I'll attach the unix version of nsStaticComponents.cpp, edited with my c++-style 
com
Oops, sorry for the double attachment. peterv, please review attachment 46129 [details] [diff] [review]. 
scc, an sr= would be super too.

I have some fairly serious surgery to do on mozilla/intl/locale; see bug 94831.
I'm going to try to land this in the next day or so, and this should get the
locale stuff working right on all platforms.
Pfew, what a patch. r=peterv.

+        if ($target =~ /(.+).shlb$/)        # if this is a shared lib target
+        {
+            my $target_base = $1;
+            my $static_target = $target_base.".o";

Are our targets always named "foo.shlb"? That's a must after this change, no?
we really need to land all changes by end of this week.
what's the status right now?
Target Milestone: --- → mozilla0.9.4
Status Right Now:

On my machine, I've got the tip of the trunk (as of yesterday evening), merged 
with the tip of the static branch.

From this source, the shared lib version built. Since then, we are building the 
static version, and fixing any problems that we run into. It is building right 
now.

When this is done, I'll have to go back and prove that the shared lib build still 
works, in light changes made since yesterday evening.

+
	    if (! $main::options{"static_build"})
+
	    {
            
BuildOneProject(":mozilla:embedding:browser:powerplant:PPBrowser.mcp",         
  "PPEmbed$D",  0, 0, 0);


This isn't cool just to punt on this in a static build. Why doesn't it work?
What needs to be done to make the embedding sample work with the static build? I
would imagine that embedding clients will want to use static builds so the
embedding samples (all platforms) need to work with this.
If you check in mozilla/xpfe/bootstrap/nsStaticComponents.cpp and
mozilla/modules/staticmod/nsMetaModule_[mail|crypto].cpp, you will break the
static build on Win32 and Unix: there are different sets of modules compiled on
different platforms (e.g., nsGfxGTKModule, which we use on Unix to talk to GTK).
Please don't do that.
jfrancis gave r=jfrancis
I give sr=sfraser on these changes.

a=asa on behalf of drivers.
Target Milestone: mozilla0.9.4 → mozilla0.9.5
Due to my taking over the JavaScript group, cathleen has asked sfraser to assume
responsibility for this bug.
Assignee: thesteve → sfraser
Boing.
Target Milestone: mozilla0.9.5 → mozilla0.9.6
To 0.9.7
Target Milestone: mozilla0.9.6 → mozilla0.9.7
->0.9.9
Target Milestone: mozilla0.9.7 → mozilla0.9.9
Need to get a test build for QA to do some perf tests.
Status: NEW → ASSIGNED
Target Milestone: mozilla0.9.9 → mozilla1.0
We decided that this was not worthing doing for 1.0.
Target Milestone: mozilla1.0 → mozilla1.1alpha
Keywords: topembed → embed, topembed-
Attaching snapshot of my current scripts. Patch is not a final patch.
Attachment #35897 - Attachment is obsolete: true
Attachment #35899 - Attachment is obsolete: true
Attachment #36147 - Attachment is obsolete: true
Attachment #36149 - Attachment is obsolete: true
Attachment #36155 - Attachment is obsolete: true
Attachment #40600 - Attachment is obsolete: true
Attachment #41843 - Attachment is obsolete: true
Attachment #41869 - Attachment is obsolete: true
Attachment #43741 - Attachment is obsolete: true
Attachment #46128 - Attachment is obsolete: true
Attachment #46129 - Attachment is obsolete: true
Attachment #47067 - Attachment is obsolete: true
Not applicable since we're going to Macho-O
Status: ASSIGNED → RESOLVED
Closed: 22 years ago
Resolution: --- → WONTFIX
Summary: make mac static build → make mac CFM static build
v wontfix.
Status: RESOLVED → VERIFIED
Product: Browser → Seamonkey
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: