301249 - Checking out and building DBM as part of NSS in mozilla builds

Status: RESOLVED FIXED

(Firefox Build System :: General, defect, P2)

Description

[Wan-Teh Chang]

The NSS team now owns the Berkeley DB 1.85 code
in mozilla/dbm (also called "DBM).

The C code in mozilla/dbm can be built with two
sets of makefiles.

1. The Mozilla autoconf-based Makefile.in's in
mozilla/dbm.

2. The NSS coreconf-based Makefiles in a separate
directory, mozilla/security/dbm.  (Those makefiles
use GNU make's VPATH variable to point to the source
files in mozilla/dbm.)

Right now Mozilla clients use the first set of makefiles
to build DBM.  We would like to switch to the second
set of makefiles.  In addition, we would like to
check out mozilla/dbm and mozilla/security/dbm using
the same CVS tag as NSS proper (mozilla/security/coreconf
and mozilla/security/dbm).  We don't want to move the source
files in mozilla/dbm to mozilla/security/dbm.

The biggest challenge is that mozilla/dbm is part of
the SeaMonkeyAll CVS module, but we often check out
SeaMonkeyAll and NSS using two different CVS tags
(e.g., the HEAD and NSS_CLIENT_TAG).  I think it is
a problem if we check out two overlapping CVS modules
using different CVS tags.

I heard that we can't remove mozilla/dbm from SeaMonkeyAll.
If so, I think the best solution is to copy the .h and .c
files in mozilla/dbm to mozilla/security/dbm, preferrably
with CVS histories, and cvs remove the all the files under
mozilla/dbm.

Any objection?  Any better solution?

Once we agree on a solution to this problem, Doug Turner
said he will implement the Mozilla client makefile changes
to check out and build DBM as part of NSS.

Comment 1

[Wan-Teh Chang]

*** Bug 328518 has been marked as a duplicate of this bug. ***

Comment 2

[Benjamin Smedberg]

If it would be easier I have plans to stop using the SeaMonkeyAll CVS module for mozilla checkouts (using explicit checkout directories in client.mk instead), and I can push that up to make this happen.

Comment 3

[Wan-Teh Chang]

Ben, yes, it would be easier if we stop using the SeaMonkeyAll
CVS module for mozilla checkouts.

Comment 4

[Benjamin Smedberg]

WTC, can I get you to tag mozilla/dbm with NSS_CLIENT_TAG so that I can pull it with NSS?

Comment 5

[Benjamin Smedberg]

Attachment: Build dbm with NSS, rev. 1

This patch works without any changes to NSS except the necessary tagging of mozilla/dbm and mozilla/security/dbm with the appropriate NSS_CLIENT_TAG.

Comment 6

[Doug Turner (:dougt)]

Nelson and I spoke for a bit this morning.  I think we both agree that we should drop using NSS_CLIENT_TAG and instead have the mozilla build pull the right NSS release tag.  This will allow us to easily determine what version of NSS shipped with a specific version of a mozilla client.  thoughts?

Comment 7

[Benjamin Smedberg]

I couldn't agree more! What is the current tag to use for mozilla trunk?

Comment 8

[Nelson Bolyard (seldom reads bugmail)]

Wan-Teh is presently away on bereavement leave, until the end of March, approx.

Whenever a numbered NSS release is made, and QAed, the source for that release
is tagged with a numbered NSS release tag, such as NSS_3_11_RTM for NSS 3.11.

It *appears* that all the parts of NSS _except_ DBM are presently also tagged 
with NSS_CLIENT_TAG, and that NSS_CLIENT_TAG present corresponds to the 
NSS 3.11 release (that is, to NSS_3_11_RTM).  

However, I think that needs to be better confirmed, to ensure that those
two tags match throughout ALL of NSS, before we simply tag DBM with an NSS_CLIENT_TAG that corresponds exactly to the NSS_3_11_RTM tag.  
(Ah, the joys of using mutable tags.  How does one ever know what they mean?).

I would like to propose an alternative to tagging DBM with NSS_CLIENT_TAG.
That alternative amounts to the abolitiion of NSS_CLIENT_TAG (going forward).
I propose that the clients simply switch from NSS_CLIENT_TAG to the present
release tag, NSS_3_11_RTM.  That change will cause the clients to pull 
everything in NSS, including DBM, in a consistent, tested state, with a 
known immutable name.  

IINM, one of the goals of the revision control in the client products is the
ability to go back and pull the sources as they were at the point of a past
release, perhaps one made long ago.  NSS_CLIENT_TAG, being a moving mutable
tag, seems useless for that purpose.  The numbered immutable NSS release 
tags, OTOH, seem ideal for that purpose.  

At some point in a future, when NSS has produced a newer release (and newer
release tag), the clients may switch from pulling with the NSS_3_11_RTM tag
to pulling with the newer tag (e.g. NSS_3_11_1_RTM or NSS_3_12_RTM) at their
discretion and pleasure.   

Comments?

Comment 9

[Benjamin Smedberg]

Attachment: Build dbm with NSS, rev. 1.1

Like this! This is awesomely simple.

Comment 10

[Doug Turner (:dougt)]

Comment on attachment 216151 [details] [diff] [review]
Build dbm with NSS, rev. 1.1

great as a first step.  I would like to see only one DBM at some point.

r= assuming that is the correct NSS VERSION, nelson or one of the NSS peers might be able to answer this.

Comment 11

[Nelson Bolyard (seldom reads bugmail)]

> ------- Comment #10 from [e-mail]  2006-03-24 14:20 PST -------
> (From update of attachment 216151 [details] [diff] [review])
> great as a first step.  I would like to see only one DBM at some point.

how many DBMs do we have?  
NSS uses only one, and its in mozilla/dbm.
I've been repeatedly told that NSS is the only user of DBM.
So, if NSS is the only user of DBM, and the only DBM that nss uses is
the one mozilla/DBM, then why do we have others?
Can we CVS remove them?

> r= assuming that is the correct NSS VERSION, nelson or one of the NSS peers
> might be able to answer this.

AFAIK NSS_CLIENT_TAG == NSS_3_11_RTM everywhere in NSS except DBM,
where presently there is no NSS_CLIENT_TAG.  So, yes, AFAIK, NSS_3_11_RTM
is the right tag for DBM as well as for the rest of NSS.

Comment 12

[Benjamin Smedberg]

Fixed on trunk. There is only one set of DBM sources: mozilla/dbm

The files in mozilla/security/dbm are makefiles which drive the coreconf build of mozilla/dbm and these two directories should be unified at some point. (preferably in security/dbm).

Comment 13

[Benjamin Smedberg]

There was an extra bustage-fix in client.mk when checking out by date.

Comment 14

[Mark Mentovai]

Additional breakage caused by this patch is in bug 332012 and bug 332050.  Both are fully diagnosed and only need cvs tags to move, if it's permissible to move the NSS_3_11_RTM tag now.

If not, we won't be able to use NSS_3_11_RTM.

Comment 15

[Nelson Bolyard (seldom reads bugmail)]

It is not and never will be permissible to move an NSS release tag.
It's best to stop thinking of moving tags.  
Tags ought not move.
Instead, new tags ought to be adopted as needed.
If you can't use NSS_3_11_RTM, then we need to get you a newer tag.
Not move a tag, but have you switch to a newer one.

Comment 16

[Doug Shelton]

If I understand the comments, this changed tag is the reason for pulling an older (non-useable) version of mozilla/security/nss/cmd/shlibsign/sign.sh.  This tag causes CVS to pull version 1.16, which causes BeOS build bustage.  Current version is 1.17.  See bug 331879.

Comment 17

[Kai Engert (:KaiE:)]

cvs rdiff -u -r NSS_3_11_RTM -r NSS_CLIENT_TAG mozilla/security/nss mozilla/security/coreconf

This command gives a 1 MB sized diff, > 28000 lines.

We should have made that comparison prior to making the change.

So we are now in a state where we are using old code and things are broken.
I think we should fix that asap.

I propose the following as an immediate fix:
  cvs co -r NSS_CLIENT_TAG mozilla/security/nss
  cvs co -r NSS_CLIENT_TAG mozilla/security/coreconf
  cvs co -r NSS_3_11_RTM mozilla/dbm
  cvs co -r NSS_3_11_RTM mozilla/security/dbm
  cvs tag _SOME_NEW_TAG_ mozilla/dbm mozilla/security/dbm mozilla/security/nss mozilla/security/coreconf

Nelson, is that what you proposed yesterday?

I see a tag 
  NSS_3_11_20060331_TAG
has been created.
Is that identical to the above?

Comment 18

[Kai Engert (:KaiE:)]

To answer myself:

cvs rdiff -u -r NSS_3_11_RTM -r NSS_3_11_20060331_TAG mozilla/dbm
gives a difference in a comment only

cvs rdiff -u -r NSS_3_11_RTM -r NSS_3_11_20060331_TAG mozilla/security/dbm/
cvs rdiff -u -r NSS_CLIENT_TAG -r NSS_3_11_20060331_TAG mozilla/security/coreconf
cvs rdiff -u -r NSS_CLIENT_TAG -r NSS_3_11_20060331_TAG mozilla/security/nss
do not produce any differences

Are we ready to change mozilla/client.mk to pull NSS_3_11_20060331_TAG ?

Comment 19

[Benjamin Smedberg]

I performed a diff of NSS_CLIENT_TAG against the new tag and it looks good. I am checking in a change to client.mk to make this change for the mozilla tree.

Comment 20

[Benjamin Smedberg]

ok, fixed

Comment 21

[Wan-Teh Chang]

Attachment: A better way to determine if an NSS_CO_TAG can pulled by date

The current code tests for _RTM or _TAG in NSS_CO_TAG.
This prevents us from using NSS_3_11_1_BETA as
NSS_CO_TAG.

NSS uses the naming convention that all the branch tags
end in _BRANCH.  This patch allows the tip (an empty
NSS_CO_TAG or NSS_CO_TAG=HEAD) and branch tags to be
pulled by date.

Comment 22

[Nelson Bolyard (seldom reads bugmail)]

*** Bug 331879 has been marked as a duplicate of this bug. ***

Comment 23

[Nelson Bolyard (seldom reads bugmail)]

*** Bug 332050 has been marked as a duplicate of this bug. ***