tinderbox scripts need to be checked in to the source tree

RESOLVED FIXED in 3.11.8

Status

NSS
Test
P2
normal
RESOLVED FIXED
12 years ago
11 years ago

People

(Reporter: Neil Williams, Assigned: Slavomir Katuscak)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 5 obsolete attachments)

(Reporter)

Description

12 years ago
Recently bug 331111 was opened against a Windows tinderbox. The source of the problem turned out to be the PATH environment variable which was set wrong by a batch file that is run before the tinderbox scripts are started. Since the neither the batch file nor (?) the tinderbox perl scripts are checked into cvs the fix was not saved anywhere but on the formerly failing tinderbox. If the files used to run the tinderboxes is mozilla source they should be checked into the NSS source tree.
Repeating my comment in bug 331111:
ALL the scripts used in tinderbox on all tinderbox test platforms MUST be 
checked in.  
Tinderbox must not depend on anyone's personal home directories or files.
All the files that are used for Tinderbox, to set it up, restart it, etc,
including any crontab entries, must be checked in, and must be usable by
any NSS developer.  
Priority: -- → P2
Target Milestone: --- → 3.11.1
In Today's meeting, Sandeep suggested that all the tinderbox files are 
checked in under mozilla/tools/tinderbox.  So I checked out that directory
and found it is empty.  Perhaps the files exist only on a branch?  

In any case, on whatever branch they may be, I doubt that these files are
actually the files that are running on the NSS tinderbox.  The recent
tinderbox failures on windows were fixed by changing a PATH assignment in
a .bat file or shell script.  Is that change now checked in to some file
under mozilla/tools/tinderbox?  I'm sceptical.
Assignee: nobody → Sandeep.Konchady
Version: unspecified → 3.11

Comment 3

12 years ago
Tinderbox scripts that I modified for testing NSS and JSS were picked up from http://lxr.mozilla.org/mozilla/source/tools/tinderbox/.
Priority: P2 → --
Target Milestone: 3.11.1 → ---
Version: 3.11 → unspecified
Sandeep, please read and reply to comment 2
Priority: -- → P2
Target Milestone: --- → 3.11.1
Version: unspecified → 3.11

Comment 5

12 years ago
Nelson,

[1] I just checked out mozilla/tools/tinderbox from the tip and all the files needed to run tinderbox exist. I am not sure why you got an empty directory.

[2] As for checking in the changes that were done to the scripts, no I have not yet done. I wanted to discuss this issue with you before I checkin the script. 

For now the NSS tinderbox is run as the NSS QA user within Sun and does not reside in an users home directory. I have done some fixes to these scripts that are checkedin to our local repository as an intermediate solution (redcvs.red.iplanet.com). Once I have tested all the changes, which I will list below, I will request a review in this bug. Once approved I will check in the changes back to mozilla.

I have a question about the environment setup scripts that we use. These are specific to the way we setup our QA/tinderbox machines. Should I check in these files to mozilla?

List of modifications to NSS tinderbox:

[1] Have the ability to build and test 32 and 64 bits alternately
[2] Have the ability to build and test OPT and DBG builds alternately
[3] Maintain 10 previous versions of the workspace for debuging purpose
[4] Ensure that the tinderbox script does not sleep if the build+test time is over 1 hr.

Also I would like to file bugs against these so that I can track the changes. Shoud I file these under NSS or mozilla tools?

Thanks,
Sandeep
Priority: P2 → --
Target Milestone: 3.11.1 → ---
Version: 3.11 → unspecified
Priority: -- → P2
Target Milestone: --- → 3.11.1
Version: unspecified → 3.11

Comment 6

12 years ago
Created attachment 219801 [details] [diff] [review]
Patch for running tinderbox for NSS and JSS

The patch contains four files, two of which are new and two are modified. The new files are:

[1] build-nss.pl: The script that invokes NSS/JSS tinderbox
[2] build-nss-util.pl : Utility script for running the NSS/JSS tinderbox. This script also knows how to checkout/build/test 32 and 64 bit version of NSS and JSS.

The modified files are:

[1] build-semonkey-util.pl : The only modification here is to make OpenLOG and CloseLOG into a seperate function.
[2] tinder-defaults.pl : This is the initializer file that contains data for initializing the tinderbox parameters. To be able to run the tinderbox for NSS, the parameter $BuildTree needs to be set to NSS, but this is not default. The only parameter added to this file is 32/64 bit platform specific build location (EG: for Solaris 9, 32 bit DBG it is SunOS5.9_DBG.OBJ and for Solaris 9, 64 bit DBG it is SunOS5.9_64_DBG.OBJ). These are set for the parameters $Dbg32ObjDir, $Opt32ObjDir, $Dbg64ObjDir and $Opt64Objdir.

The files that are not part of this but are needed for running the tests are setenv.sh and setenv.bat. These are platform and domain specific, hence will not be checked in.
Attachment #219801 - Flags: superreview?(alexei.volkov.bugs)
Attachment #219801 - Flags: review?(slavomir.katuscak)
The NSS module owners are not also the owners of mozilla/tools/tinderbox,
so changes to that module will require review & approval from the owners 
of that module.  Alterntively, we can check in the files under 
mozilla/security somewhere TBD.
(Assignee)

Updated

12 years ago
Attachment #219801 - Flags: review?(slavomir.katuscak) → review+
Assignee: Sandeep.Konchady → slavomir.katuscak
(Assignee)

Comment 8

12 years ago
Comment on attachment 219801 [details] [diff] [review]
Patch for running tinderbox for NSS and JSS

I'm working on newer version of this patch, I will submit it later.
Attachment #219801 - Attachment is obsolete: true
Attachment #219801 - Flags: superreview?(alexei.volkov.bugs)
(Assignee)

Comment 9

12 years ago
I'm working on new patch, based on attachment 219801 [details] [diff] [review].

In old version are touched 4 files:
build-nss.pl - main script to run testing
build-nss-util.pl - script based on original build-seamonkey-util.pl from modified to test NSS, many functies were skipped and are used refferneces to build-seamonkey.pl
build-seamonkey-util.pl - modified logging system
tinder-defaults.pl - added new options

In my version there will be:
build-nss.pl - main script to run testing, I have removed there  tree_specific_overides from build-nss-util.pl to prevent any configuration variable in build-nss-util.pl
build-nss-util.pl - I have modified it to run on any platform without modification inside (now supported all except Windows, working on this). I also plan to remove all references to build-seamonkey-util.pl by copying functions from build-seamonkey-util.pl here.
build-seamonkey-util.pl - this file will stay untouched like is now in CVS (will be not used in NSS testing)
tinder-defaults.pl - added some more options

Idea of this changes is:
- not touch any other tinderbox files from CVS (no need to track changes there)
- be independent on other tinderbox files from CVS
- not to have any settings parameters in build-nss-util.pl 
- keep parameters from tree_specific_overides in build-nss.pl (this contains CVS directories, make parameters, can change once per time, not very often, but the same for all machines)
- keep machine specific parameters in tinder-config.pl, default values in tinder-defaults.pl
- finally we should have on every machine the same version of tinderbox, only parameters in tinder-config.pl will need to be set

I can't create diff now, because I have some problem with connect to mozilla CVS server, but I'll send my current version in short time.
(Assignee)

Comment 10

12 years ago
Created attachment 229831 [details] [diff] [review]
Patch for running tinderbox for NSS and JSS (snapshot)

This is today's snapshot of my patch, not a final version. It's configured to run on attic and it's still dependent on build-seamonkey-util.pl.
(Assignee)

Updated

12 years ago
Attachment #229831 - Attachment is obsolete: true
(Assignee)

Comment 11

12 years ago
Created attachment 233571 [details]
New Tinderbox for testing NSS

I finished new version of Tinderbox for NSS testing. Many things are rewrited, many removed, so it's more like a new program independent on original Tinderbox source. It doesn't make a sense to create patch, I'm sending complete sources. It's also stored on redcvs in ns/securityqa/tinderbox.

I also updated documentation at http://wikihome.sfbay.sun.com/jes-security/Wiki.jsp?page=Tinderbox .
(Assignee)

Updated

12 years ago
Attachment #233571 - Flags: superreview?(nelson)
Attachment #233571 - Flags: review?(neil.williams)
(Reporter)

Comment 12

12 years ago
Comment on attachment 233571 [details]
New Tinderbox for testing NSS

This looks good. If this is what is running it's very close to what we need IMO. Couple of things:

I'd still like to see this in patch form (easier to review).

It looks like tinder-defailts.pl is being updated from the mozilla original. Who are the reviewers for mozilla/tools?

The documentation on Sun's wiki is pretty good--some of it should be checked in with the patch.

The keystore is not generated by the scripts, is it? How does one do this?
(Assignee)

Comment 13

12 years ago
Hi Niel,

I removed many options from tinder-defaults.pl, because many of them were not used at all or not usable in our testing, original tinder-defaults was very general, and for many options there was not very clear their meaning. Now it's hopefully clear enough. 

Also I removed many unused parts of code. build-seamonkey-util.pl is now renamed to tinder-utils.pl (it contains basic tinderbox functions), because it doesn't have anything to do with seamonkey now. This is the reason why it doesn't make a sense to create patch, because it will looks like removed big parts of code (and files) and added big parts of other code (and files), not like simple diffs with small changes.

Keystore - I use copy from attic machine stored in apps directory. 
Slavo, are there now tinderbox systems running your new script? 
Is there a tinderbox web page where I can see the results?
How many files are in this .tgz attachment?  
If only one or two, please just attach it.them uncompressed and untarred.
(Assignee)

Comment 15

12 years ago
Hi Nelson,

All running Tinderboxes uses new version. 
Results are on http://tinderbox.mozilla.org/showbuilds.cgi?tree=NSS

We have just now 2 machines off - sxreports due to SCSI failure and intel due to unknown reason (can't connect to ssh, I created labqueue request for this already).

In tgz there are 7 code+config files + config for every machine + keystore.
All files are also in CVS (cvs -d :pserver:svbld@redcvs.red.iplanet.com:/m/src co ns/securityqa/tinderbox).
Slavo, there are tools that help reviewers to review patches.  
They require that the patches be attached as uncompressed output of cvs diff.
The command 
   cvs diff -upN <file names> 
is a good way to generate these patch files.
Please email me separately if you have questions about this.
Comment on attachment 233571 [details]
New Tinderbox for testing NSS

Please see comment 16.
Please put the code into patch form for review.  
Email me if that's not clear.
Attachment #233571 - Flags: superreview?(nelson) → superreview-
(Reporter)

Comment 18

12 years ago
Comment on attachment 233571 [details]
New Tinderbox for testing NSS

See Nelson's comment.
Attachment #233571 - Flags: review?(neil.williams) → review-
(Assignee)

Updated

11 years ago
Target Milestone: 3.11.1 → 3.11.6
(Assignee)

Comment 19

11 years ago
Created attachment 254412 [details] [diff] [review]
Tinderbox as patch to mozilla CVS.

This version of Tinderbox is already running on 7 testing machines, I added there some improvements (support for more branches, test selection over TESTS variable, some bugfixes,...).

I plan to put Tinderbox to mozilla/security/tinderbox directory in mozilla cvs. I would like to have write access to this directory without reviewing of patches, because it usually takes too much time and sometimes I need to apply a fix on all testing machines ASAP. Now I usually check fix into cvs (redcvs) and then update cvs on all machines and restart Tinderboxes. 

This version is still using jarsigner (bug 363222). Migration to signtool is planned, but it has lower priority and there are still some issues with it, so it's not part of this patch.
Attachment #233571 - Attachment is obsolete: true
Attachment #254412 - Flags: review?(nelson)
(Assignee)

Updated

11 years ago
Attachment #254412 - Flags: review?(nelson) → review?(neil.williams)
(Reporter)

Comment 20

11 years ago
Comment on attachment 254412 [details] [diff] [review]
Tinderbox as patch to mozilla CVS.

This looks good to me. It also looks a lot different from what I remember. One of the problems we had before was that the Windows  tinderbox was using Windows find command instead of the MKS version because the env wasn't set up properly. Do the utils subroutines check that sort of thing?
Attachment #254412 - Flags: review?(neil.williams) → review+
(Assignee)

Updated

11 years ago
Target Milestone: 3.11.6 → 3.11.8
(Assignee)

Comment 21

11 years ago
Created attachment 265109 [details] [diff] [review]
Latest version of Tinderbox.

Latest version of Tinderbox (already running). Contains many improvements, support for both braches and config files for all currently used machines.
Attachment #254412 - Attachment is obsolete: true
Attachment #265109 - Flags: review?(nelson)
(Assignee)

Comment 22

11 years ago
(In reply to comment #20)
> This looks good to me. It also looks a lot different from what I remember. One
> of the problems we had before was that the Windows  tinderbox was using Windows
> find command instead of the MKS version because the env wasn't set up properly.
> Do the utils subroutines check that sort of thing?
> 

Windows paths are set in setenv.bat. I don't see any problem with Windows now.
Comment on attachment 265109 [details] [diff] [review]
Latest version of Tinderbox.

Slavo, comments & questions:

1. This patch adds a bunch of new files in a new directory 
named tinderbox.  It doesn't give any clue about where that
new directory exists (or will exist) in the source repository.
Is it in nss/tests/tinderbox?  or where?

2. Many of the new files contain MANY lines of perl code that
are all commented out.  I'd rather that we don't have all 
those copies of commented out code.  

If you take out all the commented-out code, some of these files
have very few lines left.  And I think (not sure) that quite a
few of them are duplicates of each other.  Seems like we should
not need all (or any) duplicate copies of files.  Instead we 
should have some script that can select the right file for the
local system somehow.  

So, please eliminate the duplication and the many new commented
out lines of code.

/Nelson
Comment on attachment 265109 [details] [diff] [review]
Latest version of Tinderbox.

Please see my previous comment for the review feedback.
Attachment #265109 - Flags: review?(nelson) → review-
(Assignee)

Comment 25

11 years ago
Created attachment 269371 [details] [diff] [review]
New patch.

To Nelson's comments:

1. Suggested directory: mozilla/security/tinderbox. I would like to have Tinderbox outside of NSS directory, because it tests also JSS.

2. Originally I created config files like copies of tinder-config.pl and just modified some variables. I rewrited this part, and put config for all machines to tinder-config.pl with detection by hostname, also I removed commented variables there.
Attachment #265109 - Attachment is obsolete: true
Attachment #269371 - Flags: review?(nelson)
Comment on attachment 269371 [details] [diff] [review]
New patch.

r=nelson. This patch is a lot better than the last one, IMO.  
I didn't do my usual exhaustive review of all this new code,
but it's what we're using now (IINM), and it's working well
enough for Tinderbox.  Thanks.
Attachment #269371 - Flags: review?(nelson) → review+
(Assignee)

Comment 27

11 years ago
RCS file: /cvsroot/mozilla/security/tinderbox/build-nss-util.pl,v
done
Checking in tinderbox/build-nss-util.pl;
/cvsroot/mozilla/security/tinderbox/build-nss-util.pl,v  <--  build-nss-util.pl
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/security/tinderbox/build-nss.pl,v
done
Checking in tinderbox/build-nss.pl;
/cvsroot/mozilla/security/tinderbox/build-nss.pl,v  <--  build-nss.pl
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/security/tinderbox/gettime.pl,v
done
Checking in tinderbox/gettime.pl;
/cvsroot/mozilla/security/tinderbox/gettime.pl,v  <--  gettime.pl
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/security/tinderbox/setenv.bat,v
done
Checking in tinderbox/setenv.bat;
/cvsroot/mozilla/security/tinderbox/setenv.bat,v  <--  setenv.bat
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/security/tinderbox/setenv.sh,v
done
Checking in tinderbox/setenv.sh;
/cvsroot/mozilla/security/tinderbox/setenv.sh,v  <--  setenv.sh
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/security/tinderbox/tinder-config.pl,v
done
Checking in tinderbox/tinder-config.pl;
/cvsroot/mozilla/security/tinderbox/tinder-config.pl,v  <--  tinder-config.pl
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/security/tinderbox/tinder-defaults.pl,v
done
Checking in tinderbox/tinder-defaults.pl;
/cvsroot/mozilla/security/tinderbox/tinder-defaults.pl,v  <--  tinder-defaults.pl
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/security/tinderbox/tinder-utils.pl,v
done
Checking in tinderbox/tinder-utils.pl;
/cvsroot/mozilla/security/tinderbox/tinder-utils.pl,v  <--  tinder-utils.pl
initial revision: 1.1
done
Status: NEW → RESOLVED
Last Resolved: 11 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.