527744 - Set up a buildbot for Dehydra/Treehydra tests and connect to Tinderbox

Status: RESOLVED INCOMPLETE

Description

[David Mandelin [:dmandelin]]

We would like a buildbot that runs the Dehydra/Treehydra unit tests and reports the results to the TraceMonkey Tinderbox. The buildbot needs:

- Our standard buildbot linux setup for building Firefox. (This includes the patched GCC compiler (see bug 504828))
- Pull Mozilla code from TraceMonkey repo (hg.mozilla.org/tracemonkey)

The basic steps it should run for one cycle are:

1. Build JS libraries and headers
   - pull from TM repo
   - build optimized JS shell
   - (needed files land in dist/lib and dist/include)

2. Build Dehydra/Treehydra

# pull dehydra
hg clone http://hg.mozilla.org/rewriting-and-analysis/dehydra/
cd dehydra
# configure dehydra build
export CXX=$SPECIAL_GCC_DIR/bin/g++
./configure \
  --js-libs=$JS_DIR/js/src \ 
  --js-headers=$JS_DIR/js/src  
# Build
make

3. Run unit tests for tinderbox.
make check_both_tinderbox

I'll put the answers to your standard questionnaire in the next comment.

Comment 1

[David Mandelin [:dmandelin]]

    * do you want builds? yes -- but we only need JS built. Use TraceMonkey tree.
          o which o.s.? linux
                + All o.s. or subset of linux, mac, win32, linux-arm, WinCE
          o incremental-build-on-checkin? y
          o nightlies? n
          o NOTE: we cant yet provide nightly updates for project branches like this. We're working on this, but its not possible yet.
          o Are en-US builds enough, with no l10n? y
                + If you 'must' have l10n, do you need all locales from m-c or a subset of locales?

    * do you want unittests? yes -- run 'make check' on Treehydra
          o which o.s.? linux
                + All o.s. or subset of linux, mac, win32, linux-arm, WinCE

    * do you want talos?no
          o which o.s.?
                + All o.s. or subset of linux, mac, win32

    * name of branch owner, who will: n/a, this isn't really a new branch.
          o be doing periodic refreshes from m-c
          o be contact person for misc setup questions
          o decide when to land back project branch onto m-c
          o decide when to terminate the project branch

    * timeline:
          o when can we start project branch (any pre-req landings pending needed before starting project branch out from mozilla-central?) now
          o approx expected life span of project branch - if known? indefinite

    * misc:
          o need any changes to toolchain used in m-c? no
          o need any changes to the compile/link/repack steps used in m-c? yes, see above
          o preference on tinderbox waterfall name? "Dehydra unit test"
          o preference on where to put builds on ftp.m.o? builds not needed
                + used for places tinderbox-builds/my-project-branch, nightly/latest-my-project-branch/, or nightly/2008-08-08-08-my-project-branch/
          o preference on name of project branch in hg? n/a, not a branch
          o what unofficial projectname do you want on this project branch?
                + (ie we use Minefield on mozilla-central, shiretoko on mozilla-191)
    * any other info that might be helpful to us?

Comment 2

[Chris AtLee [:catlee]]

(In reply to comment #0)
> 1. Build JS libraries and headers
>    - pull from TM repo
>    - build optimized JS shell
>    - (needed files land in dist/lib and dist/include)

Can you give more detail for these steps please.

Comment 3

[David Mandelin [:dmandelin]]

(In reply to comment #2)
> (In reply to comment #0)
> > 1. Build JS libraries and headers
> >    - pull from TM repo
> >    - build optimized JS shell
> >    - (needed files land in dist/lib and dist/include)
> 
> Can you give more detail for these steps please.

The way I would do it normally is:

cd $(JS_SRC_DIR)
hg clone http://hg.mozilla.org/tracemonkey
mkdir $(JS_BUILD_DIR)
cd $(JS_BUILD_DIR)
$(JS_SRC_DIR)/configure
make
# results are now in $(JS_BUILD_DIR)/dist and can be passed to dehydra configure

Comment 4

[Chris AtLee [:catlee]]

(In reply to comment #3)
> (In reply to comment #2)
> > (In reply to comment #0)
> > > 1. Build JS libraries and headers
> > >    - pull from TM repo
> > >    - build optimized JS shell
> > >    - (needed files land in dist/lib and dist/include)
> > 
> > Can you give more detail for these steps please.
> 
> The way I would do it normally is:
> 
> cd $(JS_SRC_DIR)
> hg clone http://hg.mozilla.org/tracemonkey
> mkdir $(JS_BUILD_DIR)
> cd $(JS_BUILD_DIR)
> $(JS_SRC_DIR)/configure
> make
> # results are now in $(JS_BUILD_DIR)/dist and can be passed to dehydra
> configure

What are JS_SRC_DIR / JS_BUILD_DIR?  Won't this end up building the entire tree?

Comment 5

[Chris AtLee [:catlee]]

>     * do you want unittests? yes -- run 'make check' on Treehydra

Is this in addition to 'make check_both_tinderbox'?

Comment 6

[Ben Hearsum (:bhearsum)]

Handing to John for prioritization.

Comment 7

[(dormant account)]

What is the status of this?

Comment 8

[Chris AtLee [:catlee]]

I don't think we're going to be able to get to this before Q1.

Comment 9

[Chris Cooper [:coop] pronoun: he]

Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.

Comment 10

[John Ford [:jhford] CET/CEST Berlin Time]

Before I start working on this, is it something that we still want?

Comment 11

[Chris Cooper [:coop] pronoun: he]

(In reply to comment #0)
> export CXX=$SPECIAL_GCC_DIR/bin/g++
> ./configure \
>   --js-libs=$JS_DIR/js/src \ 
>   --js-headers=$JS_DIR/js/src  
> # Build
> make

I assume JS_DIR refers to the TM source checkout, but what does $SPECIAL_GCC_DIR refer to? 

See also comment #4. We need to know what $JS_SRC_DIR and $JS_BUILD_DIR refer to as well.

Comment 12

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #10)
> Before I start working on this, is it something that we still want?

It has been two weeks with no updates.  I am going to mark this bug as INCOMPLETE.

If this is something that we still want, please file a new bug with complete and up to date information required to run the tests.