1331765 - Simplify JSS Makefile build and test

Status: RESOLVED FIXED

(JSS Graveyard :: Tests, defect)

Description

[Matthew Harmsen]

Attachment: build_java.pl

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0
Build ID: 20161211151355

Steps to reproduce:

The Makefile rules to build and test require simplification in order to streamline this process.


Actual results:

The current process requires numerous manual "hacks" in order to be executed successfully.


Expected results:

Prepare a work area

mkdir sandbox
cd sandbox
hg clone https://hg.mozilla.org/projects/nspr
hg clone https://hg.mozilla.org/projects/nss
hg clone https://hg.mozilla.org/projects/jss
cd ..

(Don't clone every time. At later times, you can use:
  cd nspr; hg pull -u -v; cd ..; cd nss; hg pull -u -v; cd ..; cd jss; hg pull -u
-v; cd ..
)

IMPORTANT:  Until they have been checked into the source repo, manually replace Makefile, rules.mk, and build_java.pl in the sandbox/jss directory with the attached replacements.

NOTE:  JSS will now automatically check that nspr, nss and jss have been checked out next to each other before continuing.



Prepare an interactive shell for building:
export JAVA_HOME=/etc/alternatives/java_sdk_1.8.0_openjdk
export USE_64=1

NOTE:  JSS will now attempt to verify whether or not these two environment variables have been set (JAVA_HOME is mandatory; USE_64 is mandatory on 64-bit platforms when building 64-bit).



# Build jss:
cd sandbox/jss
make clean all
cd ../..

(or you can run "# script -c 'make clean all' typescript.build')

NOTE: If nss has not been built, it will now automatically be built before jss; if nss has already been built, only jss will be built/re-built.



#  Run JSS tests (Linux only):
cd sandbox/jss
make test_jss
cd ../..

(or you can run "# script -c 'make test_jss' typescript.tests)

NOTE: This command is currently only available on Linux platforms; currenty JSS must be built via 'make clean all' before execution of this command (e.g. - build is separate from test).

Comment 1

[Matthew Harmsen]

Attachment: Makefile

Comment 2

[Matthew Harmsen]

Attachment: rules.mk

Comment 3

[Elio Maldonado]

Attachment: simplifies JSS Makefile build and test

Matt's attachments in the form of a patch so its easier to make review comments.

Comment 4

[Elio Maldonado]

Comment on attachment 8827666 [details] [diff] [review]
simplifies JSS Makefile build and test

Review of attachment 8827666 [details] [diff] [review]:
-----------------------------------------------------------------

r+, As is it works on Linux. We still need some small change to support macOS. I have tested these changes plus the modified version I'm proposing on Fedora: F25, Rawhide (upcoming F26), RHEL-7.3 (both with system wide FIPS enabled and not), Debian 8, openSUSE 4.2.2, and macOS latest.

::: build_java.pl
@@ +202,5 @@
> +              "# END:  Building NSS #\n" .
> +              "######################\n");
> +    }
> +
> +    if( $os eq 'Linux' ) {

Please change to
if ( $os eq 'Linux' || $os eq 'Darwin' ) {
so it will also work on macOS.

@@ +428,5 @@
>  }
> +
> +sub test {
> +    # Test JSS presuming that it has already been built
> +    if( $os eq 'Linux' ) {

Please change to
if ( $os eq 'Linux' || $os eq 'Darwin' ) {
so it will also work on macOS.

Comment 5

[Elio Maldonado]

Attachment: simplifiy JSS Makefile build and test V2

This version includes Comment 4 recommended changes to support macOS.

Comment 6

[Elio Maldonado]

Attachment: simplifiy JSS Makefile build and test V2

Comment 7

[Matthew Harmsen]

Comment on attachment 8827676 [details] [diff] [review]
simplifiy JSS Makefile build and test V2

Review of attachment 8827676 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good with a change to one of the die error messages since it now runs on MacOS as well as Linux systems.

::: build_java.pl
@@ +446,5 @@
> +        } else {
> +            die "JSS builds are not available at $jss_symlink.";
> +        }
> +    } else {
> +        die "make test_jss is only available on Linux platforms.";

die "make test_jss is only available on Linux and MacOS platforms.";

Comment 8

[Christina Fu]

Comment on attachment 8827676 [details] [diff] [review]
simplifiy JSS Makefile build and test V2

Review of attachment 8827676 [details] [diff] [review]:
-----------------------------------------------------------------

I followed the instruction in the Description of this bug for my Fedora 24 and it works without issues.
However, please check the following info provided by Kai's email dated 12/06/2016

===Kai's email follows===
You need to make the following work (which are the commands that the automation
script executes):

cd jss
export PLATFORM=`make platform`
cd ..
export HGDIR=`pwd`
cd jss/org/mozilla/jss/tests
perl all.pl dist $HGDIR/dist/$PLATFORM
cd $HGDIR

Comment 9

[Elio Maldonado]

Attachment: simplifiy JSS Makefile build and test V3

With the change that Matt requested.

Comment 10

[Elio Maldonado]

Pushed: https://hg.mozilla.org/projects/jss/rev/7727ae8b500759eff655d8df84660b0951b62308

Comment 11

[Elio Maldonado]

Pushed: https://hg.mozilla.org/projects/jss/rev/031cb8b9f5d2701566ec01cf87ad7283234485c7
Removed unwanted whitespace that Matt pointed out.

Comment 12

[Elio Maldonado]

On debian 8 I had to do things a bit diffrent.

/usr/sbin/alternatives: No such file or directory
There is no /usr/sbin/alternatives in Debian, Ubuntu, etc. Debian clones. Instead there is 
/usr/bin/update-alternatives.
 
From http://askubuntu.com/questions/597015/cannot-remove-jdk-usr-sbin-alternatives-no-such-file-or-directory 

I suggest a temporary symbolic link to work around the badly converted RPM package:

mkdir -p /usr/sbin
ln -sT -- /usr/bin/update-alternatives /usr/sbin/alternatives

there is also a wiki https://wiki.debian.org/DebianAlternatives

Comment 13

[Elio Maldonado]

While testing in openSUSE system I had to use 'export JAVA_HOME=/etc/alternatives/java_sdk'

Comment 14

[Elio Maldonado]

on macOS, you'll have something like this:
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home

Comment 15

[Matthew Harmsen]

Attachment: jss-use-installed-nspr-and-nss.patch

This patch adds the ability to build against the NSPR and NSS versions that are already installed on the build system.

Comment 16

[Matthew Harmsen]

Attachment: jss-use-installed-nspr-and-nss.patch

This patch adds the ability to build against the NSPR and NSS versions that are already installed on the build system.

Comment 17

[jmagne]

Comment on attachment 8837707 [details] [diff] [review]
jss-use-installed-nspr-and-nss.patch

Review of attachment 8837707 [details] [diff] [review]:
-----------------------------------------------------------------

Looks quite straight forward. We are giving the option to build using the system nss/nspr OR build using freshly built nss/nspr instances.

I observed Elio performing a build WITHOUT the system nss/nspr, which would take place upstream.

I also observed the detailed build output of a downstream build, that uses the environment variables telling the build to use system nss/nspr.

ACK

Comment 18

[Elio Maldonado]

Pushed: https://hg.mozilla.org/projects/jss/rev/eab6e889bc566c844fec1ff614983a6252c8df67

Comment 19

[Elio Maldonado]

https://hg.mozilla.org/projects/jss/rev/f3cd986eadbcc3f7e42ee278d2e8ed3497273134

Comment 20

[Elio Maldonado]

For clarity, 

1) To the top of the old README I added

# This file has been renamed to "jss/org/mozilla/jss/tests/README_bbenv", and
# the build/test procedure documented below has been replaced; the new build/test
# procedure has been documented in top-level "jss/README".

2) and then renamed it README_bbenv. 

Matt's review was conducted offline on IRC.