Create module for PuppetAgain to install Java 7

RESOLVED FIXED

Status

task
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: whimboo, Assigned: whimboo)

Tracking

({dev-doc-needed})

Details

(Whiteboard: kanbanzilla[Needs Investigation], )

Attachments

(1 attachment)

On all of our Mozmill CI nodes we need Java JRE to be installed. We need the necessary modules for PuppetAgain to run a necessary installation during startup.

Necessary instructions can be found at:
https://mana.mozilla.org/wiki/display/websites/QA+Automation+ESX+Service
I'm working on this module.
Assignee: nobody → andreea.matei
Status: NEW → ASSIGNED
You might wanna check existing modules and packages first.
Component: Infrastructure → RelOps: Puppet
Product: Mozilla QA → Infrastructure & Operations
QA Contact: dustin
Summary: Create module for PuppetAgain to install Java → Create module for PuppetAgain to install Java 7
Version: unspecified → other
Whiteboard: kanbanzilla[Needs Investigation]
Dustin, currently we are installing Java that way:
http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html

Looks like we have to cache the ppa:webupd8team/java repository? That's at least for Ubuntu. Or would it be better to simply host the installers as can be found here: http://java.com/en/download/linux_manual.jsp?locale=en

For OS X we are downloading the installer from http://java.com/en/download/mac_download.jsp
Mirroring the PPA is probably the best solution.

And best to use debmirror, like we do for other repos:
  https://wiki.mozilla.org/ReleaseEngineering/PuppetAgain/Packages

If you mirror that on your puppetmaster (at /data/repos/apt/webupd8) and get things working the way you'd like, I can rsync it over to the moco distinguished master and we can keep it up to date from there.

Similarly for OS X -- put the DMGs in /data/repos/DMGs/, let me know about them when you're happy, and I'll sync them upstream.
Thanks for the info Dustin. One more question I have is if we should really use the official Oracle version of Java or if we use Open JRE on Ubuntu. But not sure if other Linux distributions also ship this by default. Also it may be better to rely on the same software across all platforms.
We decided that I'll work on the flash module since I'm also duty this week and is less blocking for the goal.
Assignee: andreea.matei → hskupin
I'm currently trying the 'icedtea-netx' package on our slave nodes. This package contains the javaws command and used the 'openjdk-7-jre'. If that works I would like to keep it, and not install the Oracle version of Java.
Looks like it works nicely so far. I will give it a couple of days time for testing.
OpenJDK works on Ubuntu, but I think we should not take it but better base on a tool which is available across all platforms. So lets use Oracle's version of Java.
Depends on: 1031584
So I have uploaded the DMG and EXE file to /data/repos. But when I try to install the package it fails. Reason is that we are trying to use the proxy:

Debug: Executing '/usr/bin/curl -o /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/d20140627-78104-1fzhkr9/java-7.0.600.dmg.dmg -C - -k -L -s --fail --url http://puppet/repos/DMGs/java-7.0.600.dmg --proxy proxy.dmz.scl3.mozilla.com:3128'
Debug: curl -o /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/d20140627-78104-1fzhkr9/java-7.0.600.dmg.dmg -C - -k -L -s --fail --url http://puppet/repos/DMGs/java-7.0.600.dmg --proxy proxy.dmz.scl3.mozilla.com:3128 did not transfer [java-7.0.600.dmg].  Falling back to local file.
Error: Could not set 'present' on ensure: No such file or directory - http://puppet/repos/DMGs/java-7.0.600.dmg at 30:/etc/puppet/environments/hskupin/env/modules/packages/manifests/pkgdmg.pp

Looks like puppet doesn't take the no_proxy setting into account:
https://ask.puppetlabs.com/question/3270/need-to-set-no_proxy-for-windows-agent/

It may be that we temporarily need an ACL so a request to puppetmaster1/puppet is served directly by squid.
Dustin, I assume that we cannot remove the proxy settings from the puppetmaster and block it from reaching the network, or? In such a case we would have to find out where to include the webproxy module.
Depends on: 1032133
(In reply to Henrik Skupin (:whimboo) from comment #11)
> Dustin, I assume that we cannot remove the proxy settings from the
> puppetmaster and block it from reaching the network, or? In such a case we
> would have to find out where to include the webproxy module.

I filed bug 1032133 for the proxy issue.
Ok, so here the updated information regarding the Java packages on Ubuntu... The referenced URL from the Webupd8 team only contains the installer. That tool downloads and installs Java. It's a nice wrapper but mirroring that necessary repository, doesn't make that much sense given that the installer file would also have to be mirrored. I feel that's a bit too much handwork to be done here.

So I would propose that in case of Ubuntu we make use of icedtea-netx and openJDK. Possibly it should also work for Cent OS. Dustin, what do you think of that?
It's really quite out of my league.  Whatever suits your needs, and works without relying on external sites, works for me.
WIP patch just for sharing implementation details with Andreea.
Dustin, how is the uninstallation packages managed? Is there such a way in puppet?

For testing the installation I called this command to remove Java:
sudo rm -fr /Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin

But now puppet agent does no longer install it, even when I run it against production first. So somewhere it keeps traces for the installation happened.
OS X doesn't support uninstallation of packages.

Puppet keeps track of what it has installed with some semaphore files:
  https://github.com/puppetlabs/puppet/blob/master/lib/puppet/provider/package/pkgdmg.rb#L64

so removing that file should allow you to re-run the install.
Ah, that worked. Ok, re-tested and the installation works fine now. Something we also have to do is to update the system settings on OS X, so Java is not automatically checking for updates. This will raise a notification, which can conflict with Mozmill. Further we don't want updates until we have a new version managed by puppet.

Looks like this has to be done here:
~/Library/Preferences/com.oracle.javadeployment.plist

Dustin, should this check go into the java package, or to disableservices?
The appropriate key in this file is:

<key>deployment.macosx.check.update</key>
<string>false</string>
I think that will be best done in the package manifest, since disableservices would first need to figure out whether Java is installed or not.  Also, I think that's part of installing the package -- ensure that it stays installed :)
Comment on attachment 8448707 [details] [diff] [review]
Install Java v1

This patch works with the proxy turned off for the agent now. I tested it on OS X and Ubuntu.

Something which I wonder is that the installation happens async, so puppet exists while the Java is not installed yet. Is that always the case for installations? 

Dustin, please also sync the repo folder from our puppetmaster, so that the Java DMG is getting synced. Thanks.
Attachment #8448707 - Attachment description: WIP v1 → Install Java v1
Attachment #8448707 - Flags: review?(dustin)
Comment on attachment 8448707 [details] [diff] [review]
Install Java v1

sync is complete
Attachment #8448707 - Flags: review?(dustin) → review+
Thanks Dustin. So I got this landed on our repo:

https://hg.mozilla.org/qa/puppet/rev/639837b0fa09 (default)
https://hg.mozilla.org/qa/puppet/rev/0994a64bb26f (production)
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Depends on: 1038646
You need to log in before you can comment on or make changes to this bug.