Push SUMO 1.2 release on Thursday June 25



Infrastructure & Operations
WebOps: Other
8 years ago
4 years ago


(Reporter: laura, Assigned: fox2mike)





8 years ago
This is the metrics dashboard release.

You will need to have Kettle set up in production (bug 499963).  

There is a tag coming, and a bunch of SQL to be run.


8 years ago
Depends on: 499963

Comment 1

8 years ago
Instructions for the push,

1) Download the requested version off Sourceforge, current one we're using is 3.2.0 (pdi-ce-3.2.0-stable.tar.gz) and extract to /opt/pentaho/ (create if needed). At the end of this you should have a folder /opt/pentaho/data-integration/ with a bunch of files and directories.
2) Make sure the .sh files inside data-integration are executable.
3) Make sure the machine has jdk-1.6 or above. Install if needed.
4) Get the entire scripts/metrics directory from svn (http://svn.mozilla.org/projects/sumo/trunk/scripts/metrics/) and put it into /opt/sumo/metrics/
5) If kettle will run as a non-root user, create the user. For example, kettleuser.
6) In the user's home directory : 
mkdir /home/kettleuser/.kettle
cp /opt/sumo/metrics/kettle.properties.dist /home/kettleuser/.kettle/kettle.properties
7) Edit the kettle.properties and provide the required database values (servernames/IP addresses/usernames and passwords)
8) At this point, you should be ready to run the ./kitchen.sh script with the required parameters and things should just magically work :D 

Also need to check on the various DB perms.


8 years ago
Assignee: server-ops → shyam

Comment 2

8 years ago
The tag is:
The SQL in https://bugzilla.mozilla.org/attachment.cgi?id=383979&action=edit needs to be run on production before the push.
The SQL in https://bug500008.bugzilla.mozilla.org/attachment.cgi?id=384675
(bug 500008) also needs to be run on production before the push

Comment 5

8 years ago
We have two mysql users in kettle.properties.

The first user (the user with permissions on the data warehouse) needs
INSERT/UDPATE/DROP/CREATE access to the data warehouse for the _initialization
steps_.  You can reduce it to just INSERT/UPDATE/DELETE after running the
kitchen.sh bits for the first time.

The second user (with permissions on the tiki database) ONLY needs SELECT
permissions on the tiki database.  Even though these are scripts so there's no
risk of external malicious injection, it's better safe than sorry.

the /opt/pentaho/data-integration/kitchen.sh scripts with parameters MUST be
run from the /opt/pentaho/data-integration/ folder and as the appropriate user
(kettleuser).  So when cronjobbing it, make sure to cd first and make sure to
sudo su kettleuser before doing the initializing.  Otherwise you get all sorts
of weird errors.

Comment 6

8 years ago
Oh and I wanted to add steps to comment #1:

6.1) create the database that will be used for the data warehouse having the schema/importing the data in /sumo/metrics/dwschema
6.2) create user with appropriate permissions (INSERT/UDPATE/DROP/CREATE) with access to this DB.

9) Reduce permissions for user created in 6.2 to INSERT/UPDATE (if needed)
10) cronjob cd /opt/pentaho/data-integration/; ./kitchen.sh -norep -file /opt/sumo/metrics/j_sumo_etl.kjb for every night at 4 AM.

Comment 7

8 years ago
Man, I'm so tired I completely forgot I'd filed bug 499963 with essentially those steps.  Can't brain today.

Comment 8

8 years ago
Okay, all done. Thanks Dave, Reed, Trevor, Laura, Cheng and anyone else I inadventently missed out :)

Cron has been setup for kettleuser to run at 4am PDT everyday.
Last Resolved: 8 years ago
Resolution: --- → FIXED
Post-production push, I verified bug 480587, bug 499031, and bug 497461; Laura verified bug 458642 and bug 444701.

Additionally (and most importantly), https://support.mozilla.com/tiki-metrics.php is up and displaying data from its SQL query.

Verified FIXED; SUMO 1.2 is deployed!  (Big thanks to all!)
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.