1214897 - Deploy telemetry-experiment-server for Search beahvior recorded in Places

Status: RESOLVED FIXED

(Infrastructure & Operations :: MOC: Service Requests, task)

Description

[Bill Maggs (bmaggs)]

+++ This bug was initially created as a clone of Bug #1131864 +++

Please deploy the Telemetry Experiment detailed in https://bugzilla.mozilla.org/show_bug.cgi?id=1174937 

Bsmedberg has signed off on the deployment. 

This revision should be tagged with release_tag.

Comment 1

[Benjamin Smedberg]

This isn't ready yet. https://wiki.mozilla.org/Telemetry/Experiments has the checklist:

The code hasn't been pushed to http://hg.mozilla.org/webtools/telemetry-experiment-server/
Has there been any QA? Typically there has to be release validation QA against dev: https://telemetry-experiment-dev.allizom.org/
Engineers normally do the tagging: MOC just deploys to CDN.
A release manager needs to sign off on deployment.

Going to resolve this INCOMPLETE until that's done.

Comment 2

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Reopening to track QA work for the release.

Comment 3

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Hi Kamil,

I'm on PTO Friday, but hopefully we can get this process started today. A few questions:

1. Does the QA process require changing the start and end dates to include today's date?

2. We aren't using the telemetry server to collect data; we're using datadog[1], a data collection and dashboarding service. Do you have an account? If not, we'll have to get one for you from cloud ops.

3. What's your schedule look like next week, and how much time is typically needed to QA these experiments?

Thanks,

Jared

Comment 4

[Kamil Jozwiak [:kjozwiak]]

> 1. Does the QA process require changing the start and end dates to include
> today's date?

I usually don't change the start/end dates. However I do change the dates on the machines to make sure that the experiments are correctly starting and expiring. There's been a few issues in the past where experiments haven't correctly expired or have expired prematurely. Would this cause any problems as the data being sent back might have incorrect dates/times etc..?

If I do need to change the dates for some reason, I usually pull the experiment locally and change the epoch values in the manifest before re-building and hosting it locally.

Not sure this answers your question? Let me know :)
 
> 2. We aren't using the telemetry server to collect data; we're using
> datadog[1], a data collection and dashboarding service. Do you have an
> account? If not, we'll have to get one for you from cloud ops.

Nope, I don't have an account :/ Let me know what the process is and I'll get it started. Is there any documentation/wiki pages available?

> 3. What's your schedule look like next week, and how much time is typically
> needed to QA these experiments?

It usually takes me about a full day to go through an experiment if I don't run into any problems. Scheduling shouldn't be a problem as Q4 just started so it's not very hectic like it usually is near the end of a quarter :)

Comment 5

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

(In reply to Kamil Jozwiak [:kjozwiak] from comment #4)
> > 1. Does the QA process require changing the start and end dates to include
> > today's date?
> 
> I usually don't change the start/end dates. However I do change the dates on
> the machines to make sure that the experiments are correctly starting and
> expiring. There's been a few issues in the past where experiments haven't
> correctly expired or have expired prematurely. Would this cause any problems
> as the data being sent back might have incorrect dates/times etc..?

Nope, the data we're sending back is time independent, so that shouldn't matter.

However, we're looking at the distribution of the percentage of history views that are search result pages, so we do need to be sure that the experiment uninstalls itself after it runs exactly once.

> 
> If I do need to change the dates for some reason, I usually pull the
> experiment locally and change the epoch values in the manifest before
> re-building and hosting it locally.
> 
> Not sure this answers your question? Let me know :)

It does :-) I'll just take a guess at the start date, and I guess re-push as needed.

>  
> > 2. We aren't using the telemetry server to collect data; we're using
> > datadog[1], a data collection and dashboarding service. Do you have an
> > account? If not, we'll have to get one for you from cloud ops.
> 
> Nope, I don't have an account :/ Let me know what the process is and I'll
> get it started. Is there any documentation/wiki pages available?

There's a tiny bit of documentation on mana, but I think it's probably best just to ask someone in cloud ops.

Looks like none of the cloud ops folk are in Toronto, but if you ping ckolos or travis on IRC, they should be able to get you access. ckolos is on Eastern time.

> 
> > 3. What's your schedule look like next week, and how much time is typically
> > needed to QA these experiments?
> 
> It usually takes me about a full day to go through an experiment if I don't
> run into any problems. Scheduling shouldn't be a problem as Q4 just started
> so it's not very hectic like it usually is near the end of a quarter :)

Fantastic :-)

After thinking about it a little, it seems like maybe not the best idea to do an unfamiliar, potentially destructive thing (pushing to hg.m.o) at the end of the day before a long weekend. So, I'll push the branch Monday, and I'll ping you on IRC (or in this bug) when the code's on the staging server.

Thanks again for your help!

Comment 6

[Benjamin Smedberg]

Since this bug is a MOC deployment bug, could you move the QA discussion back to bug 1174937? That way this bug is clean for the MOC final deployment step.

Comment 7

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

The telemetry experiment in bug 1174937 has gotten approval from QA and release-drivers, and the patch has been landed.

Reopening this bug to get the release process started.

Comment 8

[:Felipe Gomes (needinfo for replies!)]

Deployment instructions:
https://mana.mozilla.org/wiki/pages/viewpage.action?pageId=38553804#telemetry-experiment.cdn.mozilla.net%28FirefoxTelemetryExperiments%29-Update/Pushprocedure

Usul, you did the last deployment (in bug 1217613). Could you do this one too?

Comment 9

[Ludovic Hirlimann [:Usul]]

on it.

Comment 10

[Ludovic Hirlimann [:Usul]]

What revision should be pushed ?

Comment 11

[:Felipe Gomes (needinfo for replies!)]

To simplify it we always mark the one to be pushed with the hg tag release_tag. It is currently rev ce604a104dc2, but I think checking out from the tag name works, and then we can update the instructions to make this clear.

Comment 12

[Ludovic Hirlimann [:Usul]]

/me tries

Comment 13

[Ludovic Hirlimann [:Usul]]

[e-mail]1 telemetry-experiment.cdn.mozilla.net]$ sudo ./update.sh release_tag
DIR: /data/genericrhel6/src/telemetry-experiment.cdn.mozilla.net
ENV: genericrhel6
SITE: telemetry-experiment.cdn.mozilla.net
APP: telemetry-experiment
Updating code...
/data/genericrhel6/src/telemetry-experiment.cdn.mozilla.net/telemetry-experiment /data/genericrhel6/src/telemetry-experiment.cdn.mozilla.net
/data/genericrhel6/src/telemetry-experiment.cdn.mozilla.net
[2015-11-18 04:41:27] Running rsync_project
[2015-11-18 04:41:27] [localhost] running: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --exclude '.svn*' --exclude 'CVS' --exclude '.bzr*' --delete /data/genericrhel6/src/telemetry-experiment.cdn.mozilla.net/ /data/genericrhel6/www/telemetry-experiment.cdn.mozilla.net/
[2015-11-18 04:41:27] [localhost] finished: /usr/bin/rsync -aq --include '.gitkeep' --exclude '.git*' --exclude '.hg*' --exclude '.svn*' --exclude 'CVS' --exclude '.bzr*' --delete /data/genericrhel6/src/telemetry-experiment.cdn.mozilla.net/ /data/genericrhel6/www/telemetry-experiment.cdn.mozilla.net/ (0.069s)
[2015-11-18 04:41:27] Finished rsync_project (0.070s)
[2015-11-18 04:41:27] Running commit_www
[2015-11-18 04:41:27] [localhost] running: cd /data/genericrhel6/www && /usr/bin/git add .; /usr/bin/git commit -a -m 'deploy ['telemetry-experiment.cdn.mozilla.net']'
[2015-11-18 04:41:32] [localhost] finished: cd /data/genericrhel6/www && /usr/bin/git add .; /usr/bin/git commit -a -m 'deploy ['telemetry-experiment.cdn.mozilla.net']' (4.220s)
[localhost] out: [master 6eb8b4c] deploy [telemetry-experiment.cdn.mozilla.net]
[localhost] out: 10 files changed, 434 insertions(+), 1 deletion(-)
[localhost] out: create mode 100644 telemetry-experiment.cdn.mozilla.net/telemetry-experiment/experiments/serp-fraction-counts/DOCUMENTATION
[localhost] out: create mode 100644 telemetry-experiment.cdn.mozilla.net/telemetry-experiment/experiments/serp-fraction-counts/code/bootstrap.js
[localhost] out: create mode 100644 telemetry-experiment.cdn.mozilla.net/telemetry-experiment/experiments/serp-fraction-counts/code/install.rdf
[localhost] out: create mode 100644 telemetry-experiment.cdn.mozilla.net/telemetry-experiment/experiments/serp-fraction-counts/experiment.xpi
[localhost] out: create mode 100644 telemetry-experiment.cdn.mozilla.net/telemetry-experiment/experiments/serp-fraction-counts/filter.js
[localhost] out: create mode 100644 telemetry-experiment.cdn.mozilla.net/telemetry-experiment/experiments/serp-fraction-counts/manifest.json
[localhost] out: delete mode 100644 telemetry-experiment.cdn.mozilla.net/web-output/[e-mail]/experiment.xpi
[localhost] out: rewrite telemetry-experiment.cdn.mozilla.net/web-output/firefox-manifest.json (100%)
[localhost] out: create mode 100644 telemetry-experiment.cdn.mozilla.net/web-output/[e-mail]/experiment.xpi
[2015-11-18 04:41:32] Finished commit_www (4.220s)