865822 - Setup mozilla-hosted dnt-dashboard.mozilla.org

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task)

Description

[Anurag Phadke[:[e-mail]]]

Hello IT Team,
I know this is last minute and I apologize in advance.
The metrics team has built a DNT dashboard: https://metrics.mozilla.com/protected/dnt/v2/ which is currently being hosted on a non-mozilla property.

Is it possible to create a virtual machine and host the above dashboard on dnt-dashboard.mozilla.org? 

NYTimes is running an article on DNT and Mozilla's role in DNT on Sunday, the above DNT dashboard will be featured in the article. Harvey was featured in the Senate hearing on 4/24 talking about DNT. With so many high profile involved in DNT and Mozilla's role, we really prefer to host it on a mozilla property.

Again, I know this is last minute and I really really understand the favor I am asking you guys.

-anurag

Comment 1

[Anurag Phadke[:[e-mail]]]

Also, we technically don't need a new VM and are open to using existing resources. The DNT dashboard is a static page and doesn't include any DB etc.

The DNT graphs are intended to provide a view into how many Firefox users are seeking out and enabling Do Not Track in their desktop and mobile versions of Firefox


Code: https://github.com/hamilton/public_dnt_dashboard (Hamilton will be making the repo public in couple of hours)

Comment 2

[Corey Shields [:cshields]]

moving to webops.

Sooner we can get code the sooner we can estimate if this will be possible or not.  Typically sec reviews are planned out through the quarter so having this unplanned work preempts other projects that planned ahead.  I'd suggest filing the sec review bug now and linking to this.

Comment 3

[Corey Shields [:cshields]]

also dropping priority to avoid paging oncall since, without code, this is inactionable.

Comment 4

[Curtis Koenig [:curtisk-use curtis.koenig+bzATgmail.com]]]

(In reply to Corey Shields [:cshields] from comment #2)
  I'd suggest
> filing the sec review bug now and linking to this.
Bugs filled and blocking this, we also did an out-of-band assignment to get these going now (luckily we have some open bandwidth)
bug 865832: SecReview: DNT Dashboard
bug 865833: SecReview: DNT Dashboard (OPSec Review)

Comment 5

[Brandon Burton [:solarce]]

Code is available at https://github.com/hamilton/public_dnt_dashboard

I'll setup a dev site for testing

Comment 6

[Brandon Burton [:solarce]]

-> % svn ci -m "adding dnt-dashboard-dev.allizom.org apache configs, bug 865822"
Adding         domains/dnt-dashboard-dev.allizom.org.conf
Transmitting file data .
Committed revision 64323.

-> % svn ci -m "adding dnt-dashboard-dev.allizom.org apache configs, bug 865822"
Sending        domains/dnt-dashboard-dev.allizom.org.conf
Transmitting file data .
Committed revision 64327.

-> % svn ci -m "dnt-dashboard dev and stage dns, bug 865822"
Sending        dnsconfig/zones/allizom.org
Transmitting file data .
Committed revision 64324.

SSL certificate for prod ordered in bug 866148

Comment 7

[Brandon Burton [:solarce]]

Dev site is almost ready, where do we pull the .json files from?

Comment 8

[Ali Almossawi]

The .json files are available here: https://metrics.mozilla.com/protected/dnt/v2/data/

And zipped here: https://metrics.mozilla.com/protected/dnt/v2/data/data.zip


Thanks.

Comment 9

[Ali Almossawi]

If you could just not include the ones under "deprecated" since they contain finer grain data that the PR team didn't want released.

Comment 10

[Brandon Burton [:solarce]]

(In reply to Ali Almossawi from comment #9)
> If you could just not include the ones under "deprecated" since they contain
> finer grain data that the PR team didn't want released.

Is the data not publicly available?

Comment 11

[Ali Almossawi]

We didn't want to post it publicly before the NYT article goes out.  Otherwise, it's just aggregated percentages, so there's no PII at all.

Comment 12

[Brandon Burton [:solarce]]

-> % svn ci -m "adding dnt-dashboard.allizom.org apache configs, bug 865822"
Adding         genericrhel6-stage/etc-httpd/domains/dnt-dashboard-dev.allizom.org.conf
Transmitting file data .
Committed revision 64329.

-> % svn ci -m "adding dnt-dashboard.mozilla.org apache configs, bug 865822"
Adding         genericrhel6/etc-httpd/domains/dnt-dashboard.mozilla.org.conf
Transmitting file data .
Committed revision 64330.

-> % svn ci -m "adding dnt-dashboard.mozilla.org weblog directory, bug 865822"
Sending        manifests/genericrhel6.pp
Transmitting file data .
Committed revision 64331.

Comment 13

[Brandon Burton [:solarce]]

Everything has been setup with the current data.

The production site is ready to launch once we have approval that the code has passed the security review.

Comment 14

[Brandon Burton [:solarce]]

I forgot to specifically call this out but the dev site is at http://dnt-dashboard-dev.allizom.org/

Comment 15

[Ali Almossawi]

Thank you.

The plan is to release this a few hours before the article goes live, so either myself or Anurag will update this bug once we get word of that.

Comment 16

[Raymond Forbes[:rforbes]]

I have gone through the js code and nothing stares out at me as a problem.  I will run a scan of the site using zap tomorrow evening.

Comment 17

[Raymond Forbes[:rforbes]]

ok, i just ran a vega scan on the dev site and everything looks good.

Comment 18

[Brandon Burton [:solarce]]

https://dnt-dashboard.mozilla.org/ has been deployed and is behind MoCo LDAP for now.

When you know the launch time please update this bug and if it is after 5PM contact the oncall sysadmin in #it

For IT:

To launch the site.

1. Comment out the Basic Auth lines  in modules/webapp/files/genericrhel6/etc-httpd/domains/dnt-dashboard.mozilla.org.conf and commit to SVN
2. Wait for the puppetmaster to update
3. On genericadm.private.phx1 run 'sudo issue-multi-command genericrhel6 puppetctl run --test' and confirm the puppet run applies the changes to the whole cluster
4. The site should now be live without basic auth

Comment 19

[Anurag Phadke[:[e-mail]]]

current ETA from NYTimes is Thursday 9pm PST. Mike Manning from PR will let us/me know few hours before that and I'll update the bug once that happens

Comment 20

[Jake Maul [:jakem]]

This is all done.