Closed Bug 1184773 Opened 9 years ago Closed 9 years ago

Hosting for trackertest.org

Categories

(Infrastructure & Operations Graveyard :: WebOps: Engagement, task)

Product:
Infrastructure & Operations Graveyard
Component:
WebOps: Engagement
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: ckprice, Assigned: fox2mike)

References

Details

(Whiteboard: [kanban:https://webops.kanbanize.com/ctrl_board/2/1432] ) 

We have an odd little bird we'd like to see if you guys can help us set up. See bug 1177978 for more information.

In bug 1184735 you set up a domain for us (thanks!).

The specifications for this page are pretty simple

- HTML/CSS/JS only
- No backend
- Good amount of traffic, should be CDN'd (will be serving images to high traffic pages)

So, we'd need it hosted somewhere that we could upload assets (images, HTML, CSS, JS) to. It's not going to be a complicated site.

Let me know what all else is needed here.
Whiteboard: [kanban:https://webops.kanbanize.com/ctrl_board/2/1432]
Assignee: server-ops-webops → nmaul
Blocks: 1185008
See Also: 1185008 → 
:w0ts0n and I will work on this.

First thing is, would it be reasonable for you to shove these assets into a source control repo somewhere (github, git.m.o, hg.m.o, svn.m.o... doesn't matter)? I don't know how much "stuff" we're looking at or the change rate, or the technical level of the updaters, so that may or may not be convenient for you. It makes deployment simple and automatic though, as we just check out the repo on a schedule and it's magically "live".


If that's no good, we can look at other options. The next choice might be an S3 bucket that you can upload to directly. Archive data and change management would be on you (that is, whomever is updating it)- we wouldn't be able to help with rollbacks, for example.


CDN is fairly straightforward either way. It's a bit more painful cost-wise and setup-time-wise to host CDN with a vanity domain like this (trackertest.org) as opposed to our standard (<something>.cdn.mozilla.net, or something vendor-provided like <something>.cdn.cloudfront.net), so if the CDN name doesn't matter to you we can optimize that. Your stuff would still be available at trackertest.org, but that would be a non-CDN version of it... so not good for high traffic.
We can push to GitHub np
I actually wondered about github. Their resiliency seems good enough unless China is hacking them. Would we consider putting something mission-critical there?

An s3 bucket would also be fine (archiving is not a concern).

The nature of the site is a concern. This is a domain that Mozilla is hosting that will contain no tracking. Ideally we're even in control of server logs, but it's not a requirement. 

Definite requirements are:

1. Use own host name (trackertest.org). This name is going to be blocklisted on purpose as a fake tracking domain. 
2. We cannot have any other third-party loads on the site. Definitely no analytics or other third-party assets that would track. 
3. Simple site, a single flat page linking to some sub resources (a few images, some small sample JS files)

The whole concept is that this is a developer resource that simulates a bad thing (tracking) while offering pure safety because it isn't doing that bad thing.
Group: mozilla-employee-confidential
Hi :jakem,

Thanks for looking after this.

Spoke with Javaun and team...

(In reply to Jake Maul [:jakem] from comment #1)
> :w0ts0n and I will work on this.
> 
> First thing is, would it be reasonable for you to shove these assets into a
> source control repo somewhere (github, git.m.o, hg.m.o, svn.m.o... doesn't
> matter)? I don't know how much "stuff" we're looking at or the change rate,
> or the technical level of the updaters, so that may or may not be convenient
> for you. It makes deployment simple and automatic though, as we just check
> out the repo on a schedule and it's magically "live".

This sounds great, let's use GitHub.

> It's a bit more painful cost-wise
> and setup-time-wise to host CDN with a vanity domain like this
> (trackertest.org) as opposed to our standard (<something>.cdn.mozilla.net

It's a requirement that all assets are served from trackertest.org. So, unfortunately, we will need a vanity domain name.

If GH is used with a CDN, will we be able to merge images into master and have those auto CDN'ified?

Thanks
Let me know if anything needs further clarification before we can move forward. Thanks!
Flags: needinfo?(rwatson)
We also need to host fake tracking assets (i.e. a single static JS file) as part of bug 1149825. I suggest we roll all of our fake trackers into the same repo and point all of our test domains to a single site.

I created a repo for this and can add anyone that needs to push to it to the contributors (currently me and MattN): https://github.com/mozilla/itisatracker.org
(In reply to François Marier [:francois] from comment #6)
> We also need to host fake tracking assets (i.e. a single static JS file) as
> part of bug 1149825. I suggest we roll all of our fake trackers into the
> same repo and point all of our test domains to a single site.
> 
> I created a repo for this and can add anyone that needs to push to it to the
> contributors (currently me and MattN):
> https://github.com/mozilla/itisatracker.org

Hi :francois

This ticket is working with WebOps to create a resilient host. Our site will be subjected to production-level traffic, so we are going to go ahead and follow IT's lead here as they create the GH repo for trackertest.org.

Part of the integration will be setting up a Git push that will auto deploy merges on the master branch. If you'd like to meet sometime next week, we can discuss the possibility of your team utilizing this workflow for your needs. Feel free to email me (cprice@mozilla.com).

Thanks!
(In reply to Cory Price [:ckprice] from comment #7)
> This ticket is working with WebOps to create a resilient host. Our site will
> be subjected to production-level traffic, so we are going to go ahead and
> follow IT's lead here as they create the GH repo for trackertest.org.

Sure, I don't mind what the repo is called or how it's setup. I was simply saying that we also need to host a mostly empty static file (https://github.com/mozilla/itisatracker.org/blob/master/tracker.js) and then point two more domains at that same host (itisatracker.com and itisatracker.org).

If we can get that done, then I don't need to be involved or have access to anything :)
Hi :jakem,

Just wondering on status for this, or if you need any more clarification.

Also, could you confirm per comment 8 that we'll be able to add CNAME's for itisatracker.com (and/or .org).

Thanks!
Flags: needinfo?(nmaul)
Note: the itisatracker.org and itisatracker.com domains are not yet transferred to Mozilla (they are in my personal account) but I am happy to point DNS to the mozilla servers if that's easier for you to manage.

Let me know what servers you want me to use and I'll update the registrar.
Just updating this bug. We have a demo we are running this month.

Is it possible to see the hosting set up before August 12th?

Thanks
Due Date: 2015-08-14 → 2015-08-12
https://github.com/mozilla/trackertest/ is set up.

:ckprice admin access
:fox2mike admin access.

:francois once the GH push and domains are set up, you should be able to open PR's against master.

Changing due date for completion this week.
Due Date: 2015-08-12 → 2015-08-14
Flags: needinfo?(rwatson)
Flags: needinfo?(nmaul)
I'll get to this tomorrow.
Assignee: nmaul → smani
Threw in a placeholder html page and this is live http://trackertest.org/ 

We can do SSL and the other changes next week.
Shyam, how would you like me to CNAME the itisatracker.org domain?

Ideally, I'd like to just point the DNS to Mozilla DNS servers, but I'm happy to do whatever you think is best.
Flags: needinfo?(smani)
François,

Totally upto you. If you'd like us to manage DNS, you're welcome to point to our nameservers and I'll do the rest.
Flags: needinfo?(smani)
(In reply to Shyam Mani [:fox2mike] from comment #16)
> Totally upto you. If you'd like us to manage DNS, you're welcome to point to
> our nameservers and I'll do the rest.

Great, I will do that. What nameservers should I use?
Flags: needinfo?(smani)
NS5-65.AKAM.NET
NS7-66.AKAM.NET
NS4-64.AKAM.NET
NS1-240.AKAM.NET
Flags: needinfo?(smani)
Done. itisatracker.org is now using the above DNS servers.
Hi I am picking this up while Cory is on PTO.

Based on the above am I correct that this bug is resolved. It looks like DNS is working and since this is hosted on GH pushes to the repo are automagically live.

Please resolve if everything here checks out.

Thanks!
Flags: needinfo?(francois)
I think two things are missing:

1. itisatracker.org should resolve to the same thing as trackertest.org
2. trackertest.org is not automatically tracking the https://github.com/mozilla/trackertest repo because it's missing tracker.js
Flags: needinfo?(francois)
Shyam - can you comment on items 1 & 2 above.

Additionally for my knowledge can you confirm this is hosted on our infra and just has a webhook to this repo to update? Reading the above for a moment I thought this might have been hosted using GH pages or something.

We should document how this is setup on mana.
Flags: needinfo?(smani)
Checking in on this.

#1 and #2 seem to be working. Francois can you verify?

We still need to confirm that we are auto-updating based on a merge to master. Once that is sorted we can resolve.

Thanks!
Flags: needinfo?(francois)
Hey Ben, 

yeah, I setup 1 & 2 yesterday. I'll get the auto update going in a bit and comment here.
Flags: needinfo?(smani)
Awesome and as always many thanks.
Flags: needinfo?(francois)
sysadmins r107513 has the auto-update portion. It'll go live once the next puppet run completes (45-60 mins).
Looks good to me too, thanks!
Francois can you make a small commit to master to verify auto-update works, once that is sorted we can resolve.

Thanks again for the help on this.
Flags: needinfo?(francois)
(In reply to Ben (:bensternthal) from comment #28)
> Francois can you make a small commit to master to verify auto-update works,
> once that is sorted we can resolve.

Unfortunately, I don't have write access to that repo. Only Cory does I think.

We should probably keep the bug open until https://trackertest.org/ works (over HTTPS).
Flags: needinfo?(francois)
Ill see if i can be "blessed" with admin to test. HTTPS is covered in bug 1185008 so we are OK to close this one once we verify auto-update works.
(In reply to Ben (:bensternthal) from comment #30)
> Ill see if i can be "blessed" with admin to test. HTTPS is covered in bug
> 1185008 so we are OK to close this one once we verify auto-update works.

If you guys can give me your github IDs, I can add you to the repo :)
(In reply to Shyam Mani [:fox2mike] from comment #31)
> If you guys can give me your github IDs, I can add you to the repo :)

I'm https://github.com/fmarier/
Looks like I was added. 

I merged a small change to master 45 minutes ago and have not seen it update (perhaps it's every hour). I looked at setting and did not see any webhooks or services set. Unsure how the auto-update here works so maybe that is not an issue.
Ok its been a few hours and the merged change to master has not appeared. 

Shyam can you double check the setup?

Thanks.
Flags: needinfo?(smani)
Howdy, can you give an eta on the auto update being fixed, we were aiming to have this sorted by EOD today.

Thanks again.
Flags: needinfo?(smani)
Flags: needinfo?(smani)
Can you provide an update here? Sorry to pester but I am trying to close this out on schedule.
emailed for update
Added the SSL to the static-SAN
C- can you jump in and help here, I have been unable to get an update for a few days on this one.

Based on https://bugzilla.mozilla.org/show_bug.cgi?id=1184773#c26 Shyam thought auto-update would be working however it is not.

Thanks in advance.
Flags: needinfo?(cliang)
OK, I have been poking at this, this morning. I think I have found the problem.

I have updated the config. 
FYI the cron is set to:
20,50 * * * * 

Can you test and let me know?
Flags: needinfo?(smani)
Flags: needinfo?(cliang)
Tested small change, auto deploy worked as expected.

Thanks Ryan!
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Status: RESOLVED → VERIFIED
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.