Hosting for trackertest.org

VERIFIED FIXED

Status

VERIFIED FIXED
3 years ago
2 years ago

People

(Reporter: ckprice, Assigned: fox2mike)

Tracking

Details

(Whiteboard: [kanban:https://webops.kanbanize.com/ctrl_board/2/1432] )

We have an odd little bird we'd like to see if you guys can help us set up. See bug 1177978 for more information.

In bug 1184735 you set up a domain for us (thanks!).

The specifications for this page are pretty simple

- HTML/CSS/JS only
- No backend
- Good amount of traffic, should be CDN'd (will be serving images to high traffic pages)

So, we'd need it hosted somewhere that we could upload assets (images, HTML, CSS, JS) to. It's not going to be a complicated site.

Let me know what all else is needed here.

Updated

3 years ago
Whiteboard: [kanban:https://webops.kanbanize.com/ctrl_board/2/1432]

Updated

3 years ago
Assignee: server-ops-webops → nmaul
See Also: → bug 1185008
Blocks: 1185008
See Also: bug 1185008

Comment 1

3 years ago
:w0ts0n and I will work on this.

First thing is, would it be reasonable for you to shove these assets into a source control repo somewhere (github, git.m.o, hg.m.o, svn.m.o... doesn't matter)? I don't know how much "stuff" we're looking at or the change rate, or the technical level of the updaters, so that may or may not be convenient for you. It makes deployment simple and automatic though, as we just check out the repo on a schedule and it's magically "live".


If that's no good, we can look at other options. The next choice might be an S3 bucket that you can upload to directly. Archive data and change management would be on you (that is, whomever is updating it)- we wouldn't be able to help with rollbacks, for example.


CDN is fairly straightforward either way. It's a bit more painful cost-wise and setup-time-wise to host CDN with a vanity domain like this (trackertest.org) as opposed to our standard (<something>.cdn.mozilla.net, or something vendor-provided like <something>.cdn.cloudfront.net), so if the CDN name doesn't matter to you we can optimize that. Your stuff would still be available at trackertest.org, but that would be a non-CDN version of it... so not good for high traffic.
We can push to GitHub np
I actually wondered about github. Their resiliency seems good enough unless China is hacking them. Would we consider putting something mission-critical there?

An s3 bucket would also be fine (archiving is not a concern).

The nature of the site is a concern. This is a domain that Mozilla is hosting that will contain no tracking. Ideally we're even in control of server logs, but it's not a requirement. 

Definite requirements are:

1. Use own host name (trackertest.org). This name is going to be blocklisted on purpose as a fake tracking domain. 
2. We cannot have any other third-party loads on the site. Definitely no analytics or other third-party assets that would track. 
3. Simple site, a single flat page linking to some sub resources (a few images, some small sample JS files)

The whole concept is that this is a developer resource that simulates a bad thing (tracking) while offering pure safety because it isn't doing that bad thing.
Group: mozilla-employee-confidential
Hi :jakem,

Thanks for looking after this.

Spoke with Javaun and team...

(In reply to Jake Maul [:jakem] from comment #1)
> :w0ts0n and I will work on this.
> 
> First thing is, would it be reasonable for you to shove these assets into a
> source control repo somewhere (github, git.m.o, hg.m.o, svn.m.o... doesn't
> matter)? I don't know how much "stuff" we're looking at or the change rate,
> or the technical level of the updaters, so that may or may not be convenient
> for you. It makes deployment simple and automatic though, as we just check
> out the repo on a schedule and it's magically "live".

This sounds great, let's use GitHub.

> It's a bit more painful cost-wise
> and setup-time-wise to host CDN with a vanity domain like this
> (trackertest.org) as opposed to our standard (<something>.cdn.mozilla.net

It's a requirement that all assets are served from trackertest.org. So, unfortunately, we will need a vanity domain name.

If GH is used with a CDN, will we be able to merge images into master and have those auto CDN'ified?

Thanks
Let me know if anything needs further clarification before we can move forward. Thanks!
Flags: needinfo?(rwatson)
We also need to host fake tracking assets (i.e. a single static JS file) as part of bug 1149825. I suggest we roll all of our fake trackers into the same repo and point all of our test domains to a single site.

I created a repo for this and can add anyone that needs to push to it to the contributors (currently me and MattN): https://github.com/mozilla/itisatracker.org
(In reply to François Marier [:francois] from comment #6)
> We also need to host fake tracking assets (i.e. a single static JS file) as
> part of bug 1149825. I suggest we roll all of our fake trackers into the
> same repo and point all of our test domains to a single site.
> 
> I created a repo for this and can add anyone that needs to push to it to the
> contributors (currently me and MattN):
> https://github.com/mozilla/itisatracker.org

Hi :francois

This ticket is working with WebOps to create a resilient host. Our site will be subjected to production-level traffic, so we are going to go ahead and follow IT's lead here as they create the GH repo for trackertest.org.

Part of the integration will be setting up a Git push that will auto deploy merges on the master branch. If you'd like to meet sometime next week, we can discuss the possibility of your team utilizing this workflow for your needs. Feel free to email me (cprice@mozilla.com).

Thanks!
(In reply to Cory Price [:ckprice] from comment #7)
> This ticket is working with WebOps to create a resilient host. Our site will
> be subjected to production-level traffic, so we are going to go ahead and
> follow IT's lead here as they create the GH repo for trackertest.org.

Sure, I don't mind what the repo is called or how it's setup. I was simply saying that we also need to host a mostly empty static file (https://github.com/mozilla/itisatracker.org/blob/master/tracker.js) and then point two more domains at that same host (itisatracker.com and itisatracker.org).

If we can get that done, then I don't need to be involved or have access to anything :)
Hi :jakem,

Just wondering on status for this, or if you need any more clarification.

Also, could you confirm per comment 8 that we'll be able to add CNAME's for itisatracker.com (and/or .org).

Thanks!
Flags: needinfo?(nmaul)
Note: the itisatracker.org and itisatracker.com domains are not yet transferred to Mozilla (they are in my personal account) but I am happy to point DNS to the mozilla servers if that's easier for you to manage.

Let me know what servers you want me to use and I'll update the registrar.
Just updating this bug. We have a demo we are running this month.

Is it possible to see the hosting set up before August 12th?

Thanks
Due Date: 2015-08-14 → 2015-08-12
https://github.com/mozilla/trackertest/ is set up.

:ckprice admin access
:fox2mike admin access.

:francois once the GH push and domains are set up, you should be able to open PR's against master.

Changing due date for completion this week.
Due Date: 2015-08-12 → 2015-08-14
Flags: needinfo?(rwatson)
Flags: needinfo?(nmaul)
(Assignee)

Comment 13

3 years ago
I'll get to this tomorrow.
Assignee: nmaul → smani
(Assignee)

Comment 14

3 years ago
Threw in a placeholder html page and this is live http://trackertest.org/ 

We can do SSL and the other changes next week.
Shyam, how would you like me to CNAME the itisatracker.org domain?

Ideally, I'd like to just point the DNS to Mozilla DNS servers, but I'm happy to do whatever you think is best.
Flags: needinfo?(smani)
(Assignee)

Comment 16

3 years ago
François,

Totally upto you. If you'd like us to manage DNS, you're welcome to point to our nameservers and I'll do the rest.
Flags: needinfo?(smani)
(In reply to Shyam Mani [:fox2mike] from comment #16)
> Totally upto you. If you'd like us to manage DNS, you're welcome to point to
> our nameservers and I'll do the rest.

Great, I will do that. What nameservers should I use?
Flags: needinfo?(smani)
(Assignee)

Comment 18

3 years ago
NS5-65.AKAM.NET
NS7-66.AKAM.NET
NS4-64.AKAM.NET
NS1-240.AKAM.NET
Flags: needinfo?(smani)
Done. itisatracker.org is now using the above DNS servers.

Comment 20

3 years ago
Hi I am picking this up while Cory is on PTO.

Based on the above am I correct that this bug is resolved. It looks like DNS is working and since this is hosted on GH pushes to the repo are automagically live.

Please resolve if everything here checks out.

Thanks!
Flags: needinfo?(francois)
I think two things are missing:

1. itisatracker.org should resolve to the same thing as trackertest.org
2. trackertest.org is not automatically tracking the https://github.com/mozilla/trackertest repo because it's missing tracker.js
Flags: needinfo?(francois)

Comment 22

3 years ago
Shyam - can you comment on items 1 & 2 above.

Additionally for my knowledge can you confirm this is hosted on our infra and just has a webhook to this repo to update? Reading the above for a moment I thought this might have been hosted using GH pages or something.

We should document how this is setup on mana.
Flags: needinfo?(smani)

Comment 23

3 years ago
Checking in on this.

#1 and #2 seem to be working. Francois can you verify?

We still need to confirm that we are auto-updating based on a merge to master. Once that is sorted we can resolve.

Thanks!
Flags: needinfo?(francois)
(Assignee)

Comment 24

3 years ago
Hey Ben, 

yeah, I setup 1 & 2 yesterday. I'll get the auto update going in a bit and comment here.
Flags: needinfo?(smani)

Comment 25

3 years ago
Awesome and as always many thanks.
Flags: needinfo?(francois)
(Assignee)

Comment 26

3 years ago
sysadmins r107513 has the auto-update portion. It'll go live once the next puppet run completes (45-60 mins).
Looks good to me too, thanks!

Comment 28

3 years ago
Francois can you make a small commit to master to verify auto-update works, once that is sorted we can resolve.

Thanks again for the help on this.
Flags: needinfo?(francois)
(In reply to Ben (:bensternthal) from comment #28)
> Francois can you make a small commit to master to verify auto-update works,
> once that is sorted we can resolve.

Unfortunately, I don't have write access to that repo. Only Cory does I think.

We should probably keep the bug open until https://trackertest.org/ works (over HTTPS).
Flags: needinfo?(francois)

Comment 30

3 years ago
Ill see if i can be "blessed" with admin to test. HTTPS is covered in bug 1185008 so we are OK to close this one once we verify auto-update works.
(Assignee)

Comment 31

3 years ago
(In reply to Ben (:bensternthal) from comment #30)
> Ill see if i can be "blessed" with admin to test. HTTPS is covered in bug
> 1185008 so we are OK to close this one once we verify auto-update works.

If you guys can give me your github IDs, I can add you to the repo :)
(In reply to Shyam Mani [:fox2mike] from comment #31)
> If you guys can give me your github IDs, I can add you to the repo :)

I'm https://github.com/fmarier/

Comment 33

3 years ago
Looks like I was added. 

I merged a small change to master 45 minutes ago and have not seen it update (perhaps it's every hour). I looked at setting and did not see any webhooks or services set. Unsure how the auto-update here works so maybe that is not an issue.

Comment 34

3 years ago
Ok its been a few hours and the merged change to master has not appeared. 

Shyam can you double check the setup?

Thanks.
Flags: needinfo?(smani)

Comment 35

3 years ago
Howdy, can you give an eta on the auto update being fixed, we were aiming to have this sorted by EOD today.

Thanks again.
Flags: needinfo?(smani)

Updated

3 years ago
Flags: needinfo?(smani)

Comment 36

3 years ago
Can you provide an update here? Sorry to pester but I am trying to close this out on schedule.

Comment 37

3 years ago
emailed for update
Added the SSL to the static-SAN

Comment 39

3 years ago
C- can you jump in and help here, I have been unable to get an update for a few days on this one.

Based on https://bugzilla.mozilla.org/show_bug.cgi?id=1184773#c26 Shyam thought auto-update would be working however it is not.

Thanks in advance.
Flags: needinfo?(cliang)
OK, I have been poking at this, this morning. I think I have found the problem.

I have updated the config. 
FYI the cron is set to:
20,50 * * * * 

Can you test and let me know?
Flags: needinfo?(smani)
Flags: needinfo?(cliang)

Comment 41

3 years ago
Tested small change, auto deploy worked as expected.

Thanks Ryan!
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED

Updated

3 years ago
Status: RESOLVED → VERIFIED
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.