Need access to web server w/o requiring MPT VPN

RESOLVED FIXED

Status

Infrastructure & Operations
NetOps: DC ACL Request
P1
major
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: Annie Elliott, Unassigned)

Tracking

Details

(Whiteboard: [triaged 20120824])

Attachments

(3 attachments)

(Reporter)

Description

5 years ago
Metrics new viz server, http://tableau1.metrics.scl3.mozilla.com resides in our DC much like our old pentaho viz server does. 

Folks do not need MPT VPN credentials to access Pentaho.

Please open up the firewall to allow similar access as is currently allowed to to the old pentaho server (which is at https://metrics.mozilla.com/pentaho)
Passing to the serverops queue for load balancer setup.
Assignee: network-operations → server-ops
Blocks: 784775
Component: Server Operations: Netops → Server Operations
QA Contact: ravi → jdow
Annie,

This bug is confusing. What exactly would you like to do? 

metrics.mozilla.com/pentaho needs LDAP auth, even today. It's not public. Also, I'm marking 784775 as a dupe of this bug, since they both seem similar.
Assignee: server-ops → server-ops-webops
Component: Server Operations → Server Operations: Web Operations
QA Contact: jdow → cshields

Updated

5 years ago
Duplicate of this bug: 784775
(In reply to Shyam Mani [:fox2mike] from comment #2)

> metrics.mozilla.com/pentaho needs LDAP auth, even today. It's not public.

I also forgot to add, it doesn't need MPT-VPN. Anyone with the right LDAP account can access it.
(Reporter)

Comment 5

5 years ago
http://tableau1.metrics.scl3.mozilla.com is also not public; requires a login. We pay per seat. There are only 10 seats at present. 

I would like for folks who have a license to not have to start the MPT-VPN before they can resolve the host name.

Presently, if I disconnect from MPT-VPN, I can no longer resolve the host. I get:

Server not found
      
          Firefox can't find the server at tableau1.metrics.scl3.mozilla.com.

I would like for folks like you said who are "with the right LDAP account" to be able to hit that server w/o having to run MPT-VPN.

Comment 6

5 years ago
In order to do this, this needs a public domain name (metrics-new.mozilla.org?), and a Zeus VIP. We can do that. Let us know what name you'd like... we shouldn't just use 'tableau1.metrics.scl3.mozilla.com'... that's a server name, and we typically like to distance those from web-accessible domain names.

This also needs some sort of OpSec/AppSec review before we can expose this more widely. Setting the sec-review? flag. If you already have a bug where this was reviewed, please let us know. Thanks!
Flags: sec-review?
Priority: -- → P3
Whiteboard: [triaged 20120824][waiting][appsec][opsec]
(Reporter)

Comment 7

5 years ago
dataviz.mozilla.org

Yes, please don't use the really long and not remember-able server name :)
(Reporter)

Comment 8

5 years ago
Created attachment 655137 [details]
security whitepaper

For purposes of the review, I've attached a 19-page white paper on security for the platform, and it occurs to me to mention that they have stuff like row-level security, etc. available. There is more info on their website as well, and I can get an SE to answer questions on top of that.
we will likely need to do some web sec testing but giving to joes
Flags: sec-review? → sec-review?(jstevensen)

Comment 10

5 years ago
Renormalizing P-levels... "P4" is normal now.
Priority: P3 → P4
(Reporter)

Comment 11

5 years ago
Hi - what is the ETA for this? We need it rolled out.
Annie, 

I'll take a look at this and we'll get back to you today.

Updated

5 years ago
Assignee: server-ops-webops → jstevensen
(Reporter)

Comment 13

5 years ago
Ok. If you need access to a Tableau FSE, let me know and we'll get it scheduled ASAP. Thanks!
(Reporter)

Comment 14

5 years ago
:joes Hi... pardon the hand wringing, but could you give an update and an ETA? I am willing to help with leg work et al at this point. It is blocking my quarterly goals.
Discussed this with Annie.

Approving firewall ACL access to tableau that mirrors what we are doing with pentaho.
Curtisk, 

Will this need a secreview?
not that I can see.
I took a look at the whitepaper and this answers most of what we normally ask (https://wiki.mozilla.org/Security/Reviews/Review_Request_Form#Questions_to_Address_within_Request_Body)
These 2 are missing but don't give me heartburn:
* What is your disclosure policy to customers in the event of a compromise of your servers, applications or any related infrastructure that interacts with the applications holding Mozilla data?
* Have you suffered a security compromise in the past 24 months? If so, please provide details and remediation that occurred as a result.

Since this is in house we should be able to test against it with our own tools as we see fit. This is also only accessible to a small number of LDAP credentials.

I am fine with this moving forward, if we want to test more extensively we can at a later date.
(Reporter)

Comment 19

5 years ago
:curtisk Thank you.

:joes So... given ^^, when could it be exposed as discussed?
Yes, this can be done now. I'll assign this to netops.
(Reporter)

Comment 21

5 years ago
I went ahead and pushed that button :)
(Reporter)

Updated

5 years ago
Assignee: jstevensen → network-operations
Component: Server Operations: Web Operations → Server Operations: Netops
QA Contact: cshields → ravi
This is for server ops unless this can't be done with the load balancer.
Assignee: network-operations → server-ops
Component: Server Operations: Netops → Server Operations
QA Contact: ravi → jdow

Comment 23

5 years ago
Yep, this is a WebOps thing now... moving to the proper component.

Need to make a VIP and DNS for dataviz.mozilla.org, SCL3 public Zeus cluster. Backend pool is just one node, tableau1.metrics.scl3.mozilla.com.
Assignee: server-ops → server-ops-webops
Component: Server Operations → Server Operations: Web Operations
QA Contact: jdow → cshields
Whiteboard: [triaged 20120824][waiting][appsec][opsec] → [triaged 20120824]
Webops,

This site should not be publicly accessible from the internet. It has not been reviewed for that. The request was for Mozilla employees to access this without the VPN. So ACLs should only permit access from the office networks.
Please hold off on opening up access. I'll work with annie to clarify what she needs.

Updated

5 years ago
Assignee: server-ops-webops → jstevensen
Taking this bug as this server is not ready for being exposed to the internet.
If this is going to be "internet" available then we are going to have to do a full review of the interfaces. As I understand it this server will have access to sensitive MoCo information. It should either be accessible on our network only or ensured that it is hardened to a very high degree.
Curtis, 

As I understand it now, Annie wants this open to the internet, so I directed her to the SecReview form.
Spoke with Annie.

This host isn't "internet-ready". Matthew Laraine, our Windows sys-admin, is going to work with Annie to get this system hardened, monitored, and whatever else needs to be done. 

Once done, we'll open up access to this host from the office network and VPN networks. 

After a formal security review is done of the vendor's software (Annie will fill out the request for this), and remediation work is complete, we can put this behind a Zeus VIP.

Updated

5 years ago
Assignee: jstevensen → mlarrain

Updated

5 years ago
Status: NEW → ASSIGNED
Unfortunately Matt is dedicated to working on release engineering infrastructure and already has several high priority projects that are behind schedule.  Sorry we don't have the cycles to help out here.
Assignee: mlarrain → jstevensen
Annie,

I'm trying to find someone from IT to help get your server up and running in a production environment. 

In the meantime, Michal Purzynski from my team is reviewing this on the security side.
Hi,

I've made a quick review of the Windows installation. I've found no critical problems that would block deployment for now and access via office and VPN only.

Of course this system is not yet ready for the Internet access.

I've found that:
1. Windows is not updated and Windows updates are not configured. That's the only thing that must be corrected before deployment.
2. a firewall is a bit more relaxed than it should be, allowing IPv6 and Teredo what in practice allows for Internet access (in and out), depending on our firewall in that zone. I've corrected the settings and IPv6 and Teredo are now disabled.
3. All of the services are running under a Network Service account. This includes all of applications processes, httpd, postgres. While it is better than most, because it is a quite limited account in Windows I'd rather see them separated before we allow Internet access.
4. No hardening has been done in form of EMET. I'd like to test the product under EMET and if it just works would use it on production. Same goes for httpd and postgres, which won't have issues.
5. System must be connected to the Active Directory domain.
6. All of httpd, postgres and application processes are running with permanent DEP and ASLR. Very good!

To make it clear - I'm fine with allowing access to that system from office and VPN networks for now and will send a list of changes to be made before it goes to Internet.
(Reporter)

Comment 33

5 years ago
:michal`  - Thank you!

I am confused on one point - For 1. above, is the Windows updates feature to be on before or after acess via office and VPN only?

:joes - It the short term to get this accessible, it looks like 0 to 1 things (ie, possibly the windows update configuration) needs to be handled. Would a person be available to do that soon?

thank you both again so much!
(Reporter)

Comment 34

5 years ago
Created attachment 665961 [details]
shows updates completed successfully

see attached
(Reporter)

Comment 35

5 years ago
Created attachment 665965 [details]
Shows automatic updates enabled.
(Reporter)

Comment 36

5 years ago
On 9/28/12 1:53 PM, Annie Elliot wrote:

Hi -

Is the scan done?

I am in a desperate home-stretch struggle to get this thing user-facing
today (actually, earlier this week but
it is what it is at this point.)


You can proceed without the scan since the updates were done.


-- 
Joe Stevensen
Operations Security Manager
jstevensen@mozilla.com
Netops, 

Can you please make this server accessible from office (MV, SFO, YVR, TOR) and the remote access VPNs?
Assignee: jstevensen → network-operations
Component: Server Operations: Web Operations → Server Operations: Netops
QA Contact: cshields → ravi

Comment 38

5 years ago
Sure.  This flow is not trivial to implement as office->DC flows have always required the VPN.  That said this will take a little time to sort out before we can begin adding the flows.
Component: Server Operations: Netops → Server Operations: ACL Request
Priority: P4 → --
Guys, ETA for this please?
Severity: normal → major

Updated

5 years ago
Assignee: network-operations → cransom
opened and verified from the admin boxes, users have the same access:
0beast:~% for f in mtv1 sfo1 yvr1 tor1; do ssh admin1a.private.$f.mozilla.com nc -vz 10.22.27.136 80; done
Connection to 10.22.27.136 80 port [tcp/http] succeeded!
Connection to 10.22.27.136 80 port [tcp/http] succeeded!
Connection to 10.22.27.136 80 port [tcp/http] succeeded!
Connection to 10.22.27.136 80 port [tcp/http] succeeded!
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
and just to clarify, this is all offices, not just the ones mentioned.
(Reporter)

Comment 42

5 years ago
Why was this closed? I still cannot access it at the agreed upon name!

As per https://bugzilla.mozilla.org/show_bug.cgi?id=784775, it should be accessible by dataviz.mozilla.com, from an office (I am currently sitting in SF) with no VPN.

I get:

Server not found
        
          Firefox can't find the server at dataviz.mozilla.com.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 43

5 years ago
Can someone who set this up explain why we didn't and/or couldn't use a public IP here to prevent having a top level mozilla.* host from having a private IP?

Comment 44

5 years ago
Just so we are on the same page this is a temporary solution.  Once the server passes sec review it will need to get a public IP.  How that happens is irrelevant.  It is just important to make that clear in this bug.
This server is a temporary solution till the application will go through a full security review.

Only after an ACK from the App Sec team and again OpSec review we can allow that server to be accessed from Internet.

See the comment #29
(Reporter)

Comment 46

5 years ago
I would be *more* than happy to have it accessible by 

http://metrics.mozilla.com/dataviz

If that makes like easier and can get this done faster.
(Reporter)

Comment 47

5 years ago
So, can giving it an alias not happen until it is put this behind a Zeus VIP?
[cransom@admin1a.private.tor1 ~]$ nc -vz dataviz.mozilla.org 80
Connection to dataviz.mozilla.org 80 port [tcp/http] succeeded!

passing to joes for further security review.
Assignee: cransom → jstevensen
(Reporter)

Comment 49

5 years ago
THANK YOU!!!!!!
(In reply to Annie Elliott from comment #47)
> So, can giving it an alias not happen until it is put this behind a Zeus VIP?

Zeus VIP is one thing, but completely separate.

It goes public after and only after AppSec team says it's safe to use.
(Reporter)

Comment 51

5 years ago
Something is not working...

The primary consumer of stuff on that server… sitting in SF with me, on the same wireless network (Mozilla), with neither of us connected to any VPN, could not see that server either with the pretty name or the full name.
Priority: -- → P1
@NetOps - this system should be accessible from the office (all of them) and VPN.

Only the access from Internet was questioned for now, here.
Annie, but you were able to access the site? What's your IP address.
Please give us the exact error messages as well.

Comment 55

5 years ago
From a host *IN* sfo mimicking what a regular user would encounter shows things working:

[root@admin1a.private.sfo1 ~]# curl --interface 10.251.24.6 http://dataviz.mozilla.org/
<html><body>You are being <a href="http://dataviz.mozilla.org/auth?destination=%2F">redirected</a>.</body></html>
(Reporter)

Comment 56

5 years ago
I am.

Here is my info:

"Wi-Fi is connected to Mozilla and has the IP address 10.251.25.193."

Laura's IP address is IP address 10.251.28.119.
(Reporter)

Comment 57

5 years ago
She gets the vanilla

Firefox cannot find the server dataviz.mozilla.org

Comment 58

5 years ago
Joel (Desktop) and AJ (SRE) are in the office and verified that each were able to hit the page and is got the same auth page as we have seen in our testing.

I've asked Joel if he can stop by your desk to look into why you appear to not be able to reach the site while others can.
(Reporter)

Comment 59

5 years ago

Just to be clear. *I* can reach it. Laura Forrest cannot.
(Reporter)

Comment 60

5 years ago
I had several colleagues try. They could all reach the login page for the server. It appears to be just laura, who unfortunately is the main consumer at the moment.
Just checked in with Laura and mrz resolved the issue, which was related to DNS. Laura can now access the site.

Updated

5 years ago
Assignee: jstevensen → network-operations

Updated

5 years ago
Flags: sec-review?(jstevensen)
I don't see any remaining issues here.
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.