Closed Bug 647509 Opened 15 years ago Closed 15 years ago

Take F1 servers offline due to a privacy issue

Categories

(Mozilla Messaging Graveyard :: Server Operations, defect)

Product:
Mozilla Messaging Graveyard
Component:
Server Operations
Platform:
All
Other
Type:
defect
Priority:
Not set
Severity:
blocker

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: faaborg, Unassigned)

Details

The F1 service is currently putting twitter Direct Messages into people's public twitter streams. This is a very significant privacy violation, and appears to already be known over in bug 644356. In my testing, any DM, regardless of if the user name was correct or not, ended up in my public stream. Since DMs could contain very sensitive personal information, broadcasting them to every single one of your contacts is perhaps one of the biggest privacy violations one of our products could do. I believe the press and general reputation hit we could take from this is serious enough that we should take the servers offline until we can figure out exactly what is going on. We may want to wait for Bryan or David to make the call though. While we commonly do zero day security fire drills, I'm not familiar with us every having a similar experience with a privacy issue. That said, I believe this issue is equally serious.
Assignee: server-ops → server-ops
MoCo IT doesn't have admin logins to MoMo systems. On all is trying to reach someone in their side to see about disabling this function.
Alex, thanks for pointing out the privacy bug which somehow slipped by. The timing's obviously not great (weekend before everyone traveling), but I'm fine w/ taking it down until we can at the very least take out the DM UI. Now, we don't have a procedure in place for this, so I'm not sure what the exact steps we should follow are. I've cc'ed the key players with various levels of access to the existing servers and software. I'll send gozer a page, but I don't know if that will wake him up at 1am or not. It'll be interesting to see what the impact of the outage will be on our In the AM, I'll call Shane and James who are likely most able to disable the DM UI fastest, and we can spin up the servers then. [just got a call from moco IT who'se trying to triage this bug]. [will amend the bug in a minute]
Assignee: server-ops → nobody
Group: infra
Component: Server Operations: Web Content Push → Server Operations
Product: mozilla.org → Mozilla Messaging
QA Contact: mrz → server-ops
James Burke is online and has access to the server. We'll deal with it.
Servers are down. We'll investigate the bug, fix, and spin back up when fixed.
closing this bug, we can deal w/ the actual bug in the original bug.
Status: NEW → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
FYI -- james fixed a bunch of things and spun the server back up. We've tested it and cannot reproduce the bug. We've remove direct messaging from the UI (a change which requires cache invalidation or 24 hours), and also fixed the back-end so that even for users with the old UI cached, the service won't actually broadcast the DM-intended messages, they'll just get a yellow-bar "problem doing the share". We understand the cause of the problem, we've removed the feature that surfaced the bug, and fixed the server. We'll definitely need to do a post-mortem, and a cleanup phase. Thanks to all for prompt action.
I have patches for reenable direct messages in bug 648372.
You need to log in before you can comment on or make changes to this bug.