2003932 - Accesses to /source/* on Phabricator are CPU intensive and possible cause of past outages

Status: NEW

(Conduit :: Phabricator, enhancement, P2)

Description

[David Lawrence [:dkl]]

Bots seem to be scraping /source/* which is a CPU intensive endpoint--syntax highlighting and other calculations.

Possible ideas going forward:

• shut off /source/* feature altogether
• challenge on all /source/* requests
• maybe switch to interactive challenges (we think we're using non-interactive challenges currently)
• rewrite /source/* requests to github urls and redirect

We should look at auditing when and if access to /source/* is truly needed steps to mitigate future abuse going forward.

Comment 1

[David Lawrence [:dkl]]

One thing we can try on prod and see if/when something breaks is to configure the Diffusion application to only be accessible to logged in users.

You do this by:

1. Go to Applications.
2. For Diffusion, choose Configure to the far right.
3. Set "Can Use Application" policy to "All Users". This is the same as saying must be logged in.
4. Set "View Policy" to "All Users".
5. Save changes.

When accessing anything under /source/ it will ask to Login first if not already. Otherwise it looks the same. This should deter most bot attacks who are hitting /source/ unauthenticated. And we can see if getting the Login screen causes the same CPU load as getting let through which I would assume would not.

Comment 2

[David Lawrence [:dkl]]

Would need to see if this would negative affect any external tools that need to access files from /source/ and do not current authenticate. I would think they would already use the Conduit API but not sure.

Comment 3

[David Lawrence [:dkl]]

(In reply to David Lawrence [:dkl] from comment #0)

• shut off /source/* feature altogether
• challenge on all /source/* requests

FWIW, we already have this in place via Fastly and seems to have been working up til now.

• maybe switch to interactive challenges (we think we're using non-interactive challenges currently)

• rewrite /source/* requests to github urls and redirect

Not sure how feasible this is without hacking the Phabricator code to redirect to any code that is mirrored from Github.

Comment 4

[Olivier Mehani [:shtrom]]

See also bug 2002695 from the previous incident.