718155 - No longer able to view raw html files direct from hg (reftest-analyzer broken & other pages)

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Boris Zbarsky [:bzbarsky]]

Clicking the "reftest analyzer" links on tbpl gives a prompt to download.  This is because https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml is now being served up as "Content-Type: application/binary" instead of a useful type like text/html or whatnot.

This sort of gets in the way of getting data out of try server and other trees...

Comment 1

[Boris Zbarsky [:bzbarsky]]

In fact, almost certainly fallout from the second item at http://mercurial.selenic.com/wiki/UpgradeNotes#A1.9.1:_guessmime.2C_revert_behavior_restored as philor points out.

Comment 2

[Phil Ringnalda (:philor)]

Depending on how much other fallout from that there is (I assume it was some other sort that was getting mentioned before), we could probably fork reftest-analyzer.xhtml into tbpl - it only got 8 patches in 6 pushes last year.

Comment 3

[Axel Hecht [:Pike]]

I'd love us to restore the old behaviour, I've got a good deal of pages that I use straight off of hg instead of bothering about a deployment.

Comment 4

[Mark Banner (:standard8)]

I too also have some html pages that I've been serving from hg, e.g.

http://hg.mozilla.org/users/bugzilla_standard8.plus.com/drivertools/raw-file/default/bugtracking/index2.html

Whilst these could move to people, deployment is more of a pain.

Comment 5

[Corey Shields [:cshields]]

I've filed 718180 for infrasec to review and sign off on this since serving raw as binary was a security measure

Comment 6

[Michael Coates [:mcoates] (acct no longer active)]

I'd like to get a better understanding of how we're using this system and the impact of the issue so we can understand the risk of reverting this security control.

My understanding of the issue is that previously, an html file within source could be clicked on through the hg website and the html file would be served and rendered from hg.mozillg.org. Is this correct?

A few other questions:

1. I followed the above link in comment 4 and was prompted to "open with" as a file download. Opening with Firefox renders the html as a local file. I'm assuming the problem is that the file needs to run on the server to interact with server side data? Is this correct?

2. Are these html files uploaded by users of the hg site? Or did an admin log on to the hg box and add these files somewhere? Could anyone upload an html file? 

3. Does hg.mozilla.org support login for any users?

4. If this change wasn't reverted, what actions would have to be taken to get everything working as needed? E.g what's the impact of this issue?

Comment 7

[Axel Hecht [:Pike]]

You can see the actual html files if you go to the file instead of the raw-file url. In my case, that'd be http://hg.mozilla.org/users/axel_mozilla.com/fx-triage/file/default/new-requests.html. That's a web page that uses the bugzilla api to do some requests and display them. There's no server-side data at least in my case, just html, css, and js. As such, there's also nothing to log in to, in my case. one could probably log in with bugzilla credentials to file bugs etc.

Without serving the web files as html/js/css etc, one would need to actually deploy the code in parallel to publishing its source. I suspect that folks would mass move to either gh-pages or readthedocs to get files published on the web on push.

Comment 8

[Michael Coates [:mcoates] (acct no longer active)]

Others, 

Do the ideas suggested in comment 7 work? Or are there other concerns we should explore?

Comment 9

[Corey Shields [:cshields]]

(In reply to Michael Coates [:mcoates] from comment #6)
> 3. Does hg.mozilla.org support login for any users?

hg ssh connections go to one host which then has a jailed command that allows for a few select hg functions - no interactive ssh sessions are allowed.  hg http sessions go to a pool of servers that serve hgweb with no end-user ssh connectivity allowed.  All of these servers share /repo/hg via nfs.

> 4. If this change wasn't reverted, what actions would have to be taken to
> get everything working as needed? E.g what's the impact of this issue?

I'm still learning of impact here..  I saw mention on irc of an xml file breaking some external app because of its new mime type.  Hopefully we can collect these more specifically here in the bug.

Comment 10

[Boris Zbarsky [:bzbarsky]]

> I'm assuming the problem is that the file needs to run on the server to interact with
> server side data?

The file needs to run.  I don't think it _necessarily_ needs to run on hg.mozilla.org (in particular, reftest-analyzer is not getting the logs it operates on from hg.mozilla.org, obviously).

> Are these html files uploaded by users of the hg site? 

These are just files checked into some hg repository.

> Could anyone upload an html file? 

Anyone who has permission to check into a repository that lives on hg.mozilla.org

> Does hg.mozilla.org support login for any users?

You have to have an LDAP account with the right bits set to push to repositories on hg.mozilla.org.  Which bits depends on the repository.

Not sure if that's what you're asking.

> what actions would have to be taken to get everything working as needed?

If we keep the current hg setup, you'd need to set up some place which runs a web server, checks out the relevant files, and serves them up as HTML/JS/CSS/whatever.  You'd then need infrastructure to watch pushes to hg.mozilla.org and check out files to this web server.  Probably for all repositories stored on hg.mozilla.org, given current usage patterns.

Back to the security concerns....  The only thing a webpage stored on hg.mozilla.org can do that other web pages cannot do is do random binary POST to hg.mozilla.org and read arbitrary data exposed via GET urls on hg.mozilla.org.  If we don't have anything on hg.mozilla.org that's available via GET but not "world readable" and if it doesn't do anything interesting with POST requests, then I don't think there's a security issue.  ccing Jonas to verify that.

Given that and the fact that only people who have push access to some repository already can get files up on hg.mozilla.org, I suspect it's probably safe to revert to the old behavior.

Comment 11

[Michael Coates [:mcoates] (acct no longer active)]

If we're controlling access to hg repositories (and hence to hg) to only trusted users then we should be ok.  

In addition, if a malicious html file was somehow uploaded it sounds like the hg website doesn't support login. Therefore there's really not much for a malicious html file to do against the hg site specifically (e.g. no concerns of XSS stealing cookies since there's no user accounts)

Unless Jonas adds anything new of concern then I'm ok reverting back to old behavior for this specific website (note: we need to look at any other sites on a case by case basis)

Comment 12

[Corey Shields [:cshields]]

I've set guessmime back on the web heads.  This should all be fixed.

Comment 13

[(not currently active) Ted Mielczarek]

Can someone CC me on bug 718180?