1441641 - Determine how often branches are pulled from the review repo

Status: RESOLVED FIXED

(MozReview Graveyard :: General, enhancement)

Description

[Māris Fogels [:mars] (please needinfo)]

Part of our data-gathering for the MozReview sunset project.  How often do people pull code from the MozReview review repo?

Comment 1

[Māris Fogels [:mars] (please needinfo)]

Assigning to glob for help.

Comment 2

[Gregory Szorc [:gps]]

This is an audit of the HTTP logs on reviewboard-hg.

`hg pull` can be identified by requests to URLs with "cmd=getbundle" in them.

MozReview also advertises an `hg import` one-liner. Those are URLs like /rev-raw/<40 hex chars> with a User-Agent containing "mercurial."

Comment 3

[:glob ✱]

> ~/dev/mozilla-central$ hg pull -r a8619239a004fc8f4b1d80725c6c24bd76003c95 https://reviewboard-hg.mozilla.org/gecko
generates
> [28/Feb/2018:05:08:55 +0000] "GET /gecko?cmd=getbundle HTTP/1.1" 200 42382561 "-" "mercurial/proto-1.0 (Mercurial 4.5)"

there's been ~28k of these requests to gecko so far this year.

> ~/dev/mozilla-central$ hg import https://reviewboard-hg.mozilla.org/gecko/rev/a8619239a004fc8f4b1d80725c6c24bd76003c95
generates
> [28/Feb/2018:05:14:34 +0000] "GET /gecko/rev/a8619239a004fc8f4b1d80725c6c24bd76003c95 HTTP/1.1" 200 2481 "-" "mercurial/proto-1.0 (Mercurial 4.5)"

there's been ~1k requests of these requests to gecko so far this year.

--

also..

> ~/tmp$ hg clone https://reviewboard-hg.mozilla.org/gecko/
> destination directory: gecko
> requesting all changes
> abort: HTTP Error 413: Request Entity Too Large

this is due to the number of heads (bug 1350136), which are occasionally pruned.
cloning via ssh works, but it's very slow (1.5 hours on my 90Mbps connection, resulting in a 7.7G directory).

the getbundle command is present in the logs for full clones; however, the 413 error isn't captured by the server logs.  i can't see any way to determine if a getbundle request is for a full clone or for a single revision.

Comment 4

[Māris Fogels [:mars] (please needinfo)]

So the 28k number is surprisingly high for two months of reviews.  There are probably some automated systems generating traffic on the review repo, like static analysis and autoland.