799603 - Stop fetching all bug comments when starring a bug, since duplicate detection is broken anyway

Status: RESOLVED FIXED

(Tree Management Graveyard :: TBPL, defect)

Description

[Ed Morley [:emorley]]

TBPL's server-side multiple comment detection feature has been broken for over a year, but yet we are still fetching all bug comments for each bug starred, to compare against the broken regex found in bug 748848. 

In addition, given that many orange bugs have an excessive number of comments, we would be better off doing this in TBPL directly (eg if submitBugzillaComment.php finds someone else has submitted a comment since the annotated summary was opened, then don't post the comment, instead update the TBPL UI with the new comment and a confirmation etc).

For now, we should just stop doing:
    31 // check to make sure that the bug has not already been commented on
    32 $bug_comments = file_get_contents("https://api-dev.bugzilla.mozilla.org/latest/bug/$bugid/comment");
    33 if ($bug_comments !== false) {
    34   if (preg_match("/([\d]+\.[\d]+\.[\d]+)\.gz$/", $_POST['comment'], $matches)) {
    35     $logid = $matches[1];
    36     if (strpos($bug_comments, $logid) !== false)
    37       exit;
    38   }
    39 }

https://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/file/9c7f496beb97/php/submitBugzillaComment.php#l32

Comment 1

[Ed Morley [:emorley]]

Attachment: Patch v1

Stops fetching all bug comments for a bug when starring it, given that:
* The duplication detection doesn't work anyway (bug 748848)
* Even if we were to fix the regex issue in bug 748848, it's not really the best approach, seeing how many comments some of the orange bugs have.

Comment 2

[Ed Morley [:emorley]]

As a rough estimate to how many bzapi calls this will save - OrangeFactor says there were 2200 failures in the last 7 days on what it calls trunk (but is effectively mozilla-central + inbound + fx-team, so omits all other project branches, Try & aurora/beta/esr). 

Even just considering those (admittedly likely more active/starred) trees, we'll save ~300 requests of https://api-dev.bugzilla.mozilla.org/latest/bug/$bugid/comment a day. In addition, [orange] bugs often have many comments - and the comments can contain large log excerpts (bug 795460 should improve this). 

As an example, the top 5 TBPLBot-starrable oranges for the last 7 days are:

https://api-dev.bugzilla.mozilla.org/latest/bug/797263/comment
-> Starred 179 times; 204 comments; json each time = 285 KB

https://api-dev.bugzilla.mozilla.org/latest/bug/766826/comment
-> Starred 133 times; 168 comments; json each time = 5.7 MB (!!)

https://api-dev.bugzilla.mozilla.org/latest/bug/782991/comment
-> Starred 94 times; 252 comments; json each time = 484 KB

https://api-dev.bugzilla.mozilla.org/latest/bug/747602/comment
-> Starred 88 times; 310 comments; json each time = 333 KB

https://api-dev.bugzilla.mozilla.org/latest/bug/762893/comment
-> Starred 84 times; 157 comments; json each time = 767 KB


--> The starring of just these five bugs alone (on only the OF 'trunk' subset of trees) was responsible for the fetching of ~900MB of (effectively unused) bug comments a week.

Comment 3

[Ed Morley [:emorley]]

https://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/rev/ce385caa1ea1