Allow developers to watch directories for changed/get notified, e.g. by RSS feed, email (hgchanges replacement)

NEW
Unassigned

Status

defect
P3
normal
3 years ago
Last year

People

(Reporter: aryx, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

>>Problem:
mozilla-central is a big repository with many changes getting commit. Developers and other stakeholders (see below) are often interested in changes in only a few selected components, e.g. to get aware of new functions, features, check the code. Watching Bugzilla components is insufficient because changes can affect multiple components or e.g. require a refactoring of some platform functionality and thus be from a bug which is in a component not watched by the developer. https://hg.mozilla.org only provides feed per repository.

Other groups interested in code changes:
- sheriffs: a web frontend for allows much easier to find related changes to e.g. a failing test or other reported abnormal behavior by listing only the changes in the folder (these would also need this for autoland and mozilla-inbound)
- researchers: easier for them to see what's new, e.g. for security testing, performance measurement
- documenters: changes in code which require updates on https://developer.mozilla.org
- localizers on Nightly: as a push notification for them and feed with changes (e.g. Italian localizes per bug, provides the context without clutter). l10n has a page which they can check manually
- customizers: e.g. Linux distros which perform customization according to their guidelines, partners
- addon developers who want to be aware of upcoming changes or keep their addon working with development builds

>>Solution:
Prolog:
Dave Townsend ("Mossop") wrote the tool "hgchanges" for this purpose: https://www.oxymoronical.com/blog/2013/04/Get-notifications-about-changes-to-any-directory-in-mercurial (Thank you, Dave) He is now busy with other things  and has no time to maintain it anymore. Huge merges like servo into mozilla-central were able to break it. The changes were preserved for approximately a week. The code is at https://github.com/Mossop/hgchangefeed

1. The tool would get notified about changes to the repositories (e.g. by pulsebot)
2. It would update the local repository (only talking about one here to make it easier).
3. Identify the new commits.
4. For each commit, starting with the oldest one: Get the changes and the affected files and write the commit information into a database, for each folder from root of the repository to the file (including the path) - people shouldn't have to use two pages (the other one hg.mozilla.org) to subscribe to these changes. Deletions of files and folders should be logged as such and preserved, moves are difficult because they can be ambiguous (for this reason, the file path should be concatenated with a separator and an increasing number to track the incarnation (e.g. a.txt as a.txt#0, a move to c/b.txt could still be served for a request for a.txt#0, even after a second a.txt got created after the move)
5. Changes will be accessible
- as a browsable web front-end (folder structure with files)
- as feeds; subscription from the web front-end
- bonus if it comes with accounts (login with Firefox Accounts) to allow email notifications (issue with feeds is that the amount of items might be capped and people could miss something)

gps mentioned that such a tool is desired (no idea if it was a workweek or e.g. a post to the firefox-dev mailing list).

One of the bigger questions is how the tool we detect changes to the commit history on autoland. gps might provide more information about this.

>>Mozilla Top Level Goal:
Quality and security, because developers can check automatically get notified what got checked in, if that follows the rules of the components (owner/peer review) and find new functionality which makes their live easier and generates better code (e.g. by following testing/mochitest/BrowserTestUtils)

>>Existing Bug:
No bug

>>Per-Commit:
preferably, at least every 5 minutes

>>Data other than Pass/Fail:
No

>>Prototype Date:
Not provided

>>Production Date:
Not provided

>>Most Valuable Piece:
Not provided

>>Responsible Engineer:
Not provided

>>Manager:
Not provided

>>Other Teams/External Dependencies:
Not provided

>>Additional Info:
Not provided
:ahal, would you have some cycles to tackle this?
Flags: needinfo?(ahalberstadt)
I wonder if this is something we could change hg.mozilla.org to provide instead of standing up a separate web service. Gps, would having something like this make sense to have built-in to hg.mozilla.org?

As far as standing up a web service goes, it would take me more than a few cycles (probably more like a quarter). Tbh I don't really know what I'm doing when it comes up web services.
Flags: needinfo?(ahalberstadt) → needinfo?(gps)
Phabricator's Herald feature (https://secure.phabricator.com/book/phabricator/article/herald/) allows subscribing to all kinds of events (like commits or reviews touching paths) and taking custom actions when those occur, including sending an email. I'm not sure if it exposes RSS feeds for subscribed events. But that seems like a legitimate upstream feature request if it doesn't.

I feel like this should be WONTFIX'd and we should use Phabricator, if possible.

Regarding hg.mo, it publishes changes of everything to Pulse and SNS: https://mozilla-version-control-tools.readthedocs.io/en/latest/hgmo/notifications.html.

Because we need some kind of state and UI to manage subscriptions, I'd prefer this be separate from hg.mo. Such a service can consume Pulse or (preferably) SNS events and react to them.
Flags: needinfo?(gps)
Unassigning from inactive individual.

Needinfo mcote to triage, since his team would be the next best operator for such a service.
Assignee: jgriffin → nobody
Flags: needinfo?(mcote)
Priority: -- → P3
aryx, can you take a look at the Herald documentation (https://phabricator.services.mozilla.com/book/phabricator/article/herald/) and see how many of your needs it would accomplish?
Flags: needinfo?(mcote) → needinfo?(aryx.bugmail)
Thank you, Herald fits the notification requirement. Is it possible to generate a list of notifications/revisions which would have recently matched to see like a history for the alert hook?
Flags: needinfo?(aryx.bugmail)
You need to log in before you can comment on or make changes to this bug.