1366429 - Web Extension for Debiasing Code Reviews in Splinter Experiment

Status: NEW

(Bugzilla :: Bugzilla-General, enhancement)

Description

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

Design, build, test, and deploy a web extension for bugzilla.mozilla.com, which when used, will redact the author of a change log/patch.

It is understood that this will not work in all situations (pull requests linked from GitHub, diffs pasted from email.)

The extension should also append an additional field (which we will check if can be done) indicating that the extension was active when the review was done.

Comment 1

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

:dylan, can we send an additional value back up with a review that corresponds to a custom field or flag?

Comment 2

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

To redact attachment creators in splinter, the nodes associated with #attachmentCreator would need to be changed.

To redact comment-creators, the nodes associated with .attach-author would have to be changed. 

To redact comments, look for nodes with the class .change-author. You will need to search within the comment itself, which is in a PRE tag, and use that to scrub comment authors.

Comment 3

[Don Marti]

A reviewer may want to go through the code "blind" and then _after_ commenting, re-show the personal info in order to add a polite welcome for a new contributor.

We should be able to log if the review has had some comments done before turning personal info back on -- so not just blind/name-attached but also part blind.

Comment 4

[Dylan Hardison [:dylan] (he/him)]

How do you intend on hiding the information on the show bug page?

Comment 5

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

The web extension will use a content script to hook into class names used to display attachment authors and pattern matching in comment text on show bug, and on the attachment detail page.

See comment #2 above.

Comment 6

[Gregory Szorc [:gps]]

I fully support this effort to tease out some biases. However, there are risks that *complete* blinding could be counter-productive. As a reviewer, I often take prior interactions and experience into account when reviewing. For example, if I know they've done hundreds of patches, I can usually be more concise and blunt. Conversely, I need to be more gentle and hand holding to newcomers. On a personal level, if I know the author is from a Commonwealth country, I may use "proper" English spellings so as to not cause offence with my American/Low English. Or I may work in a meme or cultural reference I know they can relate to. This personalizing of review feedback helps soften what can be a conflict-oriented interaction, leading to a more positive outcome.

So my request would be to conditionally blind authors. Perhaps we could only blind people if they have less than N attachments or something. That would achieve the stated goal of debiasing for new contributors while allowing learned context to improve future interactions.

Comment 7

[Don Marti]

Right now Bugzilla-General is a better product for this bug than Community Ops, just moved it.

Comment 8

[Don Marti]

Related: the add-on now also hides user info on GitHub pull requests: https://addons.mozilla.org/en-US/firefox/addon/blind-reviews/

Comment 9

[marynahasela]

Hello! It seems like a promising initiative to tackle biases in the code review process. I appreciate the effort put into designing, building, testing, and deploying this extension specifically for bugzilla.mozilla.com.

While I understand that this extension may not be applicable in all scenarios, such as pull requests from GitHub or diffs shared via email, the idea of redacting the author of a change log/patch is intriguing. Additionally, the proposal to add a field indicating the extension's activation during the review is worth considering, like this: https://andersenlab.com/services/web-development

However, I believe we should be cautious about completely blinding authors. As a reviewer, I often take previous interactions and experience into account when providing feedback. It allows me to tailor my approach based on the author's background and familiarity with the codebase. Personalizing review feedback can lead to a more positive outcome, especially for newcomers or individuals from different linguistic backgrounds.

Instead, I suggest exploring conditional blinding based on certain criteria, such as the number of attachments.