722338 - Evaluate opportunities for volunteer engagement beyond user's initial bug report(s)

Status: RESOLVED INACTIVE

(Marketing :: General, task)

Description

[Mike Hoye]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:12.0a1) Gecko/20120123 Firefox/12.0a1
Build ID: 20120123031112

Steps to reproduce:

Signed on as a new user.


Actual results:

Got the same experience signing on to file a bug that veteran users do.


Expected results:

Figuring out how to file a bug against a Mozilla product is not a trivial exercise; people who have made it over those hurdles and filed that bug (even if it's a duplicate, even if it's filed in anger, even if it's invalid or wontfix or illegible) should be actively engaged to contribute further.

They can be pointed to the beta or aurora channels for the product they filed the bug against, they can be surveyed to see what level of expertise they have, what sort of contribution they're able to make. This person has shown that they are willing to expend effort to make Mozilla better, so Bugzilla should either directly, or via a human at Mozilla, jump on the chance to figure out what this person's superpower is.

There's a variety of ways this can happen -
- People who say they'd like to try their hand at this "coding" thing they've heard should be pointed automatically towards good-first-bugs, polyglots from exotic foreign locales can be directed towards internationalization - and lots of ways to manage that relationship, too. If a first-time user's bug gets flagged as a duplicate rather than just flipping that bit and moving on Bugzilla can forward that change (mark it "pending verification"?) to somebody who can contact that user, and ask them to verify that it is the same or similar to the earlier bug before doing the duplication, for example. But that process should be actively pursued, and it should start as soon as somebody's filed their first bug.

Don't pass up that opportunity, basically. The first time somebody has successfully used Bugzilla, that person has climbed a big hill. Mozilla should put a journalist, a tour guide and a cheerleader at the top of that hill so that when they get there Mozilla can figure out who they are, where they can go next, and some encouragement to get there.

Comment 1

[:glob ✱]

this looks like a bmo-specific request, moving.

Comment 2

[:glob ✱]

(In reply to [e-mail] from comment #0)

> Actual results:
> Got the same experience signing on to file a bug that veteran users do.

this isn't true; new users get:
  https://bugzilla.mozilla.org/enter_bug.cgi?format=guided

while "veteran users" (those with CANCONFIRM rights) get:
  https://bugzilla.mozilla.org/enter_bug.cgi?format=__default__

> They can be pointed to the beta or aurora channels for the product they
> filed the bug against, they can be surveyed to see what level of expertise
> they have, what sort of contribution they're able to make

i've filed bug 722363 about adding a link to bugsahoy (http://www.joshmatthews.net/bugsahoy/) to the createaccount page, which is an excellent "hey, i want to help, but how?" page.

> There's a variety of ways this can happen -
> - People who say they'd like to try their hand at this "coding" thing
> they've heard should be pointed automatically towards good-first-bugs,
> polyglots from exotic foreign locales can be directed towards
> internationalization - and lots of ways to manage that relationship, too. If
> a first-time user's bug gets flagged as a duplicate rather than just
> flipping that bit and moving on Bugzilla can forward that change (mark it
> "pending verification"?) to somebody who can contact that user, and ask them
> to verify that it is the same or similar to the earlier bug before doing the
> duplication, for example. But that process should be actively pursued, and
> it should start as soon as somebody's filed their first bug.
> 
> Don't pass up that opportunity, basically. The first time somebody has
> successfully used Bugzilla, that person has climbed a big hill. Mozilla
> should put a journalist, a tour guide and a cheerleader at the top of that
> hill so that when they get there Mozilla can figure out who they are, where
> they can go next, and some encouragement to get there.

the contributor engagement team are probably better equipped to address these points.

Comment 3

[Mike Hoye]

<i>the contributor engagement team are probably better equipped to address these points</i>

Are those teams notified when a new first-time contributor appears? (A CHALLENGER APPEARS!) 

Or is that worth filing another bug?

Comment 4

[:glob ✱]

(In reply to [e-mail] from comment #3)
> Are those teams notified when a new first-time contributor appears? (A
> CHALLENGER APPEARS!) 

excellent timing :) bug 719526 (also see bug 721206)

Comment 5

[Mike Hoye]

Nice! But that's just for first-patch, not for first-bug.

Comment 6

[Mike Hoye]

"Just", heh. What has two thumbs and is still part of the problem? This guy.

Comment 7

[David Boswell]

(In reply to Mike Hoye from comment #6)
> "Just", heh. What has two thumbs and is still part of the problem? This guy.

:)

Mike, I think your suggestion is definitely worth thinking through.  Now is also a great time to audit the whole Bugzilla experience for new contributors.  As Byron points out we recently filed a similar bug.  

Maybe we can approach this in a more coordinated way and come up with a plan that looks at the full Bugzilla experience and addresses all of the key milestones for new users?  We'd love your help in doing that.

A good place to talk more about it would be this Wednesday's Coding Contribute Group meeting.  Meeting information is below.  Please feel free to come to that if you are available.

https://wiki.mozilla.org/Stewards/Coding/Group_02_01_12

CC'ing the Coding Stewards as well for their feedback.

Comment 8

[Josh Matthews [:jdm]]

It would be interesting to think about how many other contextual emails we could send users in addition to the proposed "congratulations, your patch looks good!" email - the first duplicate bug one sounds reasonable to me, as well as first submitted bug. I think it would be important to provide suggestions for avenues of action: testing whether a bug is reproducible in more recent versions of Firefox, reading the duplicated bug to see if developers are asking for feedback from people experiencing the problem, etc.

Comment 9

[David Boswell]

Mike, I'm assigning this to you since it would be great to have you drive this.  Since you're new to Bugzilla I think you'll have a fresh perspective about where all the opportunities for improving volunteer engagement may be.

So that might look something like:

* Milestone: Sign-up for Bugzilla account (send welcome to Bugzilla email)
* Milestone: Have third patch approved (send email about adding name to Credits page)
etc.

Comment 10

[Mike Hoye]

Well played, Mr. Boswell. Well played indeed.

I accept.

Comment 11

[David Boswell]

(In reply to Mike Hoye from comment #10)
> Well played, Mr. Boswell. Well played indeed.
> 
> I accept.

Cool.  As they say -- be careful what you express interest in around here because you might end up owning it :)

Comment 12

[Mike Hoye]

OK, so:

I'm putting together a presentation around this if people are interested in seeing it, but the major bullet points that I want to land on are the following: 

I'd like to radically reimagine the first-contact experience with bugzilla. As it stands right now, there are two major things missing from the Bugzilla join-up process. There is:

- nothing to indicate that a first bug is different than any other bug, and 
- nothing in place to find out anything at all about the person who files the bug.
   - who are they, where they're from
   - what else they're good at 
   - how else they can contribute

What I've been telling people recently is, just _figuring out how to file a bug in Bugzilla_ is a huge hill to climb. When a person gets to the top of that hill there should be three people there waiting for him.

Comment 13

[Mike Hoye]

AND THOSE PEOPLE ARE, damn you tab not working in a textbox....

- Somebody from volunteer engagement with thanks, congratulations, and a survey. Mozilla needs to know who these people are, where they are so they can be connected with regional Moz orgs, and maybe most importantly: what their superpower is, so we can point them towards other things that they can help with.

- A developer who knows what's going on in Bugzilla really well, who can prioritize first-time-contributor triage.

The first touch to bugzilla should, in my opinion, look nothing like the typical bug-filing experience, and should be as simplified as possible. Is it in firefox or thunderbird? Is it a site that doesn't work right, or a menu that doesn't work right, or other?

As bare and simply-directed as possible, sparsely laid out, very Web-2.0-ish rounded edges, super friendly. Collects as much information about the user's computer as we can get at via the browser (maybe ask politely if you can run a hardware-information-collection dingus, maybe not...) but have all of that be as hidden and painless as it can be made.

The user should two things back directly:

- an email that is _not at all_ the standard Bugzilla reply but that's written in plain, human, variable-width text saying thank you, we're grateful for your contribution, and

- a request to take a survey telling us about themselves and what their super-power is.


Here's where the plan gets really interesting: 


- No first-time user's bug is filed "invalid" or "duplicate"; First time users are, even if it's just through a form letter, talked through the process of verifying their own bugs or pointed to other bugs, and asked "can you check to see if the bug you've filed is a lot like this other one? If so, can you mark yours as a duplicate? Explain what this means, how it still means the bug gets fixed, but we can verify that other bug now, and so forth. 

- People who express interest in contributing can (with hardware info? With their expression of interest?) Be asked to verify other recent first-timer bugs as well, as a way of gauging their interest in the project and further contributions in a low-cost way.

The idea is that you give the new contributor as much ownership over their own bug as you can, and as much feedback as you can about fixing it. When a patch lands that fixes it, the first-time contributor is asked to verify whether Mozilla needs that verification or not. When that patch makes it into Aurora, the bug filer gets told it's available, and when it makes it into a release, that bug filer gets a thank you note saying they've made Firefox better.


All of this can be built with existing APIs, but it needs to be a real project, I think, with a calendar and one person in charge of it. But I think it will make a huge difference.

To sum up: 

- The on-ramp to Bugzilla for first time contributors is dramatically simplified, streamlined and humanized.
- Bug filers have as much ownership over the lives of their bugs as Mozilla can reasonably give them.
- Mozilla treats those people as soft targets for engagement, tries to learn as much as they can about new bug filers and treats them as potential contributors to be fostered. One thing (among many many) that this implies is that bug triage is a critical function of the community engagement team.

Thanks you. I hope that these suggestions are worthwhile. I'd love to talk about them some more, but even more than that, I'd like to help build them.

Comment 14

[David Lawrence [:dkl]]

Glob, BMOs resident admin has done wonders with the guided bug entry form as far as guided new bug submitters though the submission process. Higher level users normally do not see this easier form so I am not sure they all know about it. You can take a look at it at

https://bugzilla.mozilla.org/enter_bug.cgi?format=guided

Please let us know about any improvements we can make with the guided form in line with your earlier ideas.

dkl

Comment 15

[Mike Hoye]

Oh, and: the third person at the top of the hill is a sherpa. These are the next steps, the next things you can do.

Comment 16

[Mike Hoye]

dkl: 

I think that the guided format is a strong start, but I think that it should be more aggressive in what it takes away. Almost everything in the menu bar at the top, for example, is meaningless to somebody who isn't already a Bugzilla user, and the stuff in that first box (I need support, etc) is a way of shuttling this user's information into slots that already exist.

I don't think that's inherently a bad thing, but I do think that it strongly prioritizes "directing this user's input towards the a specific database" over "making this user happy and engaging them as a contributor."

In my opinion, that menu should read "Firefox", "Thunderbird", "Other", that's all. More importantly, the "search for duplicates" options should be skipped entirely;  again, this prioritizes the state of the database over the engagement of the user, which is exactly what we want to avoid at this point. The new contributor should own, as much as possible, their issue. 

Later on, sure, regular users need that filtering. But if they're regular users they're not the focus of this exercise.

The user gets a single, large text field in which to describe their issue, and buttons that say "start over" and "submit". That's all, clean and simple. They click submit, they get a bug number and a thank you email. 

The important stuff next happens behind the scenes, as far as they're concerned, and is as I've outlined above - the bug is triaged by a human and given a summary title by a human. The contributor gets a reply saying "here's your bug" and an email from contributor-engagement people saying "tell us more about you" and it rolls from there.

Comment 17

[:glob ✱]

mike,

the "tell-us-more" project <https://wiki.mozilla.org/Firefox/TellUsMore> has goals to address most of your points in comment 16.

in a nutshell it's a greatly simplified form in terms of data entry requirements and process (eg. authentication happens after collecting the data), along with changes to assist triage.

Comment 18

[David Boswell]

(In reply to Mike Hoye from comment #13)

Mike, thanks for following up with these suggestions.  There are lots of good ideas in here.

There are a few different threads in here, so it might help to separate them out.

> - The on-ramp to Bugzilla for first time contributors is dramatically
> simplified, streamlined and humanized.

As Byron mentioned, it seems like the Tell Us More project will address some of the changes you'd like to see.  It may make sense to file any streamlining-things-for-first-time-contributor bugs that aren't addressed by Tell Us More as feature enhancement ideas for that in the input.mozilla.com component.

> - Bug filers have as much ownership over the lives of their bugs as Mozilla
> can reasonably give them.

I like the idea of Bugzilla providing more insight and guidance for people as they progress through the process of contributing.  This seems to build on the idea in bug 721206 by identifying other critical points where we could have more communication with contributors.  Would it make sense to document these critical touch points on the wiki or as an attachment so it's easier to reference for people who haven't read this whole bug thread?

> - Mozilla treats those people as soft targets for engagement, tries to learn
> as much as they can about new bug filers and treats them as potential
> contributors to be fostered. One thing (among many many) that this implies
> is that bug triage is a critical function of the community engagement team.

I definitely agree about this.  This seems to be something to bake into the community building strategy we're creating for coding at

https://wiki.mozilla.org/Contribute/Coding#Dashboards

For instance, having someone reach out to people who have stopped contributing patches to encourage them to contribute again or to learn information that will help other people stay involved, seems like a good thing to be doing.

Creating questions for a survey, conducting the survey and/or finding other audiences of contributors to reach out to with these questions are all things we could start doing today.

The best place for discussion about this topic is probably the Coding Contribute Group meetings every Wednesday at 1 pacific.  The next one is tomorrow if you can make it.

https://wiki.mozilla.org/Stewards/Coding/Group_03_21_12

Comment 19

[Mike Hoye]

I apologize for missing these CCG meetings; I really do need to get to them.

(In reply to Byron Jones ‹:glob› from comment #17)
> mike,
> 
> the "tell-us-more" project <https://wiki.mozilla.org/Firefox/TellUsMore> has
> goals to address most of your points in comment 16.

Respectfully, I don't think they do, either in substance or tone. 

I think that there's interesting work to be done there, for sure, particularly around feedback and keywords, but the most noteworthy that tellusmore page is that it lists arbitrary and very technical limitations on what feedback users will be allowed to provide, rather than (1) focusing on making it easy for contributors to provide feedback, and (2) making it very easy for Mozilla to have a way to quickly target users with particular issues.

"Bug Summary: 75 characters max length, Required field".

Why 75? If a user wants to put 2k of info in there, why not let them? If they want to _do more_, why put a flag in their face saying stop? The idea of having a short, sharp feedback loop in the "make me sad" part there is totally worthwhile, no question. But implementing it from a DB-centric approach isn't the right thing.

This is probably worth its own bug, but here's an idea: Take the "makes me happy" and "makes me sad" options and instead of having them be basically a tweet, let users put as much in there as they want. 4k, 8k, anything, but have the backend start flagging keywords and evaluating that feedback immediately, as it's submitted.

Next, go to different parts of the Mozilla organization, internally, and say "We've got this new way of looking at user feedback and asking them to follow it up right away. We'd like you to give us a list of keywords, so that when a user says "video" or "crash" or "security" or "usability" or "blind" or "colours" or whatever, we can immediately ask them to tell us more about what the problem is, and then have either people from the right internal group, or community engagement, follow that up with them so that we learn more. 

So the next time somebody says "something something video something accessibility something crash", then the system knows that David Bolter (picked out of the blue here for my own example's sake) has asked that when those words appear together in a "makes me sad" report that the user immediately be asked for more information, and dbolter@mozilla gets cc'ed the answer, etc...

Maybe David only wants that to happen 1% of the time, maybe in the future he'll want something else or for that to be turned off, maybe he'll never use it. The important thing here is that pretty quickly we're not talking about technology and tooling limitations, but we're using flexible tools to help us foster relationships between useful people who want the product to be better.

And then, if we see a particularly detailed or useful piece of feedback from some user, by chance, then those users become soft targets for community engagement as well, and the circle of life continues. Better still, a very easy contributor's onramp for those people is actually _Triaging other user feedback_ in some simple way - letting somebody who's demonstrated that they provide valuable feedback evaluate other incoming feedback from other people for usefulness, so that they can be turned into bugs in an accessible community oriented way as well. 

So, yeah. That's what I have to say here. These tools don't need to be perfect; they just need to be accessible, flexible and, if not directly social, then something that facilitates sociability, and social feedback, in both directions; from the community to Mozilla and vice versa.

And starting your specification with DB constraint isn't the right place to start that.

Thanks.

Comment 20

[Mike Hoye]

So, with a little bit of luck I'll be making that presentation I mentioned early on at Mozilla Toronto this coming Thursday Afternoon. The time's not carved in stone, but I'd like it to be something that all the Pac-Rim offices can see. I'm thinking 1:00 PM EST. Broadly, the topic will about how I think Mozilla can turn their existing tools and processes into effective on-ramps for community members, and foster community contributors thereby. 

But I'll probably have a catchier title than that by Thursday, so.

Comment 21

[Mike Hoye [:mhoye]]

New Mike. Like New Coke. Think of it that way.

Comment 22

[Pierros Papadeas [:pierros]]

Moved to Marketing General component, following a clean up of Mozilla Communities product