1117183 - Create survey for collecting remaining blockers for developers transitioning to Treeherder

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P2)

Description

[Ed Morley [:emorley]]

Current draft:
https://docs.google.com/forms/d/1hviR9_oO6LZFRbQv5lADqzjYN4-MwW_c5TfP1zX43RQ/viewform?usp=send_form

Comment 1

[Ed Morley [:emorley]]

The survey is now available at:
http://goo.gl/forms/aBxjFlRNxO

An email has been sent to dev.tree-management, dev.platform, dev.b2g, dev.gaia, firefox-dev, auto-tools about it:
https://groups.google.com/forum/#!topic/mozilla.dev.tree-management/CtkSFWbtAd0

Responses at:
https://docs.google.com/a/mozilla.com/spreadsheets/d/1Pnps4JlyMoL0Af4S3KvUSCPHAfveL3_qG1x7E-YmvT0/edit

Once new responses dry up, I'll sort through the results, file bugs where needed, CC people on issues they've mentioned & update the deps of bug 1059400.

Comment 2

[Ed Morley [:emorley]]

I've created a new sheet in the google doc linked in comment 1, with the processed responses. I've split the responses to one per row (for people who left move than one comment), and annotated with comments/bug numbers. I've also emailed ~12 respondents with follow-up questions - I'll update the spreadsheet once their results are in. Once the analysis is complete, I'll file further bugs for anything not already filed, send a follow-up email to the newsgroups, update this bug & CC the respondents to bugs that exist for the problems they reported.

Comment 3

[Ed Morley [:emorley]]

Number of respondents: 36
Total items of feedback from them: 77

"How often do you use TBPL instead of Treeherder":
...always: 11%
...majority of the time: 42%
...some of the time: 47%
(bearing in mind only users who still use TBPL were asked to complete the survey)

Amount of feedback by rough category:
* UI complexity/discoverability (bug 1125264)	17
* Perf (bug 1067846)	9
* Familiarity/habit (not a bug)	6
* Infra menu (bug 1082771)	5
* Ingestion speed (bug 1084493)	4
* Repo "tabs" UX (bug 1125352)	3
* Log viewer: Wanting full log with links (bug 1087317)	3
* Colours/fonts	3
* FIXED: Embedable (bug 1115904)	2
* Shorter/cleaner URLs (bug 1042737)	2
* Commit initials (bug 1136160)	2
* Buildbot bookmarklet (bug 1053228)	2
* FIXED: Tree status broken for comm-central	1
* FIXED: Tinderboxprints	1
* FIXED: Push refresh	1
* FIXED: Infra/reliability	1
* FIXED: Hard to copy/paste bug numbers	1
* FIXED: Missing e10s reftest analyser links	1
* Tree status bug linkification	1
* Better mobile UI/perf (bug 1066286)	1
* Log viewer: Not overly intuitive	1
* Log viewer: fixed width font (bug 1081587)	1
* Links break when dom.storage.enabled is false (bug 1136232)	1
* Linkification bug lookup (bug 1082693)	1
* Less obvious tree status (bug 1042613)	1
* Hard to cppy-paste revison SHAs (bug 1136918)	1
* Build directory links for pending jobs (bug 1057314)	1
* Unknown/TBC: 2

For more details see:
https://docs.google.com/a/mozilla.com/spreadsheets/d/1Pnps4JlyMoL0Af4S3KvUSCPHAfveL3_qG1x7E-YmvT0/edit

Comment 4

[Ed Morley [:emorley]]

dev.tree-management emailed with a follow-up (and survey respondents BCCed):
https://groups.google.com/forum/#!topic/mozilla.dev.tree-management/FewfAtJioL8