What problem would this feature solve? ====================================== Measure the impact of changing the layout of reference pages to include a short usage example on top. Who has this problem? ===================== Staff contributors to MDN How do you know that the users identified above have this problem? ================================================================== It is generally agreed that action-oriented developers would appreciate a short example at the top of pages. It is a labor-intensive project to re-write all the reference pages in this way. By measuring the impact of the change for a number of high-traffic page, it will easier to decide if to continue the project, and if additional resources should be requested. How are the users identified above solving this problem now? ============================================================ A number of JS reference pages were adjusted several months ago. It should be possible to compare traffic in the month prior and month after the change. Do you have any suggestions for solving the problem? Please explain in detail. ============================================================================== We added a content experiment framework (bug 1341770) explicitly for the purpose of segmenting traffic for this experiment. Is there anything else we should know? ====================================== The technical specification is at: https://docs.google.com/document/d/1CUe6-EqNbpSkLMLxfd0RWW6-TbPZFRtLc0rMh2Uu3G0/edit?usp=sharing
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/6c83216216476632589fbca2ac0bb9808a7a12dd bug 1356923: Content experiment data to json file Move the content experiment configuration to a .json file, loaded at startup. https://github.com/mozilla/kuma/commit/8217a84050599a02d79d6386f5b4bfe202979de2 bug 1356923: Add static-examples-on-top experiment https://github.com/mozilla/kuma/commit/68b1bad3fe6dc11b5d754dcba1de36ca2e0fc66a bug 1356923: Configure Traffic Cop for experiment https://github.com/mozilla/kuma/commit/ec0cde0d092e13941aa9675d7c439e97cf96c461 bug 1356923: Adjust CONTENT_EXPERIMENTS format Change to locale:slug keys, so that checking if a page is in an experiment is faster (25x speedup). Also change variants to a dictionary, since weights are handled on the JS side. https://github.com/mozilla/kuma/commit/26b993ef451ef03cf3be0bccec778efdb8a4aafa bug 1356923: Drop experiment-wiki-content This was a placeholder for the first "real" experiment, which is not the static examples on top experiment. https://github.com/mozilla/kuma/commit/de5d0813a52a3662e82531ba88e16dc475b0780f bug 1356923: Add Mozilla to the inner call. If we're passing in window.Mozilla to the IIFE, it should be aliased here https://github.com/mozilla/kuma/commit/9a1062bb101521a7f98d1f1f55cb24663b40e27c bug 1356923: Access variants once https://github.com/mozilla/kuma/commit/a62cdc656c1c7a10dcdb3bda711456b383f07c09 bug 1356923: Update templates test for new format https://github.com/mozilla/kuma/commit/16fb7bf9a139f90d8c7a6a40609569637f99973f Merge pull request #4183 from jwhitlock/content_exp_static_1356923 Bug 1356923: Content experiment "static-examples-on-top"
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/c2fdf2d2c437eef64e5b51ac0ce7d9faa2f00489 bug 1356923: Exclude 'Array' from the experiment The 'Array' page has had examples on top for years, and doesn't have a variant without them. https://github.com/mozilla/kuma/commit/842668822937f58b9a2324cd415bf4ba571c0d0e Merge pull request #4185 from jwhitlock/ce_static2_1356923 bug 1356923: Exclude 'Array' from the experiment
The results are in: https://docs.google.com/presentation/d/1LKiP8e2Jo0P9fJDaI0Ce5Q95OmfQ73UIIVqNc2l62ds/edit#slide=id.g1cf070bb1e_0_166 We can turn the A/B test off now. Thanks for all the work on this!
OK, I'll create a PR to turn off this experiment.
Assignee: nobody → jwhitlock
Commits pushed to master at https://github.com/mozilla/kuma https://github.com/mozilla/kuma/commit/8035f0fbd92fcf14bf3a0f67a5a982a90a73b172 bug 1356923: Retire content experiments Remove the framework-test and static-examples-on-top content experiments and related function tests. https://github.com/mozilla/kuma/commit/d2ba18c869ce44e5af46e19710b1a3c390b38371 Merge pull request #4266 from jwhitlock/stop-static-examples-on-top-1356923 bug 1356923: Retire content experiments
Pushed to production, experiment is over.
Status: NEW → RESOLVED
Last Resolved: a year ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.