2 years ago
2 years ago


(Reporter: shobson, Unassigned)






2 years ago
It's hard to asses the value of the TOC (aka "In this article") links based on analytics data.


If we decrease the number of links in the TOC, then more people will click on them because it is faster for them to assess if there is useful information.


Create variants of these two English pages which have the TOC restricted to "h2 and higher":

We already have event tracking in place for the TOC.

Running it for a week should be enough data.
Keywords: in-triage

Comment 1

2 years ago
I changed the TOC depth on I'll compare data and revert it on July 12th.

I feel it is important to do this test now before the redesign is applied so we have enough data points to do a proper comparison and so we can decide how important it is to include the TOC in the new design.

Comment 2

2 years ago
I did find something interesting in the analytics. When we launched the beta TOC usage dropped off significantly. The only explanation I can think of for this is that half of our regular TOC users went and signed up for the beta. I couldn't filter them out of the analysis because they weren't beta users before that date. This makes sense because these users would be the ones most engaged with the site. 

I had the smaller TOC on the array page for 7 days but I only compared the middle 5 days to be sure the TOC had been there for the entire day. I compared to the 5 previous matching days of the week (after the beta was launched) and in both cases I filtered out beta testers.

The difference was not statistically significant. Only about 3% of our users click the TOC links so I guess we need to make a bigger change than length have an effect.
Last Resolved: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.