Closed Bug 965707 Opened 6 years ago Closed 6 years ago

Re-export telemetry data for CHARSET_OVERRIDE_USED by locale in Firefox 26

Categories

(Mozilla Metrics :: Data/Backend Reports, defect)

defect
Not set

Tracking

(Not tracked)

VERIFIED FIXED
Unreviewed

People

(Reporter: hsivonen, Assigned: mreid)

Details

Attachments

(11 files)

mreid, in bug 906032, you exported telemetry data for CHARSET_OVERRIDE_USED  with the data samples from Firefox 22 and 23. Since then, I have made changes that should affect this  telemetry variable on a per-locale basis. In order to assess success, I'd like to  see this data for Firefox 26 release channel now that Firefox 26 is the current release, after 6 weeks for 27 release channel and later for 28, 29 and 30 as those make it to the release channel. (I hope to be done with this by Firefox 31.)

Could you, please, rerun the script you used to export the data in bug 906032 but this time looking at Firefox 26 on the release channel?

(If it's possible to go back in time 6 weeks to look at  Firefox 25 release channel data from the days when it was the current release, that would be awesome for comparison but less important than looking at Firefox 26  release channel data.)
Group: metrics-private
I've updated the export code from bug 906032 to work with the new-since-then telemetry backend and run it for one day (20140129) so far.  What sort of date range would you like for Firefox 26?

I can definitely run it for Firefox 25 as well, just need to know what date range you'd like to look at there too.
Flags: needinfo?(hsivonen)
I guess it's better to have the time range on the wider side in order to let fluctuations flatten out.

I suggest:
Firefox/Fennec 26.x: 2014-01-13...2014-02-02

Firefox/Fennec 25.x: 2013-11-18...2013-12-08

I have no concept of how expensive/slow it is to export 3-week windows or if it makes any difference to have a window longer than one week. If three weeks is unreasonable, please shorten the time windows either both windows by one week or both windows by two weeks from the beginning so that the end dates don't change.

Thank you!
Flags: needinfo?(hsivonen)
One nice thing about the AWS-based infrastructure is that it's relatively easy to determine how expensive a given job will be.

In this case, it costs between $0.60 and $1.20 per day's data.  The analysis job takes right around 1 hour, so if it's less, it'll cost the lower amount, if it's more, it'll cost the higher amount.

In any case, it's not expensive to do the full 3 weeks' worth of analysis, so I've started the required processing.

Meanwhile, I've attached the code for future reference.
Assignee: nobody → mreid
Status: NEW → ASSIGNED
The v25.x analysis is complete, and results are available here:
https://s3-us-west-2.amazonaws.com/telemetry-public-analysis/bug965707/data/bug965707.YYYYMMDD.25.clean.txt

for YYYYMMDD values from 20131118 to 20131208
The v26.x analysis is complete, and results are available here:
https://s3-us-west-2.amazonaws.com/telemetry-public-analysis/bug965707/data/bug965707.YYYYMMDD.26.clean.txt

for YYYYMMDD values from 20140113 to 20140202
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Thank you. This data is very useful.
You're welcome - let me know if you'd like to run this analysis again (or on a scheduled basis)
(In reply to Mark Reid [:mreid] from comment #11)
> You're welcome - let me know if you'd like to run this analysis again (or on
> a scheduled basis)

Yes, please. It would be good to run the following:

Firefox&Fennec 27 on the release channel from 2014-02-27 to 2014-03-16, inclusive.
Firefox&Fennec 28 on the release channel from 2014-04-07 to 2014-04-27, inclusive.
Firefox&Fennec 29 on the release channel from 2014-05-19 to 2014-06-08, inclusive.
Firefox&Fennec 30 on the release channel from 2014-06-30 to 2014-07-20, inclusive.
Re-opening so these additional analyses don't get forgotten.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Now that we are past the end date of the Firefox 28 analysis period mentioned in comment 12, it would be good to run the analyses for 27 and 28.
The analyses for v. 27 and 28 is complete.
(In reply to Mark Reid [:mreid] from comment #15)
> The analyses for v. 27 and 28 is complete.

Thank you!
Attachment #8420029 - Attachment mime type: text/plain → text/x-python
As expected, there's an improvement for Traditional Chinese and Thai in 28 compared to 27. Nice.
Could you please run the analysis for Firefox 29 and 30 per comment 12?
Flags: needinfo?(mreid)
Will do
Flags: needinfo?(mreid)
Version 29 per-data is available now at URLs like:
https://s3-us-west-2.amazonaws.com/telemetry-public-analysis/bug965707/data/bug965707.20140519.29.clean.txt

Version 30 data is on the way.
Version 30 data is also available on S3 (same URL pattern)
Status: REOPENED → RESOLVED
Closed: 6 years ago6 years ago
Resolution: --- → FIXED
Thank you!

Interpretation:
zh-CN improved, as expected, from 28 to 29. zh-TW improved, as expected, from 29 to 30. However, the improvements were within the noise when comparing over a longer period of time, so it's hard to tell if there's any improvement of *significance*. That there wasn't a notable improvement for zh-TW from 29 to 30 is particularly disappointing. The lack of improvement for ko is also disappointing.

It appears that to improve things here, it's necessary to address the behavior in the case of local files and/or the .com/.org/.net TLDs. I'll file another telemetry request to check that hypothesis.
(In reply to Henri Sivonen (:hsivonen) from comment #25)
> I'll file another telemetry request to check that hypothesis.

Bug 1056542.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.