Re-export telemetry data for CHARSET_OVERRIDE_USED by locale in Firefox 26

VERIFIED FIXED in Unreviewed

Status

Mozilla Metrics
Data/Backend Reports
VERIFIED FIXED
4 years ago
3 years ago

People

(Reporter: hsivonen, Assigned: mreid)

Tracking

unspecified
Unreviewed

Details

Attachments

(11 attachments)

(Reporter)

Description

4 years ago
mreid, in bug 906032, you exported telemetry data for CHARSET_OVERRIDE_USED  with the data samples from Firefox 22 and 23. Since then, I have made changes that should affect this  telemetry variable on a per-locale basis. In order to assess success, I'd like to  see this data for Firefox 26 release channel now that Firefox 26 is the current release, after 6 weeks for 27 release channel and later for 28, 29 and 30 as those make it to the release channel. (I hope to be done with this by Firefox 31.)

Could you, please, rerun the script you used to export the data in bug 906032 but this time looking at Firefox 26 on the release channel?

(If it's possible to go back in time 6 weeks to look at  Firefox 25 release channel data from the days when it was the current release, that would be awesome for comparison but less important than looking at Firefox 26  release channel data.)
(Reporter)

Updated

4 years ago
Group: metrics-private
(Assignee)

Comment 1

3 years ago
Created attachment 8371459 [details]
CHARSET_OVERRIDE_USED data for firefox 26 for 20140129

I've updated the export code from bug 906032 to work with the new-since-then telemetry backend and run it for one day (20140129) so far.  What sort of date range would you like for Firefox 26?

I can definitely run it for Firefox 25 as well, just need to know what date range you'd like to look at there too.
Flags: needinfo?(hsivonen)
(Reporter)

Comment 2

3 years ago
I guess it's better to have the time range on the wider side in order to let fluctuations flatten out.

I suggest:
Firefox/Fennec 26.x: 2014-01-13...2014-02-02

Firefox/Fennec 25.x: 2013-11-18...2013-12-08

I have no concept of how expensive/slow it is to export 3-week windows or if it makes any difference to have a window longer than one week. If three weeks is unreasonable, please shorten the time windows either both windows by one week or both windows by two weeks from the beginning so that the end dates don't change.

Thank you!
Flags: needinfo?(hsivonen)
(Assignee)

Comment 3

3 years ago
Created attachment 8373508 [details]
CHARSET_OVERRIDE analysis code

One nice thing about the AWS-based infrastructure is that it's relatively easy to determine how expensive a given job will be.

In this case, it costs between $0.60 and $1.20 per day's data.  The analysis job takes right around 1 hour, so if it's less, it'll cost the lower amount, if it's more, it'll cost the higher amount.

In any case, it's not expensive to do the full 3 weeks' worth of analysis, so I've started the required processing.

Meanwhile, I've attached the code for future reference.
Assignee: nobody → mreid
Status: NEW → ASSIGNED
(Assignee)

Comment 4

3 years ago
Created attachment 8373510 [details]
CHARSET_OVERRIDE post-processing code
(Assignee)

Comment 5

3 years ago
The v25.x analysis is complete, and results are available here:
https://s3-us-west-2.amazonaws.com/telemetry-public-analysis/bug965707/data/bug965707.YYYYMMDD.25.clean.txt

for YYYYMMDD values from 20131118 to 20131208
(Assignee)

Comment 6

3 years ago
The v26.x analysis is complete, and results are available here:
https://s3-us-west-2.amazonaws.com/telemetry-public-analysis/bug965707/data/bug965707.YYYYMMDD.26.clean.txt

for YYYYMMDD values from 20140113 to 20140202
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Reporter)

Comment 7

3 years ago
Created attachment 8381391 [details]
Python script for summarizing the data
(Reporter)

Comment 8

3 years ago
Created attachment 8381393 [details]
Summary data for Firefox/Fennec 25 as HTML
(Reporter)

Comment 9

3 years ago
Created attachment 8381394 [details]
Summary data for Firefox/Fennec 26 as HTML
(Reporter)

Comment 10

3 years ago
Thank you. This data is very useful.
(Assignee)

Comment 11

3 years ago
You're welcome - let me know if you'd like to run this analysis again (or on a scheduled basis)
(Reporter)

Comment 12

3 years ago
(In reply to Mark Reid [:mreid] from comment #11)
> You're welcome - let me know if you'd like to run this analysis again (or on
> a scheduled basis)

Yes, please. It would be good to run the following:

Firefox&Fennec 27 on the release channel from 2014-02-27 to 2014-03-16, inclusive.
Firefox&Fennec 28 on the release channel from 2014-04-07 to 2014-04-27, inclusive.
Firefox&Fennec 29 on the release channel from 2014-05-19 to 2014-06-08, inclusive.
Firefox&Fennec 30 on the release channel from 2014-06-30 to 2014-07-20, inclusive.
(Assignee)

Comment 13

3 years ago
Re-opening so these additional analyses don't get forgotten.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Reporter)

Comment 14

3 years ago
Now that we are past the end date of the Firefox 28 analysis period mentioned in comment 12, it would be good to run the analyses for 27 and 28.
(Assignee)

Comment 15

3 years ago
The analyses for v. 27 and 28 is complete.
(Reporter)

Comment 16

3 years ago
(In reply to Mark Reid [:mreid] from comment #15)
> The analyses for v. 27 and 28 is complete.

Thank you!
(Reporter)

Comment 17

3 years ago
Created attachment 8420029 [details]
Python script for summarizing the data for Firefox 28 or newer
(Reporter)

Updated

3 years ago
Attachment #8420029 - Attachment mime type: text/plain → text/x-python
(Reporter)

Comment 18

3 years ago
Created attachment 8420031 [details]
Summary data for Firefox/Fennec 27 as HTML
(Reporter)

Comment 19

3 years ago
Created attachment 8420032 [details]
Summary data for Firefox/Fennec 28 as HTML
(Reporter)

Comment 20

3 years ago
As expected, there's an improvement for Traditional Chinese and Thai in 28 compared to 27. Nice.
(Reporter)

Comment 21

3 years ago
Could you please run the analysis for Firefox 29 and 30 per comment 12?
Flags: needinfo?(mreid)
(Assignee)

Comment 22

3 years ago
Will do
Flags: needinfo?(mreid)
(Assignee)

Comment 23

3 years ago
Version 29 per-data is available now at URLs like:
https://s3-us-west-2.amazonaws.com/telemetry-public-analysis/bug965707/data/bug965707.20140519.29.clean.txt

Version 30 data is on the way.
(Assignee)

Comment 24

3 years ago
Version 30 data is also available on S3 (same URL pattern)
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED
(Reporter)

Comment 25

3 years ago
Thank you!

Interpretation:
zh-CN improved, as expected, from 28 to 29. zh-TW improved, as expected, from 29 to 30. However, the improvements were within the noise when comparing over a longer period of time, so it's hard to tell if there's any improvement of *significance*. That there wasn't a notable improvement for zh-TW from 29 to 30 is particularly disappointing. The lack of improvement for ko is also disappointing.

It appears that to improve things here, it's necessary to address the behavior in the case of local files and/or the .com/.org/.net TLDs. I'll file another telemetry request to check that hypothesis.
(Reporter)

Comment 26

3 years ago
Created attachment 8476515 [details]
Summary data for Firefox/Fennec 29 as HTML
(Reporter)

Comment 27

3 years ago
Created attachment 8476516 [details]
Summary data for Firefox/Fennec 30 as HTML
(Reporter)

Comment 28

3 years ago
(In reply to Henri Sivonen (:hsivonen) from comment #25)
> I'll file another telemetry request to check that hypothesis.

Bug 1056542.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.