Re-export telemetry data for CHARSET_OVERRIDE_USED by locale in Firefox 26

VERIFIED FIXED in Unreviewed

Status

defect
VERIFIED FIXED
5 years ago
5 years ago

People

(Reporter: hsivonen, Assigned: mreid)

Tracking

unspecified
Unreviewed

Details

Attachments

(11 attachments)

4.33 KB, text/plain
Details
1.05 KB, text/x-python
Details
1.07 KB, text/x-python
Details
5.51 KB, text/x-python
Details
13.28 KB, text/html
Details
13.12 KB, text/html
Details
3.88 KB, text/x-python
Details
13.41 KB, text/html
Details
13.70 KB, text/html
Details
13.79 KB, text/html
Details
13.99 KB, text/html
Details
mreid, in bug 906032, you exported telemetry data for CHARSET_OVERRIDE_USED  with the data samples from Firefox 22 and 23. Since then, I have made changes that should affect this  telemetry variable on a per-locale basis. In order to assess success, I'd like to  see this data for Firefox 26 release channel now that Firefox 26 is the current release, after 6 weeks for 27 release channel and later for 28, 29 and 30 as those make it to the release channel. (I hope to be done with this by Firefox 31.)

Could you, please, rerun the script you used to export the data in bug 906032 but this time looking at Firefox 26 on the release channel?

(If it's possible to go back in time 6 weeks to look at  Firefox 25 release channel data from the days when it was the current release, that would be awesome for comparison but less important than looking at Firefox 26  release channel data.)
Group: metrics-private
(Assignee)

Comment 1

5 years ago
I've updated the export code from bug 906032 to work with the new-since-then telemetry backend and run it for one day (20140129) so far.  What sort of date range would you like for Firefox 26?

I can definitely run it for Firefox 25 as well, just need to know what date range you'd like to look at there too.
Flags: needinfo?(hsivonen)
I guess it's better to have the time range on the wider side in order to let fluctuations flatten out.

I suggest:
Firefox/Fennec 26.x: 2014-01-13...2014-02-02

Firefox/Fennec 25.x: 2013-11-18...2013-12-08

I have no concept of how expensive/slow it is to export 3-week windows or if it makes any difference to have a window longer than one week. If three weeks is unreasonable, please shorten the time windows either both windows by one week or both windows by two weeks from the beginning so that the end dates don't change.

Thank you!
Flags: needinfo?(hsivonen)
(Assignee)

Comment 3

5 years ago
One nice thing about the AWS-based infrastructure is that it's relatively easy to determine how expensive a given job will be.

In this case, it costs between $0.60 and $1.20 per day's data.  The analysis job takes right around 1 hour, so if it's less, it'll cost the lower amount, if it's more, it'll cost the higher amount.

In any case, it's not expensive to do the full 3 weeks' worth of analysis, so I've started the required processing.

Meanwhile, I've attached the code for future reference.
Assignee: nobody → mreid
Status: NEW → ASSIGNED
(Assignee)

Comment 5

5 years ago
The v25.x analysis is complete, and results are available here:
https://s3-us-west-2.amazonaws.com/telemetry-public-analysis/bug965707/data/bug965707.YYYYMMDD.25.clean.txt

for YYYYMMDD values from 20131118 to 20131208
(Assignee)

Comment 6

5 years ago
The v26.x analysis is complete, and results are available here:
https://s3-us-west-2.amazonaws.com/telemetry-public-analysis/bug965707/data/bug965707.YYYYMMDD.26.clean.txt

for YYYYMMDD values from 20140113 to 20140202
Status: ASSIGNED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
Thank you. This data is very useful.
(Assignee)

Comment 11

5 years ago
You're welcome - let me know if you'd like to run this analysis again (or on a scheduled basis)
(In reply to Mark Reid [:mreid] from comment #11)
> You're welcome - let me know if you'd like to run this analysis again (or on
> a scheduled basis)

Yes, please. It would be good to run the following:

Firefox&Fennec 27 on the release channel from 2014-02-27 to 2014-03-16, inclusive.
Firefox&Fennec 28 on the release channel from 2014-04-07 to 2014-04-27, inclusive.
Firefox&Fennec 29 on the release channel from 2014-05-19 to 2014-06-08, inclusive.
Firefox&Fennec 30 on the release channel from 2014-06-30 to 2014-07-20, inclusive.
(Assignee)

Comment 13

5 years ago
Re-opening so these additional analyses don't get forgotten.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Now that we are past the end date of the Firefox 28 analysis period mentioned in comment 12, it would be good to run the analyses for 27 and 28.
(Assignee)

Comment 15

5 years ago
The analyses for v. 27 and 28 is complete.
(In reply to Mark Reid [:mreid] from comment #15)
> The analyses for v. 27 and 28 is complete.

Thank you!
Attachment #8420029 - Attachment mime type: text/plain → text/x-python
As expected, there's an improvement for Traditional Chinese and Thai in 28 compared to 27. Nice.
Could you please run the analysis for Firefox 29 and 30 per comment 12?
Flags: needinfo?(mreid)
(Assignee)

Comment 22

5 years ago
Will do
Flags: needinfo?(mreid)
(Assignee)

Comment 23

5 years ago
Version 29 per-data is available now at URLs like:
https://s3-us-west-2.amazonaws.com/telemetry-public-analysis/bug965707/data/bug965707.20140519.29.clean.txt

Version 30 data is on the way.
(Assignee)

Comment 24

5 years ago
Version 30 data is also available on S3 (same URL pattern)
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago5 years ago
Resolution: --- → FIXED
Thank you!

Interpretation:
zh-CN improved, as expected, from 28 to 29. zh-TW improved, as expected, from 29 to 30. However, the improvements were within the noise when comparing over a longer period of time, so it's hard to tell if there's any improvement of *significance*. That there wasn't a notable improvement for zh-TW from 29 to 30 is particularly disappointing. The lack of improvement for ko is also disappointing.

It appears that to improve things here, it's necessary to address the behavior in the case of local files and/or the .com/.org/.net TLDs. I'll file another telemetry request to check that hypothesis.
(In reply to Henri Sivonen (:hsivonen) from comment #25)
> I'll file another telemetry request to check that hypothesis.

Bug 1056542.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.