Migrate orphaned users to latest m-c

RESOLVED WORKSFORME

Status

defect
P3
normal
RESOLVED WORKSFORME
9 years ago
6 years ago

People

(Reporter: joduinn, Assigned: joduinn)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [updates])

Attachments

(2 attachments)

In today's Aurora meeting, RelEng were asked to create updates for orphaned nightly users of:

FF4.2a1pre
FF34.0.13pre 
FF4.0.12pre

...to allow them migrate to latest m-c (6.0a1) build (on nightly channel). This will help build up the nightly user coverage on m-c.
Based on metrics numbers, if we get all of those users to migrate, we will be adding ~15k users to m-c.
pentaho shows around 20k

4.2a1pre=  8,982	
13pre   = 10,255
12pre   =  2,082

I think the lower numbers that daniel passed around this afternoon were from these releases plus additional "nightly" channel info.

I'm wondering if it is possible to have a 4.2a1pre, 13pre, or 12pre version user that is not on the nightly channel, on another channel, or is reporting no channel information?

that would explain the difference in the numbers daniel's channel report passed around this afternoon and latest numbers on pentaho. 

if there are 4.2a1pre, 13pre, or 12pre that aren't on the nightly channel, is is possible to construct the snippets to capture those users too?
Looks like we already provide proper snippets for 4.2a1pre and it should be upgraded to latest 6.0a1, see

https://aus3.mozilla.org/update/3/Firefox/4.2a1pre/0000000000/Linux_x86_64-gcc3/en-US/nightly/Linux%202.6.38-8-generic%20%28GTK%202.24.4%29/default/default/update.xml?force=1

<?xml version="1.0"?>
<updates>
    <update type="minor" version="6.0a1" extensionVersion="6.0a1" buildID="20110420030555">
        <patch type="complete" URL="http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2011/04/2011-04-20-03-mozilla-central/firefox-6.0a1.en-US.linux-x86_64.complete.mar" hashFunction="sha512" hashValue="a567763a073ca26003d7cd270ebc2f531566da8c1c70023364582b59381105d5a9a224cfefec276a8670292cfe5971f3b8edfd77326560839d6ecf1b070ac9ac" size="15894800"/>
    </update>
</updates>
Assignee: nobody → rail
Priority: -- → P2
Attachment #527320 - Flags: review?(rhelmer) → review+
Comment on attachment 527320 [details] [diff] [review]
Add AUS2 exceptions for 4.0.12pre and 4.0.13pre

$ cvs commit -m "Bug 651283 - Migrate orphaned users to latest m-c. p=rail, r=rhelmer"
Checking in inc/config-dist.php;
/cvsroot/mozilla/webtools/aus/xml/inc/config-dist.php,v  <--  config-dist.php
new revision: 1.131; previous revision: 1.130
done


$ cvs tag AUS2_PRODUCTION
W inc/config-dist.php : AUS2_PRODUCTION already exists on version 1.130 : NOT MOVING tag to version 1.131

$ cvs tag -F AUS2_PRODUCTION
T inc/config-dist.php

$ cvs tag AUS2_RTM_`date +%Y%m%d%H%M`
(which created AUS2_RTM_201104202159)
Attachment #527320 - Flags: checked-in+
Depends on: 651808
(In reply to comment #2)
> pentaho shows around 20k
> 
> 4.2a1pre=  8,982    
> 13pre   = 10,255
> 12pre   =  2,082
When was this data collected? Was this collected, and measured, in the same way as the rest of teh data presented Tuesday?


> I think the lower numbers that daniel passed around this afternoon were from
> these releases plus additional "nightly" channel info.
> 
> I'm wondering if it is possible to have a 4.2a1pre, 13pre, or 12pre version
> user that is not on the nightly channel, on another channel, or is reporting no
> channel information?
> 
> that would explain the difference in the numbers daniel's channel report passed
> around this afternoon and latest numbers on pentaho. 

deinspanjer: can you clarify where the difference in numbers comes from?




> if there are 4.2a1pre, 13pre, or 12pre that aren't on the nightly channel, is
> is possible to construct the snippets to capture those users too?
aiui, 4.2a1pre, 13pre, or 12pre should *only* be on the nightly channel. The 5k users (difference between 20k and 15k) is such a tiny percentage of our ADUs, maybe these are frankenfox, or some error in our measuring? 

If deinspanjer thinks the numbers are accurate, and the cause of the number discrepancy is different channels, we should revisit. It may be possible to figure out what channels they are on, ignore how they got there, and custom-generate snippets to update them back to something normal. Not sure how these custom updates would be QA'd, but lets deal with that if we need to, after we hear from deinspanjer.
> pentaho shows around 20k
> 
> 4.2a1pre=  8,982    
> 13pre   = 10,255
> 12pre   =  2,082
When was this data collected? 

2011-04-19

Was this collected, and measured, in the same way
as the rest of the data presented Tuesday?

the number I put here are from our standard pentaho report 
https://metrics.mozilla.com/pentaho/content/pentaho-cdf/RenderHTML?solution=metrics&path=blocklist/blocklist_0&dashboard=template.html&template=new-metrics&title=Firefox%20Usage

I'm not sure about the source of the build + channel numbers I've never seen that report before.
Here are updated breakdowns by channel for those three versions.  Note there are another 5 channels reported, but there are only one or two daily pings on them so I left them out to save room:
[Channels].[Unknown], [Channels].[default-cck-opensuse], [Channels].[default-cck-mozillaonline], [Channels].[nightly-cck-yandex], [Channels].[release-cck-yandex]

http://screencast.com/t/i1J558OMZ0AU
(In reply to comment #8)
> Here are updated breakdowns by channel for those three versions.  Note there
> are another 5 channels reported, but there are only one or two daily pings on
> them so I left them out to save room:
> [Channels].[Unknown], [Channels].[default-cck-opensuse],
> [Channels].[default-cck-mozillaonline], [Channels].[nightly-cck-yandex],
> [Channels].[release-cck-yandex]
> 
> http://screencast.com/t/i1J558OMZ0AU

"default" are probably try builds, or other non-nightly builds.
(In reply to comment #9)
> (In reply to comment #8)
> > Here are updated breakdowns by channel for those three versions.  Note there
> > are another 5 channels reported, but there are only one or two daily pings on
> > them so I left them out to save room:
> > [Channels].[Unknown], [Channels].[default-cck-opensuse],
> > [Channels].[default-cck-mozillaonline], [Channels].[nightly-cck-yandex],
> > [Channels].[release-cck-yandex]
> > 
> > http://screencast.com/t/i1J558OMZ0AU
> 
> "default" are probably try builds, or other non-nightly builds.

874 of these non-nightly channel pings from 4.0b12pre builds, 
and 3635 pings from similar 4.0b13pre builds make the numbers seem very high if this is just try builds.  earlier this month there were 13,000 4.013pre "default channel" users.

the better questions are what are these other possible sources for builds, and maybe even more important are the update snippets catching them?
(In reply to comment #10)
> (In reply to comment #9)
> > (In reply to comment #8)
> > > Here are updated breakdowns by channel for those three versions.  Note there
> > > are another 5 channels reported, but there are only one or two daily pings on
> > > them so I left them out to save room:
> > > [Channels].[Unknown], [Channels].[default-cck-opensuse],
> > > [Channels].[default-cck-mozillaonline], [Channels].[nightly-cck-yandex],
> > > [Channels].[release-cck-yandex]
> > > 
> > > http://screencast.com/t/i1J558OMZ0AU
> > 
> > "default" are probably try builds, or other non-nightly builds.
> 
> 874 of these non-nightly channel pings from 4.0b12pre builds, 
> and 3635 pings from similar 4.0b13pre builds make the numbers seem very high if
> this is just try builds.  earlier this month there were 13,000 4.013pre
> "default channel" users.

We've had people blog about try builds, so that's not too surprising to me.

> the better questions are what are these other possible sources for builds, and
> maybe even more important are the update snippets catching them?

nothing happens ATM for people on "default", they're completely ignored
I've just noticed that there is a possible typo in comment #0

> FF34.0.13pre 
> FF4.0.12pre

Shouldn't they be 4.0b13pre and 4.0b12pre? If yes, the patch should be rewritten.
Yes, it should be 4.0b13pre and 4.0b12pre
tl;dr: I think we're already offering updates on the nightly channel, and I don't know what more RelEng can do to influence people to update faster. We also have to be careful about how we use our metrics data.

Not the tl;dr:
* The application versions of interest are 4.0b12pre, 4.0b13pre, 4.2a1pre; which should be offered 6.0a1 (ie current mozilla-central tip)
* I agree with Rail what we're already providing those paths for the *nightly* channel
* Also agree that attachment 527320 [details] [diff] [review] is faulty. I'd suggest reverting it

* Try builds are actually on the nightly channel, so people get updated back to mozilla-central (whether they want to or not)
* Builds end up on the default channel when there's not a 
   ac_add_options --enable-update-channel=<something>
line in the mozconfig, which means people doing their own builds, and any distributor not aware of how this stuff works
* The updater is enabled by default in Firefox builds, but (eg) linux distributions will pretty reliably disable updates because they have their own package systems
* In this scenario the build would not ping AUS, but would hit blocklist on the default channel

* The ADU in comment #8 are from blocklist data, which I can reproduce. Switching it to AUS data results in higher counts, up ~50% for 4.2a1pre, while pretty much suppressing the pings on default (see this attachment). This represents the users actually pinging us for updates, the people we can reach
* I'm not sure I believe higher AUS pings, but we do query every 8 hours for nightly builds, instead of every 24 hours (if the app is open, of course). Dunno if metrics are somehow crunching that to give unique users, or if it's just the raw number of pings. Daniel ?

* The platform and locale distributions for the 'orphaned' users are broadly consistent with unorphaned ones. So we're not leaving some platforms behind, and the vast majority are windows users - about a 50%/50% split between XP and Windows 7. (based on AUS data)

* The ADU in comment #8 do show the blocklist counts are dropping, by 50% for 4.0b13pre over 20 days, and by 80% for 4.2a1pre in the 7 days since 2011-04-13 (version change). I'm inclined to believe that people are using the updater, rather than manually upgrading using an installer. So the problem boils down to the counts not dropping fast enough

Questions:
* How do the count decays vs time compare to version changes for nightlies builds of 3.5/3.6/4.0 branches ? Have we regressed the 'punctuality' of the updater ?

* Does the updater give up on downloading an update if there's a newer one available ? I don't think so, but worth asking rs. For users that don't run Firefox for long periods, and do background updates, this could easily result in downloading only a portion of a complete update, then throwing that away for the next one.
There isn't any magic we can do to de-dupe AUS pings. Also the Linux distro problem is an important reason why we switched to using blocklist pings to track.
Comment on attachment 527320 [details] [diff] [review]
Add AUS2 exceptions for 4.0.12pre and 4.0.13pre

(In reply to comment #15)
> * Also agree that attachment 527320 [details] [diff] [review] is faulty. I'd suggest reverting it

Reverted:

$ cvs commit -m "Bug 651283 - Migrate orphaned users to latest m-c. Reverted revision 1.131."
cvs commit: Examining .
Checking in config-dist.php;
/cvsroot/mozilla/webtools/aus/xml/inc/config-dist.php,v  <--  config-dist.php
new revision: 1.134; previous revision: 1.133
done

$ cvs -q tag AUS2_PRODUCTION
W inc/config-dist.php : AUS2_PRODUCTION already exists on version 1.133 : NOT MOVING tag to version 1.134

$ cvs -q tag -F AUS2_PRODUCTION
T inc/config-dist.php

$ cvs tag AUS2_RTM_`date +%Y%m%d%H%M`
(which was AUS2_RTM_201104260458)
Attachment #527320 - Flags: checked-in+ → checked-in-
Depends on: 652788
(In reply to comment #15)

> * Does the updater give up on downloading an update if there's a newer one
> available ? I don't think so, but worth asking rs. For users that don't run
> Firefox for long periods, and do background updates, this could easily result
> in downloading only a portion of a complete update, then throwing that away for
> the next one.

rstrong: can you help clarify how updater works in this scenario?
(In reply to comment #15)

> * Does the updater give up on downloading an update if there's a newer one
> available ? I don't think so, but worth asking rs. For users that don't run
> Firefox for long periods, and do background updates, this could easily result
> in downloading only a portion of a complete update, then throwing that away for
> the next one.
You are correct. It doesn't give up on the download of the update when a newer one is available.
We *might* want to override extension compatibility checks for these nightly users by setting the extensionVersion in the snippet to the same version as the Firefox being offered the update which will cause the update to be downloaded in the background.
Might be a good thing to offer updates to the remaining users from bug 518508 in this bug if they aren't being offered updates at this time.
Back to the pool. I can grab it again if we have some clear scenario here.
Assignee: rail → nobody
Priority: P2 → --
Priority: -- → P3
Whiteboard: [updates]
Assignee: nobody → joduinn
Per RelEng meeting, these updates are already in place, and being offered to users of older nightlies. Nothing left for us to do here, so closing as WFM. 

Please reopen with details if there's anything for us to do that I missed.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.