Closed Bug 507024 Opened 11 years ago Closed 11 years ago

enable winmo nightly updates

Categories

(Release Engineering :: General, defect, P3)

ARM
Windows Mobile 6 Professional
defect

Tracking

(Not tracked)

RESOLVED INCOMPLETE

People

(Reporter: aki, Unassigned)

References

Details

(Whiteboard: [winmo] bug 548773 for releases)

Attachments

(5 files, 4 obsolete files)

8.13 KB, patch
aki
: review+
Details | Diff | Splinter Review
695 bytes, patch
morgamic
: review+
Details | Diff | Splinter Review
3.19 KB, patch
armenzg
: review+
Details | Diff | Splinter Review
10.70 KB, patch
aki
: review+
Details | Diff | Splinter Review
1.17 KB, patch
aki
: review+
Details | Diff | Splinter Review
For blassey.  stuart says before winmo fennec b1 would be good.
Would be worth checking if bug 504437 also applies to WinMo, suspect it will given the same Visual Studio version. I used the LiveHTTPHeaders extension to capture that URL.
Flags: blocking1.9.2?
Question: this is only for nightly builds, not a release channel, correct?

Looks like I'll be spending the bulk of this week on fennec l10n for b4.
Flags: blocking1.9.2? → blocking1.9.2-
why is bug 511967 blocking this?
We're doing so many updates now (with l10n nightlies) that en-US updates were showing up after noon PST.  We need to move the update work onto the build slaves where there's a lot more parallel processing power.
that's not a bad thing, but that still doesn't block doing winmo updates
Ok, I can remove it.  Aiui, doing this before bug 511967 means we'll have to do this twice; afterwards, once.
No longer depends on: 511967
(In reply to comment #6)
> that's not a bad thing, but that still doesn't block doing winmo updates

Sure, it's not strictly blocking, but unless I hear that this is a short-term necessity, I'm going to follow the path where I do this work once (bug 511967) instead of twice.
Depends on: 534025
Constantly getting pinged about this, as we have ~1500 winmo users (aiui) who aren't testing the latest builds due to this bug.

Not sure when I'll have time for this so I'm throwing back in the pool and hoping someone's interested.  Coop says this should proceed before he's done with updates-on-slaves.
Assignee: aki → nobody
(In reply to comment #9)
> Constantly getting pinged about this, as we have ~1500 winmo users (aiui) who
> aren't testing the latest builds due to this bug.
> 
> Not sure when I'll have time for this so I'm throwing back in the pool and
> hoping someone's interested.  Coop says this should proceed before he's done
> with updates-on-slaves.

Talked with blassey at the all-hands about this, and yes, we're all agreed this should go ahead. We've decided not to block on the updates-on-slaves work (bug 511967) bug we are blocked on getting complete mars generated for winmo (bug 534025). Not sure who should pick up that bug/work. blassey?
how is bug 511967 blocking this bug?
Wasn't that removed back in comment 7 ?
never mind, misread comment 10.  Sounds like nothing is blocking moving forward on this.  ETA?
I'll be picking this up tomorrow. Armen might grab it instead, but he's pretty busy with WinCE stuff right now.
blassey: one thing I'll need to proceed is the OS/ABI identifier string for a WinMo build as generated by the updater.

I don't know how easy this is to do on a WinMo device. On other platforms, it's a matter of editing the firefox.js prefs file and adding a preference for "app.update.log=true", starting the build from a console, and then watching the console output as you click the "Check for Updates..." menu option.

e.g. when I do that on my Mac, I see this in the output:

*** AUS:SVC Checker:getUpdateURL - update URL: https://aus2.mozilla.org/update/3/Firefox/3.6b4/20091124201530/Darwin_Universal-gcc3/en-US/beta/Darwin%2010.2.0/default/default/update.xml?force=1

We're looking for the WinMo-equivalent of "Darwin_Universal-gcc3" in the above output, although posting the entire output wouldn't hurt either.
coop, is this the same as the TARGET_XPCOM_ABI that is defined by configure?
(In reply to comment #16)
> coop, is this the same as the TARGET_XPCOM_ABI that is defined by configure?

Partly. Here's where the string gets put together:

http://mxr.mozilla.org/mozilla-central/source/toolkit/mozapps/update/src/nsUpdateService.js.in#1932

Casual inspection of the WinMo nightly logs indicates to me that the final string will be WINCE_arm-msvc, which is the same as for WinCE. This shouldn't bite us as long as the product name remains Fennecat least that's what I take away from bug 515748.
WINCE_arm-msvc sounds right.  But, I brad did change the target to:

--target=arm-wince-winmo

When is your nightly from?
yeah, that nightly is not correct.  blassey did post (maybe even filed a bug?) about switching the --target and the nightly box doesn't have that change.


can we make that change?  The mozconfig should have this:

ac_add_options --target=arm-wince

be

ac_add_options --target=arm-wince-winmo
bug 515748 only landed on mozilla-central so far, so the target change would break the build on m-1.9.2.
(In reply to comment #21)
> bug 515748 only landed on mozilla-central so far, so the target change would
> break the build on m-1.9.2.

Do we want to re-open bug 515748 until we get the requested 1.9.2 approval for https://bugzilla.mozilla.org/attachment.cgi?id=416128&action=edit ?
generally bugs are closed when they're fixed on trunk.
I've got a build step generating a mar for WinMo trunk in staging now. Next chunk of work is generating a complete snippet and then getting mars and snippets uploaded to the correct places.
Assignee: nobody → ccooper
Status: NEW → ASSIGNED
Priority: -- → P2
I have WinMo nightlies generating/uploading a mar and uploading a snippet in staging now. I'll post some patches for Aki to review shortly.

I'm currently investigating what AUS and patcher changes will be required to make this work.
This is running on staging-nightly-updates right now and successfully generate

I added some extra output behind the $verbose flag as well.
Attachment #419466 - Flags: review?(nrthomas)
This is my stab at updating the AUS config to accept Fennec as an updatable product. 

morgamic: please let me know if there's more work to do here.
Attachment #419468 - Flags: review?(morgamic)
There's a tiny bit of cleanup in MobileBuildFactory also.

config patch is coming next.
Attachment #419472 - Flags: review?(aki)
I made one change here that's a little more intrusive, and that's setting 'stage_base_path' as '/home/ftp/pub/mobile' for all the mobile builds. These were all inheriting from the main config before, so all our mobile builds were going into /pub/firefox. 

I *think* we want our mobile builds going directly into /pub/mobile/, but I haven't heard anything official either way.
Attachment #419474 - Flags: review?(aki)
Comment on attachment 419472 [details] [diff] [review]
Generate complete mar and snippet for WinMo (buildbotcustom)

Nit: I think catlee wants lists to have [spaces, in, between]. I don't mind either way though.

r=me; thanks for doing this.
Attachment #419472 - Flags: review?(aki) → review+
Comment on attachment 419474 [details] [diff] [review]
Generate complete mar and snippet for WinMo (buildbot-configs)

Did you intend to point to mobile/nightly instead of firefox/nightly?

Currently mobile/nightly is a softlink to ../../firefox/nightly/latest-mobile-trunk, which is a constant source of confusion.  However, unless we fix everything, we'll either a) end up with two nightly locations (if we remove the softlink first) or b) end up with firefox/nightly/latest-mobile-trunk/latest-mobile-*-l10n/ which will be worse.

So I'd recommend holding off on that and futuring the mobile upload fix. However, if you want to tackle that as part of this, then we need to warn everyone, then change all the firefox/nightly and firefox/tinderbox-builds references to mobile/* and remove the mobile/nightly softlink.
Comment on attachment 419468 [details] [diff] [review]
Add Fennec to the list of products in AUS

Missing a comma.  Otherwise, looks good.  Will have to make sure the multi-locale thing maps to a real filepath.  Other than that things should work fine.
Attachment #419468 - Flags: review?(morgamic) → review-
Comment on attachment 419474 [details] [diff] [review]
Generate complete mar and snippet for WinMo (buildbot-configs)

Hm, on second viewing looks like you've already updated firefox->mobile (minus removing the softlink) -- did it work?

Another nit -- the staging nightly times. Not sure if this is important though.
Is there a bug on adding the required app.update.* prefs to mobile.js for this to actually work?
(In reply to comment #34)
> Is there a bug on adding the required app.update.* prefs to mobile.js for this
> to actually work?

I looked, didn't find anything, so I filed bug 537181.
(In reply to comment #32)
> (From update of attachment 419468 [details] [diff] [review])
> Missing a comma.  Otherwise, looks good.  Will have to make sure the
> multi-locale thing maps to a real filepath.  Other than that things should work
> fine.

Hrmmm, thinking a little more about this, we'll want to have both mozilla-central- and 1.9.2-based versions of Fennec in AUS (I've been using m-c for testing, but 1.9.2 is the current release version). Unfortunately, Fennec makes no version distinction between m-c and 1.9.2 at present, i.e. builds from both branches have the same version. 

I filed bug 537314 on this.
(In reply to comment #33)
> (From update of attachment 419474 [details] [diff] [review])
> Hm, on second viewing looks like you've already updated firefox->mobile (minus
> removing the softlink) -- did it work?

It worked fine in staging, but I wasn't really paying attention to anything aside from WinMo. 

If it's easier to just go with firefox/* instead of mobile/*, I can do that. I'm not looking to change the world here (necessarily). Is there a bug filed for that change (firefox/->mobile/ upload dest) already?
 
> Another nit -- the staging nightly times. Not sure if this is important though.

Yeah, I'll put that back.
Comment on attachment 419474 [details] [diff] [review]
Generate complete mar and snippet for WinMo (buildbot-configs)

r+ with the staging nightly schedule nit.

We need to coordinate since there's a little bit that needs to happen on stage.m.o before this, but I should have that ready soon.
Attachment #419474 - Flags: review?(aki) → review+
There are a bunch of things that need resolution before this can push forward:

* what is the product called on WinMo? I see that bug 537396 is now fixed, but does that also apply to WinMo? Do we still want to setup AUS to handle a separate Fennec product if we're simply going to rename it to Firefox? Maybe we do for development work, but let's explicitly make that call.

* add the required prefs to allow for app updating: bug 537181

* determine how to differentiate between WinCE (tegra) and WinMo devices that are requesting updates. If tegra==Firefox and WinMo==Fennec then we dodge around this, but AFAICT both devices will request updates as WINCE_arm-msvc otherwise. I don't think this was addressed by bug 515748 because the updater relies on what the underlying OS reports. Maybe we patch the updater?

* use different version numbers for Fennec built against different branches: bug 537314
Depends on: 537314
Comment on attachment 419466 [details] [diff] [review]
Allow patch-packager to process updates for mobile builds

Sorry for the delay, I lost track of this request. r+ and for bonus points you could s/emptry/empty/.
Attachment #419466 - Flags: review?(nrthomas) → review+
what's left for this?
(In reply to comment #41)
> what's left for this?

The short list is in comment 39. AFAICT those are all product considerations rather than RelEng issues. 

* The AUS changes are trivial but it requires a decision on the product side.

* Without the required prefs, we can't update WinMo builds anyway -> bug 537181

* Differentiating between WinCE and WinMo so we don't offer updates to the wrong platform. Maybe we get a better picture of this once the update prefs are in place and we can see the request they are making, but it should be possible to piece together what that request would be now.

* Differentiate between version numbers so we don't update to the wrong branch. I don't know how Fennec versioning works right now. If building off trunk were to be renamed as 2.0pre (similar to what we do for Firefox) and 1.9.2 stays as 1.0, that would work, provided the versioning isn't done solely in the mobile code (unless we get a separate repo for mobile-1.9.2, of course).
(In reply to comment #42)
> (In reply to comment #41)
> > what's left for this?
> 
> The short list is in comment 39. AFAICT those are all product considerations
> rather than RelEng issues. 
>(In reply to comment #39)
>> There are a bunch of things that need resolution before this can push forward:
>> 
>> * what is the product called on WinMo? I see that bug 537396 is now fixed, but
>> does that also apply to WinMo? Do we still want to setup AUS to handle a
>> separate Fennec product if we're simply going to rename it to Firefox? Maybe we
>> do for development work, but let's explicitly make that call.
Fennec is the project code name, much like Namoroka. Nightlies and alphas will be called Fennec. Betas and final will be Firefox
>> 
>> * add the required prefs to allow for app updating: bug 537181
patch submitted, requested gavin's review
>> 
>> * determine how to differentiate between WinCE (tegra) and WinMo devices that
>> are requesting updates. If tegra==Firefox and WinMo==Fennec then we dodge
>> around this, but AFAICT both devices will request updates as WINCE_arm-msvc
>> otherwise. I don't think this was addressed by bug 515748 because the updater
>> relies on what the underlying OS reports. Maybe we patch the updater?
Is the problem the update url? If that's the case we can differentiate there. Also, if that's the case, we'd have the same problem with the block list.
>> 
>> * use different version numbers for Fennec built against different branches:
>> bug 537314
discussion happening in that bug.
This work is currently scheduled for March to coincide with other RelEng work on WinMo scaling. As such, it will also benefit from the work in bug 511967.

We may have the opportunity to pick this up again sooner, depending on the completion of other goals.
Assignee: ccooper → nobody
Status: ASSIGNED → NEW
Component: Release Engineering → Release Engineering: Future
Priority: P2 → P3
Priority: P3 → --
Whiteboard: [winmo]
Depends on: 540007
Comment on attachment 419472 [details] [diff] [review]
Generate complete mar and snippet for WinMo (buildbotcustom)

http://hg.mozilla.org/build/buildbotcustom/rev/6bf6a7214cea
Attachment #419472 - Flags: checked-in+
Attachment #419474 - Flags: checked-in+
Comment on attachment 419474 [details] [diff] [review]
Generate complete mar and snippet for WinMo (buildbot-configs)

http://hg.mozilla.org/build/buildbot-configs/rev/bd800f5bdecc
Comment on attachment 419466 [details] [diff] [review]
Allow patch-packager to process updates for mobile builds

Checking in patch-packager-production.cfg;
/mofo/release/patcher/patch-packager-production.cfg,v  <--  patch-packager-production.cfg
new revision: 1.6; previous revision: 1.5
done
Checking in patch-packager-staging.cfg;
/mofo/release/patcher/patch-packager-staging.cfg,v  <--  patch-packager-staging.cfg
new revision: 1.5; previous revision: 1.4
done
Checking in patch-packager.pl;
/mofo/release/patcher/patch-packager.pl,v  <--  patch-packager.pl
new revision: 1.32; previous revision: 1.31
done
Attachment #419466 - Flags: checked-in+
(In reply to comment #38)
> We need to coordinate since there's a little bit that needs to happen on
> stage.m.o before this, but I should have that ready soon.

Patches are landed and running in staging. 

I'll roll up a config patch for production but as Aki says, we'll want to fix up the symlinks on stage.m.o before making the dir change for firefox/ -> mobile/ there.
I don't think bug 537314 blocks this since the update url has both the fennec version and the gre milestone in it.
No longer depends on: 537314
Added dep.bug as we cant keep piling even more onto the single machine (prometheus-vm) generating all updates.
Depends on: 511967
as discussed earlier, that seems like a nice thing to have but in no way does it block this getting done.
(In reply to comment #50)
> Added dep.bug as we cant keep piling even more onto the single machine
> (prometheus-vm) generating all updates.

(In reply to comment #51)
> as discussed earlier, that seems like a nice thing to have but in no way does
> it block this getting done.

Valid point - strictly speaking, we could load WinMO updates onto the same machine, generated last after all the usual updates for 1.9.0, m-c, and the recently added updates for 1.9.1, 1.9.2, across all locales. The WinMO updates would be available late in the day, but so long as we stay under 24hours to generate all the updates, then we're always done before the next nightly updates start.

However, we're focusing on bug#527314 first, because generating updates on pool-o-slaves:
- is a pre req for updates on project branches (including Lorentz)
- is needed before we could support another release branch like 3.7
- eliminates single point of failure.

This is being actively worked on; sorry for not linking the dep bugs earlier so you could see that.
Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.
Component: Release Engineering: Future → Release Engineering
Priority: -- → P3
Two things are still blocking this:
* updating AUS to use the new GUID or PLATFORM_MILESTONE URL format (bug 540007). The ball is in morgamic's court on this AFAIK.
* getting the changes for bug 535369 landed on m-c, which will unblock bug 511967. I've been stuck waiting for the tree to re-open like everyone else here.
Assignee: nobody → ccooper
Status: NEW → ASSIGNED
Priority: P3 → P2
This has already been working in staging for a while.
Attachment #427445 - Flags: review?(armenzg)
Comment on attachment 427445 [details] [diff] [review]
Turn on complete mar creation for WinMo on 1.9.2 in production

Stamped!
Attachment #427445 - Flags: review?(armenzg) → review+
Comment on attachment 427445 [details] [diff] [review]
Turn on complete mar creation for WinMo on 1.9.2 in production

http://hg.mozilla.org/build/buildbot-configs/rev/53ca588e1294
Attachment #427445 - Flags: checked-in+
pm02 has been reconfig-ed. Should have complete MARs for 1.9.2 tomorrow.
Status: ASSIGNED → NEW
Priority: P2 → P3
Depends on: 546695
No longer depends on: 540007
Whiteboard: [winmo] → [winmo] bug 548773 for releases
Got asked about this today, some comments
* we're not getting any complete mars for 1.9.2 or m-c WinMo nightlies at the moment, comment #58 implies we should be
* need to publish snippets to .../build/0/{a23983c0-fd0e-11dc-95ff-0800200c9a66} instead of 0/Fennec
* in attachment 419466 [details] [diff] [review] need to s/Fennec/{a23983c0-fd0e-11dc-95ff-0800200c9a66}/, generating a partial works ok with this
* to get builds seeing updates with something like attachment 419468 [details] [diff] [review], this WFM in config-dist.php in the productBranchVersion array
    '{a23983c0-fd0e-11dc-95ff-0800200c9a66}'      =>  array(
        '*_1.9.2*' => 'mozilla-1.9.2',
        '*_1.9.3*' => 'mozilla-central'
    ),
(In reply to comment #59)
> Got asked about this today, some comments
> * we're not getting any complete mars for 1.9.2 or m-c WinMo nightlies at the
> moment, comment #58 implies we should be

This is because attachment 419474 [details] [diff] [review] is only for staging
We're using %VERSION%-%GRE_VERSION% to identify Fennec now, as decided in bug 546695.
Attachment #419468 - Attachment is obsolete: true
Attachment #432606 - Flags: review?(morgamic)
Switch to using appropriate Fennec version to generate partial patches, as decided in bug 546695.
Attachment #419466 - Attachment is obsolete: true
Attachment #432634 - Flags: review?(armenzg)
Attachment #432634 - Flags: review?(armenzg) → review+
Attachment #419474 - Attachment is obsolete: true
Attachment #427445 - Attachment is obsolete: true
Attachment #432652 - Flags: review?(aki)
Comment on attachment 432652 [details] [diff] [review]
Use the new identifier when uploading snippets for WinMo

Looks like just updates for 1.9.2?  WFM.
Attachment #432652 - Flags: review?(aki) → review+
We weren't setting the snippet var in production, which is why we haven't been generating complete mars.

(In reply to comment #64)
> (From update of attachment 432652 [details] [diff] [review])
> Looks like just updates for 1.9.2?  WFM.

Yeah, one branch at a time, especially until we're doing updates on slaves.
Attachment #432662 - Flags: review?(aki)
Comment on attachment 432634 [details] [diff] [review]
Allow patch-packager to process updates for mobile builds, v2

Checking in patch-packager.pl;
/mofo/release/patcher/patch-packager.pl,v  <--  patch-packager.pl
new revision: 1.33; previous revision: 1.32
done
Attachment #432634 - Flags: checked-in+
Attachment #432662 - Flags: review?(aki) → review+
Comment on attachment 432652 [details] [diff] [review]
Use the new identifier when uploading snippets for WinMo

http://hg.mozilla.org/build/buildbot-configs/rev/2633409cf8ae
Attachment #432652 - Flags: checked-in+
Comment on attachment 432606 [details] [diff] [review]
Add Fennec to the list of products in AUS, v2

This should work.  Looks good to me.
Attachment #432606 - Flags: review?(morgamic) → review+
Comment on attachment 432606 [details] [diff] [review]
Add Fennec to the list of products in AUS, v2

Checking in xml/inc/config-dist.php;
/cvsroot/mozilla/webtools/aus/xml/inc/config-dist.php,v  <--  config-dist.php
new revision: 1.67; previous revision: 1.66
done

morgamic: what's the procedure to get this AUS change into production now that it's landed?
Attachment #432606 - Flags: checked-in+
Mike's not CC'd here so I'll answer that instead. There's some tagging to do and then file a bug against IT. I can take care of that with attachment 425743 [details] [diff] [review] and push them both live at the same time.
Tagging done at bug 542142 comment #17. Bug 553421 to get the change here deployed.
Depends on: 553421
Into stasis it goes.
Assignee: ccooper → nobody
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → INCOMPLETE
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.