511662 - Request .CAB files generated for winCE 1.9.2 branch nightlies builds

Status: VERIFIED FIXED

(Release Engineering :: General, defect, P2)

Description

[Tony Chung [:tchung]]

Nick, Vlad, 
what kind of work is needed if we could get .cab files in the 1.9.2 nightly directories for the winCE builds?   Currently with the zip files, we still have to unpack them on another machine, and then transfer the unpacked goods by hand over to the tegra builds.   Having a .cab file stored in the nightlies would allow me to just self extract them instead nicely directly onto winCE.  This would save a lot of time since copying is so slow.   

Its worse with the Mobinova devices, cause they dont even allow you to currently talk to a windows machine with the broken cables we have.  So stuff is transferred via usb key.

Comment 1

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

It just requires running build-wince-cab.py with the -s option to generate an uncompressed cab file.  Not sure what releng's schedule is for getting this going.

Comment 2

[Tony Chung [:tchung]]

joduinn mention in pasing that this is on RE's radar.  cc'ing him for comments.

Comment 3

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Once we get the WinCE builds with zips working in production, we'll start to investigate CAB files next. Unclear whats involved with these CAB files, or if we'll need help, as we are focusing on the builds first.

/me adds dep. bug for tracking.

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

I'll take a look at this, although it's more of a build config kind of thing.

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Actually, how does this fit together with bug 502933 ? Looks like CABs are going away in favor of self-extracting zips.

Comment 6

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Different -- that's for windows mobile, for CE we still want a CAB (though we may also want a self-extracting zip as well, but we need CABs initially for integration with OEMs).

We should be able to build a cab in a very similar way to how we build the zip (or at least how we build fennec) -- if we have everything that we'd zip up in 'firefox' directory, we can do a cab with 'build-wince-cab.py -s firefox Firefox Firefox.CAB' (note the -s is required on Windows CE, and shouldn't be there for windows Mobile).

In the future I'll need to patch build-wince-cab.py so that it can inject extra stuff into the .inf file it generates, but that'll be a separate patch.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Tony, could you take 
  http://people.mozilla.org/~nthomas/test/firefox-3.7a1pre.en-US.wince-arm.cab
for a spin and see if it works. It's 31MB in size.

Vlad, this is the output from make_wince_cab.py:
  WARNING: No files in directory [firefox\defaults]
  WARNING: No files in directory [firefox\extensions]
  INFORMATION: Executing command to make Firefox CAB file (only works on BASH)
      ["D:\msvs8\SmartDevices\SDK\SDKTools\cabwiz.exe"  Firefox.inf]
The two warnings don't seem right, as the cab contains all the files in those two directories.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Ted, where would you suggest adding in cab generation into the build system. Right now mobile-browser uses the installer target to create cabs, 
 http://mxr.mozilla.org/mobile-browser/source/installer/Makefile.in#124
but installer for desktop Firefox is pretty complicated. It's tempting to have a WinCE specific call to build-wince-cab.py on the package target instead.

Vlad, WinMo is currently using the VS2005 cabwiz because 'VSINSTALLDIR=D:\msvs8' is set during packaging. Any objections to switching that to use the VS2009 one ?

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #7)
> The two warnings don't seem right, as the cab contains all the files in those
> two directories.

I really mean that they're referred to in _setup.xml. Haven't tested on a device.

Comment 10

[Tony Chung [:tchung]]

(In reply to comment #7)
> Tony, could you take 
>   http://people.mozilla.org/~nthomas/test/firefox-3.7a1pre.en-US.wince-arm.cab
> for a spin and see if it works. It's 31MB in size.
> 
Sure, i'll hop on the breadboard tomorrow morning and report back here.

Comment 11

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

(In reply to comment #8)
> Vlad, WinMo is currently using the VS2005 cabwiz because
> 'VSINSTALLDIR=D:\msvs8' is set during packaging. Any objections to switching
> that to use the VS2009 one ?

Nope, that should be fine... shouldn't be much difference between the two, though the vs2009 one is probably better since that's what we build.

(In reply to comment #7)
> Vlad, this is the output from make_wince_cab.py:
>   WARNING: No files in directory [firefox\defaults]
>   WARNING: No files in directory [firefox\extensions]
>   INFORMATION: Executing command to make Firefox CAB file (only works on BASH)
>       ["D:\msvs8\SmartDevices\SDK\SDKTools\cabwiz.exe"  Firefox.inf]
> The two warnings don't seem right, as the cab contains all the files in those
> two directories.

Pretty sure I've seen that before; I think it's harmless -- those two directories only contain other directories inside them, no actual files, which is I think what the script is indicating.

Comment 12

[Tony Chung [:tchung]]

Some Observations:
- Downloading the CAB was successful to the assigned directory
- The CAB file icon is a yellow box with a document image.  Do we have a firefox installer icon for winCE to use here?
- Upon installing, If an existing firefox exists, it pops a dialog saying "Mozilla Firefox is already installed. Re-install?"
- Clicking "OK" will then prompt the folder location to install to.  I would have expected this dialog to appear BEFORE the previous installer check, that way users arent confused that they will be still have a choice to write to another directory if they choose to.
- CAB file extracts and copies just fine.  But its dog SLOOOOWWWWW. It took around 15-18 minutes to complete the installation.  (result of .CAB file architecture?)

Everything else looks good nick!

Comment 13

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

(In reply to comment #12)
> - The CAB file icon is a yellow box with a document image.  Do we have a
> firefox installer icon for winCE to use here?
> - Upon installing, If an existing firefox exists, it pops a dialog saying
> "Mozilla Firefox is already installed. Re-install?"
> - Clicking "OK" will then prompt the folder location to install to.  I would
> have expected this dialog to appear BEFORE the previous installer check, that
> way users arent confused that they will be still have a choice to write to
> another directory if they choose to.

We can't affect any of these, unfortunately.  This is all part of the OS; the WinMo guys are building an installer which we could also package, but we need CAB files for OEMs.

> - CAB file extracts and copies just fine.  But its dog SLOOOOWWWWW. It took
> around 15-18 minutes to complete the installation.  (result of .CAB file
> architecture?)

The slow install is partially due to CABs, but also partially due to hardware issues on our current hardware.

Comment 14

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Build cabs with 'make package MOZ_PKG_FORMAT=CAB'

Moves the INSTALLER_DIR definition outside of the #ifndef MOZ_PKG_FORMAT as Ted suggested, adds the handling for MOZ_PKG_FORMAT=CAB, and sets INSTALLER_PACKAGE for WinCE to get include cab on 'make upload'. 

Works for WinCE, and there were no packaging regressions when I ran it through the try server. Also checked there was no change in MOZ_PKG_FORMAT & INSTALLER_DIR for the following OS_ARCH/MOZ_WIDGET_TOOLKIT combos: WINNT/windows, WINCE/windows, Linux/gtk2, Linux/qt, Linux/photon, Darwin/cocoa, SunOS/gtk2, OS2/os2.

Comment 15

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Add call to buildbot to create cab

Not great to add another windows special case, but this will probably be relatively short lived and get replaced by a self-extracting zip of bug 502933.

Comment 16

[[e-mail] (:bhearsum)]

Comment on attachment 397593 [details] [diff] [review]
Add call to buildbot to create cab

It sucks that we need to do this, but it looks fine.

Comment 17

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

(In reply to comment #15)
> Not great to add another windows special case, but this will probably be
> relatively short lived and get replaced by a self-extracting zip of bug 502933.

Won't be replaced, sadly -- see my comment earlier, we'll always need CAB files as a release option for ODM/OEMs.

Comment 18

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Comment on attachment 397592 [details] [diff] [review]
Build cabs with 'make package MOZ_PKG_FORMAT=CAB'


>+ifeq ($(OS_ARCH),WINCE)
>+CABARGS += -s
>+endif

I don't think this will work -- for Windows Mobile, we don't want -s (they need a compressed cab file), but for CE we do.  WINCE is defined on both.. you may be able to get away with doing a check to see if WINCE_WINDOWS_MOBILE is not defined in addition to OS_ARCH WINCE.

Comment 19

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #18)
> I don't think this will work -- for Windows Mobile, we don't want -s (they need
> a compressed cab file), but for CE we do.  WINCE is defined on both.. you may
> be able to get away with doing a check to see if WINCE_WINDOWS_MOBILE is not
> defined in addition to OS_ARCH WINCE.

Didn't hit this on the Try server because WinMo is traversing this path
 http://hg.mozilla.org/mobile-browser/file/tip/installer/Makefile.in#l114
rather than the one I added. I could put in a WINCE_WINDOWS_MOBILE guard if you like though.

Comment 20

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Add WINCE_WINDOWS_MOBILE guard for CABARGS -s

Comment 21

[Chris Cooper [:coop] (he/him)]

Comment on attachment 397783 [details] [diff] [review]
Add WINCE_WINDOWS_MOBILE guard for CABARGS -s

Ted: gentle ping on the review request. We need to get something like this in place before the end of next week.

Comment 22

[(not currently active) Ted Mielczarek]

I'll hit my review queue tomorrow or friday, promise.

Comment 23

[Nick Thomas [:nthomas] (UTC+12)]

Ted, gonna have time to review this soon ?

Comment 24

[(not currently active) Ted Mielczarek]

Dang, broke my promise. I'll queue it up for first thing tomorrow morning.

Comment 25

[(not currently active) Ted Mielczarek]

Comment on attachment 397783 [details] [diff] [review]
Add WINCE_WINDOWS_MOBILE guard for CABARGS -s

+# This is how we create the binary packages we release to the public.

Nice :)

+ifneq (,$(filter-out mac cocoa,$(MOZ_WIDGET_TOOLKIT)))

Just change this to ifneq (cocoa,$(MOZ_WIDGET_TOOLKIT)) while you're here. Gonna bitrot philor's patch in bug 495228 if you land before him anyway.

+ifeq ($(MOZ_PKG_FORMAT),CAB)
+PKG_SUFFIX	= .cab
+ifdef MOZ_FASTSTART
+CABARGS += -faststart
+endif
+ifeq ($(OS_ARCH),WINCE)
+ifndef WINCE_WINDOWS_MOBILE

The WINCE check seems redundant. I wouldn't worry about that.

Otherwise looks fine. (as much as any patch to packager.mk can.)

Comment 26

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: (as landed) CAB packaging

Addresses Ted's comments, carrying r+.

http://hg.mozilla.org/mozilla-central/rev/307da4daceb8
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/3ff8d644eedd

Comment 27

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 397593 [details] [diff] [review]
Add call to buildbot to create cab

http://hg.mozilla.org/build/buildbotcustom/rev/e51a24b98af2

p-m and p-m02 reconfiged (all branches) just before the packaging patch landed.

Comment 28

[Nick Thomas [:nthomas] (UTC+12)]

m-1.9.2 and m-c have successfully produced cabs.

Comment 29

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Nagios monitoring for nightly cabs

Comment 31

[Tony Chung [:tchung]]

Thanks nick!

Comment 32

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 397593 [details] [diff] [review]
Add call to buildbot to create cab

Forgot to reconfig the try master to pick up this change, but done now.

Comment 33

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: Ordering of -s and -faststart arguments must be correct

Landing bug 511984 and bug 516516 broke the cab creation because we passed '-faststart -s' to make_wince_cab.py instead of '-s -faststart'. Fixed by landing this patch as
 http://hg.mozilla.org/mozilla-central/rev/953b2f685290
 http://hg.mozilla.org/releases/mozilla-1.9.2/rev/2e178391791f

Doesn't affect mobile-browser because uncompressed cabs (-s) are not used there.