748389 - "Missing WebRT Files" when opening app

Status: VERIFIED FIXED

(Firefox Graveyard :: Web Apps, defect, P1)

Description

[Ian Bicking (:ianbicking)]

I installed an application using a version of Firefox compiled from m-c.  Upon opening I get the message: "Missing WebRT Files – This version of Firefox (20120418145447) cannot run web applications, because it is not recent enough or damaged"


I have several versions of Firefox installed (Release, Aurora, Nightly).  Nightly (from 04-24-2012) is the default browser.

Comment 1

[Jason Smith [:jsmith]]

Ian - How many Nightly installations do you have installed?

Comment 2

[Jason Smith [:jsmith]]

Dan - This is the third time I've seen this happen by 3 different users. Should we consider a different approach to multiple firefox installations for Mac? I believe Windows follows this approach, which does not appear to be hitting the same issues that Mac hits:

- The app runs on the firefox version the app was installed with

Mac I recall you mentioning that it runs with the latest firefox installation. Myk - Thoughts on how to resolve this?

Comment 3

[Ian Bicking (:ianbicking)]

I have two built-from-source Firefoxes, and one Nightly release.  20120418 I believe is when I installed the app, but I have updated and rebuilt that Firefox since then.

Normal users won't be building their own browsers, but someday some people will have Aurora and Beta installed, and that is a mainstream situation.  Would this also be a problem there?

Also, if we don't fully resolve this now, we should at least give a better error message to help testers understand what is going on.

Comment 4

[Dan Walkowski]

There are multiple issues here that are interacting with each other.
First, we don't bump the OS version number when we update the nightly/aurora. (Today's Nightly is v 14.0a1, just as it has been for weeks.  This means the OS has no way to tell which is 'newest'.  There is a hidden version number, (CFBundleVersion) not shown to users (who are shown CFBundleShortVersionString), which we ought to be putting the build number into,  say "14.0a1 20120424".
This would fix most of the issues people are having when they have more than one Nightly or Aurora, or whatever.  But I agree with Ian, most of our testers won't have multiple around.

Comment 5

[Myk Melez [:myk] [@mykmelez]]

Hmm, the latest nightly build on my machine does advertise itself as 15.0a1.  However, I do like the idea of including the build ID in a version string to differentiate between multiple nightly builds.

Comment 6

[Dan Walkowski]

Yes, the particular version number was not my point.  The point is that we are not taking advantage of the hidden version number, and having unfortunate user-visible behavior because of it.

Comment 7

[:Gavin Sharp [email: [e-mail]]]

Attachment: add build ID to Firefox's CFBundleVersion

This appends the build ID to Firefox's CFBundleVersion, which results in e.g.:

	<key>CFBundleVersion</key>
	<string>14.0a1.20120424121854</string>

Comment 9

[:Felipe Gomes (needinfo for replies!)]

is this something that we should do only for unbranded builds?

Comment 10

[:Felipe Gomes (needinfo for replies!)]

oh and will we be able to remove this ifdef? http://hg.mozilla.org/mozilla-central/annotate/b952bb042f12/browser/modules/WebappsInstaller.jsm#l591

I added it for debug builds but it doesn't seem to be making a difference at all on debug builds for me

Comment 11

[Myk Melez [:myk] [@mykmelez]]

I don't know if the value is case-sensitive, but the one I've been using is org.mozilla.nightlydebug.

Comment 12

[Dan Walkowski]

Ok.  I have just spent three hours trying to understand and create a strategy for how to deal with this issue.  For some time, I was convinced that app version comparison/selection was broken in Lion.  I have just figured out what is going on though.  

Let me sum up:
-------------

1) remembering paths to apps is a Bad Idea in OSX
2) the most flexible way is to ask the OS to launch the app with signature "com.bar.foo"
3) the OS knows where all apps are, using a cache, and launches the one with the signature you asked for, no matter where it is.
4) if there is more than one with the same signature, it compares version numbers, and chooses the one with the highest version number
5) if the version numbers are the same, a heuristic is used, which generally means you cannot predict which will be launched.
6) i could not get it to work, even when appending our build number to the end of our version number


And the answer is that version numbers in OS X must be in a particular format (!)

nnnnn.nn.nn{other ignored stuff}, where 'n' must be a digit.

So, the bad news is that to get this to work properly, we need to reformat our build number to fit in that shape.  The good news is that it goes in a field in Info.plist that is not visible to the user, so it doesn't matter.

Comment 13

[Ian Bicking (:ianbicking)]

To clarify the solution, this means that apps will always be run with the most recent version of Firefox available?  So if I install Nightly then all my apps will start opening with Nightly instead of Aurora, even if I otherwise don't use Nightly at all?

Comment 14

[Dan Walkowski]

Providing an internal version number of this form will eliminate a lot of user (tester) confusion, because the newest Nightly/Aurora/Firefox will always be used by webapps, regardless of how many they have on their machine. (I've seen as many as 6)

Ian, it is already the case that there is a precedence: Nightly > Aurora > Firefox, and that your apps will launch with the earliest one in that list you have.  What this change would fix is the precedence -among- all the Nightlies/Auroras/Firefoxes you have, which is not deterministic at the moment.

I do not think it is unreasonable to assume that people who install nightlies want to use them.  And in particular, during our testing periond, they are the only ones that will work.  Creating a precedence list seemed the least confusing to the most of our users.

Testers can always override the selection by editing a text file, and use a different binary.

Comment 15

[Ed Lee :Mardak]

(In reply to Dan Walkowski from comment #14)
> I do not think it is unreasonable to assume that people who install
> nightlies want to use them.
Not necessarily. Someone could have just installed Nightly once to try it out but then went back to Release/Beta/Aurora. The user might not even realize it's still installed. So a user's Nightly could be stuck on say.. 12 when Release is now on 13.

Comment 16

[Dan Walkowski]

There is no perfect solution.  There are three different apps (nightly, aurora, and firefox), all with different app signatures and different version numbers.  There are -going- to be people who end up in a spot where they need to delete one of them, or edit the Info.plist file (text) to get things to behave the way they want.
Right now, that happens to be most of them, and I'm trying to reduce that number by making the typical case work smoothly.

I'm happy to discuss alternative strategies if you have ideas.

Comment 17

[Ian Bicking (:ianbicking)]

If it was sticky that would perhaps be better.  I.e., save the location of the Firefox that originally created the application, and only do the signature launch if that path is not available.

Comment 18

[:Felipe Gomes (needinfo for replies!)]

Attachment: Add FirefoxBinary for every build (not only debug)

I copied the code to generate LOWER_MAC_APP_NAME from browser/app/Makefile.in to browser/modules/Makefile.in. not sure if it's possible to move that to a more top-level place accessible by both

Comment 19

[:Gavin Sharp [email: [e-mail]]]

Attachment: better CFBundleVersion patch

This makes the CFBundleVersion be:
[majorversion][2-digityear].[month].[day]

Comment 20

[:Gavin Sharp [email: [e-mail]]]

Comment on attachment 619137 [details] [diff] [review]
Add FirefoxBinary for every build (not only debug)

I think we should just add some logic to configure to AC_DEFINE an appropriate MAC_BUNDLE_ID (e.g. near where we handle MOZ_APP_DISPLAYNAME) and then we can use it in both places (and for other apps if desired).

Comment 21

[Dan Walkowski]

Ian, the combination of the patches from gavin and felipe (thank you both, for working with me on this and making it possible), combined with my original code, will mean that webapps will prefer to be opened by the flavor* of Firefox they were installed with.  In addition, they will use the most recent instance of that flavor.  If that flavor of Firefox isn't available, the original precedence list will be invoked.

The FirefoxBinary slot I put into the webapp Info.plist, intended for developer fiddling, is now always populated with the installing firefox signature, making it sticky.

The version number change, to the Apple standard format, means that the OS will correctly select the newest available binary with that flavor.

* Nightly, Aurora, or Firefox

Comment 22

[:Gavin Sharp [email: [e-mail]]]

Attachment: Add 'FirefoxBinary' to all webapp bundles

This also consolidates the MOZ_MACBUNDLE_ID logic in configure (for use in browser/app for Firefox's ID, as well as WebappsInstaller.jsm for the webapp installer).

Comment 23

[:Gavin Sharp [email: [e-mail]]]

Hrm, so even with both of those patches applied, PathToWebRT in webapprt.mm keeps finding an old nightly in one of my objdirs (CFBundleIdentifier=="org.mozilla.nightly" and CFBundleVersion=="14.0a1"), rather than the build that the webapp was installed from (CFBundleIdentifier=="org.mozilla.nightly" and CFBundleVersion=="1512.04.29"). I'm running OS X 10.6.8, if that makes any difference.

http://www.cocoabuilder.com/archive/cocoa/218445-finding-other-apps-paths-deterministically.html suggests that the only way to get deterministic behavior here would be to register a fake protocol handler and use LSCopyApplicationURLsForURL with that fake scheme to get a list of supporting apps (and maybe do version comparisons ourselves?). Seems like a pretty bad hack :/

Comment 24

[:Gavin Sharp [email: [e-mail]]]

(In reply to :Gavin Sharp (use [e-mail] for email) from comment #23)
> Hrm, so even with both of those patches applied, PathToWebRT in webapprt.mm
> keeps finding an old nightly in one of my objdirs
> (CFBundleIdentifier=="org.mozilla.nightly" and CFBundleVersion=="14.0a1"),
> rather than the build that the webapp was installed from
> (CFBundleIdentifier=="org.mozilla.nightly" and
> CFBundleVersion=="1512.04.29").

I tried making the CFBundleVersion be 1512.4.29 to check whether the leading 0 was throwing the comparison off, but that didn't seem to help.

Comment 25

[Dan Walkowski]

Gavin, the change you made to the version number would not have been recognized unless you touched the file.  The finder caches all that info, and doesn't expect it to change.

Comment 26

[Dan Walkowski]

Oh, and by touched the file, I mean the .app directory.  You can also move the app to a new location, which also (almost always) causes the Finder to update its cache.

Comment 27

[:Gavin Sharp [email: [e-mail]]]

(In reply to Dan Walkowski from comment #25)
> Gavin, the change you made to the version number would not have been
> recognized unless you touched the file.  The finder caches all that info,
> and doesn't expect it to change.

Hmm, OK. Touching the new bundle and also launching it seemed to make things work (touching it alone apparently wasn't sufficient).

(It would be really cool if there was a way to view the contents of this bundleID/version->application patch cache, for testing purposes. Do you have any idea where it's stored on disk, or what terms I could use for googling?)

Comment 28

[Dan Walkowski]

r+ from me

Comment 29

[Steven Michaud [:smichaud] (Retired)]

Please give me a testcase for this bug:

For example, you could attach a dummy web app and provide steps-to-reproduce.

Even better, you could provide the source for dummy web app (together with instructions on how to build it, if "building" is necessary).

Comment 30

[:Gavin Sharp [email: [e-mail]]]

I don't think there are any guaranteed STR. The best test scenario I can think of is:
1) Have a bunch of different Nightly builds installed (some older than others). Maybe launch or "touch" an old one to make sure it's "known".
2) In a recent one, install a webapp from https://apps.mozillalabs.com/appdir/ (click the install button, and then click "Allow" in the prompt)
3) Go to your Applications directory in Finder and try to launch the newly installed app

The tricky part is getting into a suitable state at 1) - since the heuristic for determining which org.mozilla.nightly bundle to use is unknown to us (per comment 12), the bug depends on OS X launching an old version that isn't suitable for launching webapps. Specifying a CFBundleVersion that matches Apple's suggested format and that can be used to disambiguate is meant to prevent that from ever happening.

But mostly I just wanted you, as someone I see as generally knowledgeable about Mac stuff, to sign off on the idea of modifying our CFBundleVersion, and to comment about the format I've chosen (perhaps comparing it with other schemes that you're familiar with). If you don't have any feedback about those things in general, feel free to ignore the feedback request - Dan Walkowski has already signed off and also is knowledgeable about this stuff, I was just being broad in my request for feedback, since this isn't my area of expertise.

Comment 31

[Steven Michaud [:smichaud] (Retired)]

Thanks for what STR you've given.  I know very little about web apps, so it's helpful.

I know more about the launch services database, and how Apple uses it to choose which app to launch under various circumstances.  I need the STR to check whether you guys have wandered off into the woods :-)

I'll dig further into this later this week.

Comment 32

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Comment on attachment 619156 [details] [diff] [review]
better CFBundleVersion patch

Review of attachment 619156 [details] [diff] [review]:
-----------------------------------------------------------------

I'm not qualified to review weird Mac things.

Comment 33

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

Comment on attachment 619425 [details] [diff] [review]
Add 'FirefoxBinary' to all webapp bundles

Review of attachment 619425 [details] [diff] [review]:
-----------------------------------------------------------------

Also redirecting.

Comment 34

[Dan Walkowski]

Note: I have in the code a number of logging statements that indicate lots of useful things like which particular Firefox is used when you launch a webapp.  Use the Console.app on OS X to see them.

Comment 35

[Steven Michaud [:smichaud] (Retired)]

Sorry, but I wasn't able to get to this today.  I'll work through it first thing on Monday (using the STR Gavin gave me).

It's not that I really think you've gotten lost in the woods.  I just want to dot the i's and cross the t's.

Comment 36

[Steven Michaud [:smichaud] (Retired)]

Comment on attachment 619156 [details] [diff] [review]
better CFBundleVersion patch

I played around with this a bit, and am satisfied with it.

I tested with two builds -- 1) one from current mozilla-central code plus both of Gavin's patches, and 2) the 2011-07-04 mozilla-central nightly (whose bundle id is org.mozilla.nightly, but which doesn't contain WebRT support).

Before any of my tests, and between tests, I rebuilt the Launch Services database with the following command (with no line breaks):

/System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices/Support/lsregister -kill -r -domain user -domain local -domain system

I installed a Web app (BarFight) from https://apps.mozillalabs.com/appdir/ (using the first of my two test builds), and did all my tests with that.

I found (by manipulating each test build's CFBundleVersion) that the OS always chose the test build with the "higher" CFBundleVersion when I double-clicked on /Applications/BarFlight.app.  I didn't get consistent results when I made both test builds' CFBundleVersion the same and just changed the dates of their top-level directories (Nightly.app).  But I don't think that's really a problem.

Comment 37

[:Gavin Sharp [email: [e-mail]]]

Thanks for that testing, Steven!

Comment 38

[Dan Walkowski]

Steven, in my testing, it appears to pick the most recently used app when more than one have the same version, even if they didn't (have the same version) during the last use.

Comment 39

[Steven Michaud [:smichaud] (Retired)]

Dan:

Now that I remember back, I think that also matches my results.

Either way, though, it's not a big deal.

Comment 40

[(not currently active) Ted Mielczarek]

Comment on attachment 619156 [details] [diff] [review]
better CFBundleVersion patch

Review of attachment 619156 [details] [diff] [review]:
-----------------------------------------------------------------

::: browser/app/Makefile.in
@@ +196,5 @@
>  else
>  APPFILES = MacOS
>  endif
>  
> +MAC_BUNDLE_VERSION = $(shell $(PYTHON) $(srcdir)/macversion.py --version=$(MOZ_APP_VERSION) --buildid=$(shell cat $(DEPTH)/config/buildid))

Can you make this script pull the buildid from the file instead of having to run a second shell just to cat it? Shell invocations add up.

::: browser/app/macversion.py
@@ +3,5 @@
> +from optparse import OptionParser
> +import sys
> +import re
> +
> +o = OptionParser()

FYI, generally we like our scripts in the build system to be encapsulated in a main() or some such, so that we can import them as modules if need be.

@@ +30,5 @@
> +twodigityear = options.buildid[2:4]
> +month = options.buildid[4:6]
> +day = options.buildid[6:8]
> +
> +print majorVersion + twodigityear + "." + month + "." + day
\ No newline at end of file

This seems a little weird, but I guess you're working within the odd constraints of the version number scheme here.

For comparison, here's what we did for Windows:
http://mxr.mozilla.org/mozilla-central/source/config/version_win.pl#254

Windows versions have 4 fields, so we used major.minor.point.<days since jan 1, 2000 to when this build was created>.

If, as you say, this is not a user-visible version, and smichaud is ok with it, then I don't have a strenuous objection.

This line would read a bit clearer to me if you used string formatting instead of concatenation.

Comment 41

[:Gavin Sharp [email: [e-mail]]]

Attachment: updated CFBundleVersion patch

Comment 42

[(not currently active) Ted Mielczarek]

Comment on attachment 619425 [details] [diff] [review]
Add 'FirefoxBinary' to all webapp bundles

Review of attachment 619425 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry for the delay on reviews. This looks fine, just a few comments.

::: browser/modules/WebappsInstaller.jsm
@@ +591,2 @@
>      <key>FirefoxBinary</key>\n\
> +#expand     <string>__MOZ_MACBUNDLE_ID__</string>\n\

You're going to change the actual value used here, does that matter?

::: configure.in
@@ +8521,5 @@
>  AC_SUBST(MOZ_MACBUNDLE_NAME)
>  
> +dnl Mac bundle identifier (based on MOZ_APP_DISPLAYNAME)
> +MOZ_MACBUNDLE_ID=`echo $MOZ_APP_DISPLAYNAME | tr '[A-Z]' '[a-z]'`
> +MOZ_MACBUNDLE_ID=org.mozilla.${MOZ_MACBUNDLE_ID}

This probably ought to be somewhat-configurable for non-Mozilla consumers, but I guess it doesn't matter that much right now since you're only using it in one place.

@@ +8526,5 @@
> +if test "$MOZ_DEBUG"; then
> +  MOZ_MACBUNDLE_ID=${MOZ_MACBUNDLE_ID}debug
> +fi
> +
> +AC_DEFINE_UNQUOTED(MOZ_MACBUNDLE_ID,$MOZ_MACBUNDLE_ID)

Note that AC_DEFINES don't necessarily make it into files where we use Preprocessor.py, so you'll want to be careful that your define is actually taking effect where you think it should be.

Comment 43

[:Gavin Sharp [email: [e-mail]]]

(In reply to Ted Mielczarek [:ted] from comment #42)
> You're going to change the actual value used here, does that matter?

Yes, that was intentional.

> This probably ought to be somewhat-configurable for non-Mozilla consumers,
> but I guess it doesn't matter that much right now since you're only using it
> in one place.

It was previously hardcoded in the plist, so this doesn't make it any worse. But indeed we probably should use MOZ_DISTRIBUTION_ID.

Thanks for the review.

Comment 44

[:Gavin Sharp [email: [e-mail]]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/c0be912b90b4
https://hg.mozilla.org/integration/mozilla-inbound/rev/1541548bee21

Comment 45

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/c0be912b90b4
https://hg.mozilla.org/mozilla-central/rev/1541548bee21

Comment 46

[Jason Smith [:jsmith]]

Verified on OS X 10.7.