Last Comment Bug 732518 - [tracking] Get an experimental build of win32 Fennec running in metro
: [tracking] Get an experimental build of win32 Fennec running in metro
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: General (show other bugs)
: Trunk
: x86_64 Windows 8.1
: -- normal with 4 votes (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
http://piercedotzler.com/asa/Developi...
Depends on: 686835 731807 732124 732127 732589 736953
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-02 11:44 PST by Jim Mathies [:jimm]
Modified: 2014-07-24 11:06 PDT (History)
38 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
setmetro.cpp (6.56 KB, text/plain)
2012-03-02 11:45 PST, Jim Mathies [:jimm]
no flags Details
Installer shortcut dual mode (41.92 KB, patch)
2012-03-03 23:34 PST, Brian R. Bondy [:bbondy]
no flags Details | Diff | Review
turn off console on fennec rel builds (627 bytes, patch)
2012-03-07 10:33 PST, Jim Mathies [:jimm]
no flags Details | Diff | Review
fix for svg 'generic' keyword collision (2.73 KB, patch)
2012-03-12 15:53 PDT, Jim Mathies [:jimm]
no flags Details | Diff | Review
widget gesture code header fixes (2.54 KB, patch)
2012-03-12 15:53 PDT, Jim Mathies [:jimm]
no flags Details | Diff | Review
build options (8.08 KB, patch)
2012-03-12 15:54 PDT, Jim Mathies [:jimm]
no flags Details | Diff | Review
wifi code header fixes (953 bytes, patch)
2012-03-12 15:54 PDT, Jim Mathies [:jimm]
no flags Details | Diff | Review
minor cosmetics for browser code (2.40 KB, patch)
2012-03-12 15:56 PDT, Jim Mathies [:jimm]
no flags Details | Diff | Review
d3d mem reporter fix (610 bytes, patch)
2012-03-12 16:11 PDT, Jim Mathies [:jimm]
no flags Details | Diff | Review
Connecting thebes to metro (28.92 KB, patch)
2012-03-16 12:27 PDT, Brian R. Bondy [:bbondy]
no flags Details | Diff | Review
experimental xre changes (67.44 KB, patch)
2012-03-16 14:22 PDT, Jim Mathies [:jimm]
no flags Details | Diff | Review
Support for metro snap (13.99 KB, patch)
2012-03-17 16:12 PDT, Brian R. Bondy [:bbondy]
no flags Details | Diff | Review

Description Jim Mathies [:jimm] 2012-03-02 11:44:21 PST
Purely for proof of concept. This bug can hold code / patches written.

I've been working my way down the ms doc trying to get a generic fennec build to show up in metro. A couple of notes thus far:

1) The PRI resource list can be built using a command line tool in the sdk - makepri.exe. We'll need to add a make check for this and then use it to package up the visual resources. For experimenting now I'm just using a copy of ie's files and manifest.

2) I'll attach a command line tool that can be used to create shortcuts and set the appropriate properties needed to launch into metro. I hope to get this integrated into our build.

I haven't started work on various make junk yet. We'll need a new platform target or maybe a new build project for this. That work can be done later after we have a poc running.
Comment 1 Jim Mathies [:jimm] 2012-03-02 11:45:34 PST
Created attachment 602421 [details]
setmetro.cpp
Comment 2 Brian R. Bondy [:bbondy] 2012-03-03 23:34:36 PST
Created attachment 602693 [details] [diff] [review]
Installer shortcut dual mode

For anyone using Jim's shortcut program, I think the correct sample command line is something like:
setmetroshortcut.exe /CREATE /T"C:\Program Files (x86)\Nightly\firefox.exe" /S"C:\USERS\bbondy\Desktop\test123.lnk" /M /AEEFEA8717BC47F65
Then move the shortcut to the start menu folder.

---

I also attached a patch for Firefox that does this in the installer.
It pops up a messagebox to tell you if it worked or not.
This is not to be included in the code, but just for testing.

---

Couple of other notes:

- When I go to metro search by the way it shows me the name I put in VisualElementsManifest.xml even if it is different from the shortcut.

- You can also get the resource files just by creating a simple test metro app by the way.
Comment 3 Jim Mathies [:jimm] 2012-03-05 04:22:53 PST
One of the things I haven't figured out from the doc is whether the exe has to be tagged with these properties. It doesn't look like it needs to be, kjust the shortcuts, but I'm not 100% sure.
Comment 4 Brian R. Bondy [:bbondy] 2012-03-05 20:49:32 PST
A bit of progress...

Assuming your app user model ID hash is EEFEA8717BC47F65 and you have a shortcut with dual mode set and that app model user id set, copy the shortcut file to:
C:\ProgramData\Microsoft\Windows\Start Menu\Programs


then import the following into regedit:

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\EEFEA8717BC47F65]

[HKEY_CLASSES_ROOT\EEFEA8717BC47F65\.exe]

[HKEY_CLASSES_ROOT\EEFEA8717BC47F65\.exe\shell]
@="open"

[HKEY_CLASSES_ROOT\EEFEA8717BC47F65\.exe\shell\open]
"CommandId"="FF.Launch"

[HKEY_CLASSES_ROOT\EEFEA8717BC47F65\.exe\shell\open\command]
@="\"C:\\Program Files (x86)\\Nightly\\firefox.exe\" %*"
"DelegateExecute"="{17FE9752-0B5A-4665-84CD-569794602F5C}



----


The interesting part is that it doesn't read the shortcut target when you click the icon in start menu. Instead it will read directly from the shell/open/command registry entry.  If you for example change the above registry to point to c:\Windows\system32\calc.exe it will open that instead even know the shortcut target is still firefox.exe.
Comment 5 Brian R. Bondy [:bbondy] 2012-03-05 20:53:07 PST
By the way, IE's app user model ID for this is Microsoft.InternetExplorer.Default
Comment 6 Brian R. Bondy [:bbondy] 2012-03-05 21:50:31 PST
I found the location of the ProgID that is used for the association:
HKEY_CURRENT_USER\Software\Microsoft\Windows\Shell\Associations\UrlAssociations\http\UserChoice

The value of ProgId is linked to:
HKEY_CLASSES_ROOT\ProgIdHere
Comment 7 Jim Mathies [:jimm] 2012-03-06 02:11:34 PST
(In reply to Brian R. Bondy [:bbondy] from comment #4)
> A bit of progress...
> 
> 
> The interesting part is that it doesn't read the shortcut target when you
> click the icon in start menu. Instead it will read directly from the
> shell/open/command registry entry.  If you for example change the above
> registry to point to c:\Windows\system32\calc.exe it will open that instead
> even know the shortcut target is still firefox.exe.

This is in the doc - 

Any shortcut that contains a PKEY_AppUserModel_ID value will cause Windows to first look for a verb handler registered for the shortcut’s target’s type registered under the AppUserModelID. Browser shortcuts will point at the browser’s executable file and thus must be registered for the ‘.exe’ type under the AppUserModelID. When the verb is invoked, it will be provided with the shortcut’s target and any arguments stored in the link (this includes arguments specified in secondary tiles). 

We'll definitely need this to make it launch. 

This doesn't explain the lack of proper tiles though unless windows is doing some internal checks first to insure the default browser is configured correctly.
Comment 8 Jim Mathies [:jimm] 2012-03-06 02:55:31 PST
I'm finding interesting tidbit using procmon and monitoring explorer.exe while launching the browser. One registry area it spends a lot of time in is HKCR/Launcher.X where 'X' has a number of variations depending on the type of app.
Comment 9 Jim Mathies [:jimm] 2012-03-06 03:25:48 PST
Here's a sample for implementing the ExecuteCommandVerb - nice, clean, C++ without any templating crap.

http://archive.msdn.microsoft.com/shellintegration
Comment 10 Jim Mathies [:jimm] 2012-03-06 04:14:25 PST
(In reply to Jim Mathies [:jimm] from comment #9)
> Here's a sample for implementing the ExecuteCommandVerb - nice, clean, C++
> without any templating crap.
> 
> http://archive.msdn.microsoft.com/shellintegration

This works pretty well. If you get it into your test project and run it once, it'll register with the shell for text file context menu handling. If your test project is set at the default browser, you can launch from metro and receive the verb action. The sample app throws up some dialog based debug output when this happens.
Comment 11 Jim Mathies [:jimm] 2012-03-07 10:33:44 PST
Created attachment 603782 [details] [diff] [review]
turn off console on fennec rel builds
Comment 12 Brian R. Bondy [:bbondy] 2012-03-08 17:53:35 PST
So we have a working metro enabled browser application (not an actual browser yet), we haven't integrated our Firefox / Fennec code yet.
Below is some gathered information by Jim and I that wasn't readily available to us, and that we had to figure out the long way.

Jim, could you add/modify/correct the info below so we can share with other browser vendors?

Useful supplementary information for creating a Desktop enabled browser
- The following document contains some useful information but is not complete:
  http://piercedotzler.com/asa/Developing_a_Metro-Style_Enabled_Desktop_Browser.html
- The shortcut mentioned in the doc should be created in %programdata%\Microsoft\Windows\Start Menu\Programs\
- You can set the default browser by going to Control panel, type Default in the search, and click on set default Programs, select the browser and click set this program as default.
- The important setting that needs to be set for metro to work for your browser is the HTTP protocol default
- You can see the progID of the registered browser here: HKEY_CURRENT_USER\Software\Microsoft\Windows\Shell\Associations\UrlAssociations\http\UserChoice, with value ProgId
- Your ProgID should be listed as a registered class and have a string value of AppUserModelID, it must have an Application key with string values of AppUserModelId, ApplicationName, ApplicationIcon, ApplicationDescription, ApplicationCompany
- You need to register your AppUserModelId as a class with default shell method open
- InternetExplorer's AppUserModelID is Microsoft.InternetExplorer.Default for reference
- You need to specify the DelegateExecute which is already registered inside this open command
- Your DelegateExecute will be invoked to start your browser
- The manifest of Metro apps is located at: HKCU\Software\Classes\ActivatableClasses\Package
- If you are registered correctly as the metro default browser, then your browser's info should be populated inside of the subkey HKCU\Software\Classes\ActivatableClasses\Package\DefaultBrowser_NOPUBLISHERID
- The list of contracts and which apps implement which contracts is located at HKCU\Software\Classes\Extensions\ContractId
- The most important contract that you need to worry about initially is the Windows.Launch contract
- If you are registered correctly as the metro default browser, then your browser's info should be populated inside of HKCU\Software\Classes\Extensions\ContractId\Windows.Launch\PackageId\DefaultBrowser_NOPUBLISHERID
- The Windows 8 start screen runs from explorer.exe
- Your DelegateExecute handler is invoked by Windows and it uses the command line -Embedding if it is an exe.
- Once setup correctly a second process will be started for your actual app where you can use the Metro Application singleton and Run() your application from there.  The command line -ServerName:DefaultBrowserServer will be passed here.

General concepts:
- There are 2 programming models for C++ in Windows 8
  - WRL: Similar to ATL, basically provides simplified COM access to the WinRT runtime
  - C++/CX: Some syntax sugar on top of C++ that makes code about 10x less large
  Both programming models compile down to native code, WRL gives you a bit more control.

Useful command line options:
- To get more information:
  /d1ZWtokens is the VC11 compiler option that dumps the C++ code generated from your C++/CX code. 
  /d1reportAllClassLayout shows class layouts
- To link to WRL include the following library in your additional linker command line options:
  /DEFAULTLIB:runtimeobject.lib
- To enable C++/CX syntax inside a Win32 application use the command line option:
  /ZW
- To link to the 2 needed winmd files add the following to the linker command line options:
  /AI"C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\vcpackages" 
  /AI "C:\Program Files (x86)\Windows Kits\8.0\Windows Metadata" 

Open questions:
- Need to figure out command line for generating PRI files.
- Need to figure out DLL activation instead of EXE activation.
Comment 13 Jim Mathies [:jimm] 2012-03-09 04:04:50 PST
This area was poorly documented on MSDN:

- Your DelegateExecute will be invoked to start your browser

- Your DelegateExecute handler is invoked by Windows and it uses the command line 
-Embedding if it is an exe.

- Once setup correctly a second process will be started for your actual app where you can use the Metro Application singleton and Run() your application from there.  

- The command line -ServerName:DefaultBrowserServer will be passed here.

The DelegateExecute handler (a COM local exe or a inproc dll server), which implements ICommandExecute and a few other interfaces is loaded to query how to open the browser. In the case of protocol or link handling this server should also launch / activate the browser as well, although we haven't made it this far to see how it works. So far we only have activation (launch/run) from a tile working, in which case Metro (explorer.exe) launches (runs) the browser.exe and the delegate server shuts down. So in the case of a tile launch this is a two step process leveraging two different executables.

Getting to this point required a lot of registry / explorer monitoring. Explorer is quite picky about stuff that must be set up before activating.

The entire test project up to getting a D3D demo running in place of the browser.exe can be found on the following user repo - 

http://hg.mozilla.org/users/jmathies_mozilla.com/metroresearch
Comment 14 Jim Mathies [:jimm] 2012-03-09 04:17:50 PST
Something I thought - we need to investigate themeing. I doubt the theme library supports metro style widget rendering, in which case we may have to emulate metro content widgets in css as nsNativeTheme may be of little use.

IE solves this currently by hard coding the use metro style themeing in both desktop and metro - this is a sucky tradeoff for ms. We can do better.
Comment 15 Mark Finkle (:mfinkle) (use needinfo?) 2012-03-09 14:43:12 PST
I have no idea if it helps out, but for Native Fennec on Android we use a messaging system to communicate between Java and Gecko (JS). Let me know if you think some background on that system might be useful to you. I can point you to some docs and code.
Comment 16 Jim Mathies [:jimm] 2012-03-09 15:34:51 PST
(In reply to Mark Finkle (:mfinkle) from comment #15)
> I have no idea if it helps out, but for Native Fennec on Android we use a
> messaging system to communicate between Java and Gecko (JS). Let me know if
> you think some background on that system might be useful to you. I can point
> you to some docs and code.

Thanks, hopefully though we won't have to jump through the same hoops - we'll be native in metro and working through a new windowing framework managed by the Metro process. So if all goes well we'll be using XUL and css for UI and D2D for display.
Comment 17 Brian R. Bondy [:bbondy] 2012-03-09 15:44:19 PST
We may use XAML for a container frame or some popup windows if applicable or useful.
Comment 18 Martin Anderson 2012-03-10 06:20:42 PST
Have you explored allowing XAML ResourceDictionaries for themeing?  XAML controls are very powerful and can be customised almost totally.  

If it is possible to allow for custom XAML styles to be used, it may be easier to implement than using XUL and rendering the UI to the DirectX surface.

Metro apps support overlaying XAML above the DirectX surface.
Comment 19 Sadiq ali 2012-03-10 11:42:39 PST
As Martin Anderson said XAML is much powerful and comes with lot of flexibility .It will much much easier to develop in c++ and innovate
Comment 20 Jim Mathies [:jimm] 2012-03-12 15:53:10 PDT
Created attachment 605181 [details] [diff] [review]
fix for svg 'generic' keyword collision
Comment 21 Jim Mathies [:jimm] 2012-03-12 15:53:49 PDT
Created attachment 605183 [details] [diff] [review]
widget gesture code header fixes
Comment 22 Jim Mathies [:jimm] 2012-03-12 15:54:25 PDT
Created attachment 605184 [details] [diff] [review]
build options
Comment 23 Jim Mathies [:jimm] 2012-03-12 15:54:53 PDT
Created attachment 605186 [details] [diff] [review]
wifi code header fixes
Comment 24 Jim Mathies [:jimm] 2012-03-12 15:56:12 PDT
Created attachment 605188 [details] [diff] [review]
minor cosmetics for browser code
Comment 25 Jim Mathies [:jimm] 2012-03-12 16:04:19 PDT
For building, use the MSVC 11 startup batch script 

https://hg.mozilla.org/mozilla-build/file/3ba6c690b036

and export a new LIBPATH as such:

LIBPATH='C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\LIB;C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\ATLMFC\LIB;C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0\ExtensionSDKs\Microsoft.VCLibs\11.0\References\CommonConfiguration\neutral;C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\vcpackages;C:\Program Files (x86)\Windows Kits\8.0\Windows Metadata;'

This is for 32-bit builds. I'll test 64-bit builds tomorrow. These changes get you a successful build using Win7 with VC11 installed. They should also work on win8, but I haven't had a chance to move the patches over there to confirm.

After all of this is added component extensions should be available for use in widget code.
Comment 26 Jim Mathies [:jimm] 2012-03-12 16:11:26 PDT
Created attachment 605202 [details] [diff] [review]
d3d mem reporter fix

forgot one.
Comment 27 Christophe Pichaud 2012-03-15 06:06:23 PDT
In reply to "comment 12", I saw this in a french information that Firefox on Metro is on track. 
On the general concepts:
<<General concepts: there are 2 programming models for C++ in Windows 8 : 
- WRL: Similar to ATL, basically provides simplified COM access to the WinRT   
- C++/CX: Some syntax sugar on top of C++ that makes code about 10x less large
Both programming models compile down to native code, WRL gives you a bit more >>

I have talked to Microsoft compiler guys and VS guys in france during TechDays after my C++ 11 talk and we have talked a lot about WinRT implementation. All is done entirely in C++, not in C++/CX. 

Every developer has the choice:
- stay with C++ standard
- use the new language extension (syntax is similar to C++/CLI but it is native)
There are a lot of fuzzy stuff since September information about BUILD conference and about C++ and WinRT presentations. Some informations were not clear about C++/CX was the new C++/CLI or a native stuff at first sight.

In my opinion, you should not use C++/CX not because you'll have to handle specific VC11 tools in build but because of the horrible syntax. it's not C++ for me ! It is specific VC11 language extension.

If you stay in standard C++, some things could be migrate from the CRT folder (Visual Studio\VC\CRT) which is MS C runtime. There is a VCCORLIB folder where there are some WinRT stuff. Everything can be called using full standard C++ and WRL. The same folks who made ATL have made WRL.
WRL is not old COM with macros and ugly things. It's all about strong C++ with newest technique of templates, stl, template meta programming.

Windows 8 WinRT and APIs are in System32 and all is built native. Every C++ compiler on the Windows platform can call these DLLs.
C++/CX is a new langage and JS is also part of the bundle to make this attractive to "others" developers. Real developers/sowftares will still be in/with C++ guys.

Check channel9 video about the real story for C++/CX:
http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-3-The-CCX-Episode-with-Marian-Luparu
Check channel9 video about the WRL story:
http://channel9.msdn.com/Shows/C9-GoingNative/GoingNative-2-C-at-BUILD-Windows-Runtime-LibraryWRL-Meet-Tarek-and-Sridhar

Hope it helps and it is only my opinion and because I love C++.
ChristopheP | www.christophep.com
Comment 28 Brian R. Bondy [:bbondy] 2012-03-16 12:27:52 PDT
Created attachment 606675 [details] [diff] [review]
Connecting thebes to metro

Metro and thebes connected.
I'm not attaching all pushes to this ticket, but just for reference if someone wants to look at it.
Comment 29 Jim Mathies [:jimm] 2012-03-16 14:22:02 PDT
Created attachment 606733 [details] [diff] [review]
experimental xre changes
Comment 30 Masatoshi Kimura [:emk] 2012-03-16 23:28:06 PDT
Comment on attachment 605186 [details] [diff] [review]
wifi code header fixes

> +#ifndef MOZ_WIN8METRO
> +#include <ntddndis.h>
> +#include <winioctl.h>
> +#endif
Are these headers even required? No one refer APIs in those headers after bug 600235.
Comment 31 Brian R. Bondy [:bbondy] 2012-03-17 06:51:55 PDT
I added in the current resource files to elm from Bug 735008 and removed the unused IE related images.  To see the splashscreen, tile, and zoomed out tile in action, just make sure you have the visual elements XML in your binary root and that you have the elm/metro/images subfolder copied in as well.  Also make sure you do NOT have a PRI file in that directory for now.   These are not the final resource but it's really cool to see them in action.
Comment 32 Ian Melven :imelven 2012-03-17 14:59:31 PDT
Just a note : according to an article Asa posted, "Metro-style browsers, including Internet Explorer 10, are 64-bit by default, and Internet Explorer 10 opts in to using HEASLR (High Entropy Address Space Layout Randomization)"

we should opt into HEASLR for Metro Firefox using the linker flag /HIGHENTROPYVA :)
Comment 33 Brian R. Bondy [:bbondy] 2012-03-17 15:24:40 PDT
Thanks for the info, by the way we do have it working with wow64.
Comment 34 Brian R. Bondy [:bbondy] 2012-03-17 16:12:23 PDT
Created attachment 606911 [details] [diff] [review]
Support for metro snap

Added support for Metro snap by adding code to thebes and cairo to resize the DX backbuffer and re-create the rendering target and surface.
Comment 35 Jim Mathies [:jimm] 2012-03-19 04:53:16 PDT
So I'm currently crashing in gfx code, the reason is that my GfxInfo rejects the hardware (I'm running in a vm) so the render mode is set to RENDER_GDI. In the call into cairo_d2d_surface_create_for_winrt_window cairo_device is null, hence the crash.

We'll need to make a choice here, either we force D2D on no matter what, which risks all sorts of rendering anomalies on hardware we've blacklisted, or we'll need some sort of a fallback rendering mechanism via GDI that paints via GDI and copies that over to the d2d surface metro uses for the view.
Comment 36 Martin Anderson 2012-03-19 05:13:59 PDT
(In reply to Jim Mathies [:jimm] from comment #35)
> So I'm currently crashing in gfx code, the reason is that my GfxInfo rejects
> the hardware (I'm running in a vm) so the render mode is set to RENDER_GDI.
> In the call into cairo_d2d_surface_create_for_winrt_window cairo_device is
> null, hence the crash.
> 
> We'll need to make a choice here, either we force D2D on no matter what,
> which risks all sorts of rendering anomalies on hardware we've blacklisted,
> or we'll need some sort of a fallback rendering mechanism via GDI that
> paints via GDI and copies that over to the d2d surface metro uses for the
> view.

Surely if the device supports Metro, it will run the Direct X surface in the metro app?  It is only Desktop which will need to support GDI rendering as fallback?
Comment 37 Jim Mathies [:jimm] 2012-03-19 07:55:13 PDT
(In reply to Martin Anderson from comment #36)
> Surely if the device supports Metro, it will run the Direct X surface in the
> metro app?  It is only Desktop which will need to support GDI rendering as
> fallback?

In a perfect world all graphics card drivers are bug free. We don't live in a perfect world. :)
Comment 38 :Ehsan Akhgari (busy, don't ask for review please) 2012-03-19 08:05:34 PDT
CCing Jeff who is the expert in this stuff.
Comment 39 Jim Mathies [:jimm] 2012-03-19 13:45:37 PDT
Good news - I've got fennec loading and displaying now in the interface. I'll check in changes in to elm later to today.
Comment 40 Brian R. Bondy [:bbondy] 2012-03-19 14:00:14 PDT
Awesome!
Comment 41 Jim Mathies [:jimm] 2012-03-20 14:55:56 PDT
Just checked in some work that gets things somewhat usable in that you can now enter urls and navigate to sites which load properly. I've also disabled plugins (which crash), remote tabs(!) which were on by default and crash, and a few other tweaks to settings.

There's an annoying bug somewhere in the mobile js interface which prevents freshly created tabs from navigating. I think it has something to do with remote tabs being disabled. So if you're playing with this navigate from about:home, which will open a new tab for you which gets around the issue.
Comment 42 Jim Mathies [:jimm] 2012-03-21 11:50:06 PDT
closing this out, the original task is complete.
Comment 43 Yuhong Bao 2012-05-09 21:36:17 PDT
"Surely if the device supports Metro, it will run the Direct X surface in the metro app?  It is only Desktop which will need to support GDI rendering as fallback?"
AFAIK, Windows 8 will have built-in DX9 software emulation for the basic framebuffer (MSBDD) driver and will not support XPDM drivers.

Note You need to log in before you can comment on or make changes to this bug.