686835 - Windows 8 / Metro support

Status: RESOLVED FIXED

(Core :: General, defect)

Description

[Brian R. Bondy [:bbondy]]

It would be good for planning purposes for us to determine if we can access XPCOM objects from within a metro style application.  Or perhaps someone already knows? 

I haven't looked at it deeply but I think that Metro applications can be developed from their JS apps, C#, VB.NET and managed C++ but not native C++.

Comment 1

[Brian R. Bondy [:bbondy]]

> Core WinRT is NOT managed - it is native and COM-based. Our language provides a
> fully-native projection of that in a high-level syntax, but you aren't tied to
> that syntax - you will be able to target the Windows Runtime using low-level 
> COM or WRL (an ATL-like template library) as well. (The C#/VB projection is 
> a different story, they generate runtime-callable wrapper objects which 
> marshal between .NET and WinRT.)

Comment 2

[Brian R. Bondy [:bbondy]]

I don't think we can simply adapt a program to be both a desktop application and a metro application. So I think we'll have to proxy the desktop interface to the tablet interface or something along those lines.

Comment 3

[Brian R. Bondy [:bbondy]]

Attachment: Metro/Deskotp app API diagram

Comment 4

[Jim Mathies [:jimm]]

There's a great deal of data available on msdn about the winrt framework, com use, and other areas you can access via these apps.

http://msdn.microsoft.com/en-us/library/windows/apps/br211369%28v=VS.85%29.aspx

Generally, they are heavily restricting access to components and apis. If it's not sanctioned by the runtime you can't access it.

Another aspect of this that's interesting, they've migrated to async apis for functionality that would normally block, like file access, file pickers, etc.. Sync calls on the ui thread are not allowed at all.

Comment 5

[Brian R. Bondy [:bbondy]]

Jim do you know off hand if you can do things like create a named pipe from a metro app to a desktop app?

Comment 6

[Jim Mathies [:jimm]]

(In reply to Brian R. Bondy [:bbondy] from comment #5)
> Jim do you know off hand if you can do things like create a named pipe from
> a metro app to a desktop app?

Looks like communication between the two might be possible. The full spec on the .net framework metro apps have access to is up on msdn via that link. I haven't read any specifics on what happens to desktop apps when your in metro though - I have to wonder if these get frozen when you're not in that new desktop shell.

Comment 7

[Brian R. Bondy [:bbondy]]

I know when you go from a metro app into a desktop app, all of the metro apps get suspended.  So I assume the reverse also happens.  That being said we could always have a service that we make that manages a new session by the same user which simply has a full screen app that we relay to the metro app in the other session.  

Pipe communication is possible across sessions.  But this is an amazing amount of work just to get a metro app :)  There is probably a better way.  I hope. Also I'm not sure of the responsiveness for new things like WebGL.

Comment 8

[Brian R. Bondy [:bbondy]]

From what I can tell:
- Both the metro interface and old interface run inside session 1 (session 0 is as before used for services)
- Winsta0 is the windows station that is used since it presents a GUI for both interfaces.
- I can only enumerate 2 desktops via EnumDesktops when running as administrator so it seems that it shares the "Default" Desktop as well.

Comment 9

[Brian R. Bondy [:bbondy]]

Sorry I meant to mention that the other Desktop is the winlogon desktop.  There is probably also the Secure Desktop for UAC if that option is enabled.

Comment 10

[Jim Mathies [:jimm]]

Spoke to bsmedberg about this a bit - fennec may be suited to doing the dual process thing if we can work out issues surrounding what we want to send over the communications pipe and things like process creation / lifetime. The metro app would be responsible for display through the direct 2d apis it has access to and everything else would run in another process. We should probably flesh out the communications side of things and summarize what can be done in metro, post that to a wiki, and then kick off a discussion on dev.planning to discuss further. Down the road we could have a planning session at a future onsite.

Comment 11

[Jim Mathies [:jimm]]

Interesting notes on how they enforce restrictions on linked win32 calls and com creation:

http://social.msdn.microsoft.com/Forums/en-US/winappswithnativecode/thread/9400e82e-c68c-4641-9ece-543bd667cbe6

Comment 12

[Brian R. Bondy [:bbondy]]

Maybe we can improve transitions enough to not even need a metro app.

Comment 13

[Brian R. Bondy [:bbondy]]

So I tried going full screen and then pressing the windows key to go to metro interface. It is actually pretty smooth back and forth.   I wonder if we can force a full screen when launched from the metro interface and exit into the metro interface via a programmtic switch.

I don't want to discount any of the strategy talks about multi process above, just a different possible direction we can take.

Comment 14

[Brian R. Bondy [:bbondy]]

If anyone on this thread knows of advantages of metro style apps not in normal apps please let us know.

Comment 15

[Peter Henkel [:Terepin]]

Tablets.

Comment 16

[Brian R. Bondy [:bbondy]]

From what I've read tablets will have full access to the Desktop environment as well. Even for ARM processors I think there will still be the Desktop environment, but there will almost for sure be a lot of work for that.  Initial ARM bugs see Bug 687033 and Bug 686843.

So specifically I'm looking for things that are better in a metro app, over tricking someone into thinking they are using a metro app (which would be just a Desktop app running full screen).

See:
http://www.engadget.com/2011/09/13/windows-8-for-tablets-hands-on-preview/

Comment 17

[Peter Henkel [:Terepin]]

What I meant was that Metro was created for touchscreens. So the question here isn't if Metro is better or worse than Aero, but how to use it. Take IE10 as an example.

Comment 18

[Brian R. Bondy [:bbondy]]

Do you have a link off hand to bring us up to speed for which ways the touch is better in Metro from what we have?  Or in particular a link showing there will be a lack of touch support in the Desktop interface?

I was hoping to have a tiny metro app which would scan for a FF (or Fennec) process in desktop mode and then unsuspend it, move it to full screen and then do the transition.  Hence the user not seeing the Desktop mode at all.  If one wasn't there it would launch a new one in full screen and then do the transition.  I think this would be an innexpensive way to get it done.

But as you said if there is no touch support in the desktop mode (?) then we need to find a more complex solution.

Comment 19

[Peter Henkel [:Terepin]]

I didn't say there isn't touch support in desktop. All I'm saying is Metro was created for better user touch experience than Aero can provide.

Comment 20

[Paul [pwd]]

Given that Windows 7 supports multi-touch, I'd suspect that the classic interface of Windows 8 will also support multi-touch. http://windows.microsoft.com/en-US/windows7/products/features/touch

I'd hazard a guess that this is a question better posed at the UX team. Microsoft are trying to reimagine the way users integrate with their systems using away from the Windowed approach to a more linear one. I'd be surprised if this hasn't been discussed by the UX team as it came up in a bug where by Dao wanted to make the buttons more touch friendly. Sadly I can't find the bug number.

Comment 21

[Jim Mathies [:jimm]]

There seems to be some confusion here. We are not saying that the "desktop mode" of firefox (chrome, titlebar, command buttons, etc) is the preferred way we want users to surf the web on Windows 8. Quite the contrary - we are looking at ways to integrate into both metro and the desktop shell.

Apps that run in the metro environment are built against a framework that gecko may not be compatible with. So Firefox or Fennec (as we know them, a single app running in the desktop environment) may not run directly in metro as a "metro app". To address this we are looking at ways to configure fx so that it appears it is a metro app even though it might be in the desktop shell. We are also kicking around some more advanced ideas that would place a fennec process directly in the metro ui.
 
This is all just speculation at this point.

Comment 22

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Porting Gecko to run on WinRT is probably an option if faking desktop FF as a Metro app is undesirable.

Comment 23

[(no longer active)]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #22)
> Porting Gecko to run on WinRT is probably an option if faking desktop FF as
> a Metro app is undesirable.

Is that feasible though, with things like WinRT not allowing sync I/O for example?

Comment 24

[Brian R. Bondy [:bbondy]]

> Porting Gecko to run on WinRT is probably an option...

I think a complete port is undesirable and there's a very good chance not even possible because of various restrictions. I think a great idea if we can't fake a Metro app is Jim's Comment 10.

Comment 25

[Jim Mathies [:jimm]]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #22)
> Porting Gecko to run on WinRT is probably an option if faking desktop FF as
> a Metro app is undesirable.

Roc, you might take a look at the api restrictions via the link in comment 4, I'd be interested in your thoughts. I don't see support for what I'd consider key apis - no std lib support, no wsa socket calls which I think nspr uses, plus a lack of support for a number of win32 api calls we have spread around. A gecko port might be a pretty big undertaking.

Comment 26

[Brian R. Bondy [:bbondy]]

So I read up a bit more on several things on Windows 8.

I know we can fake the transitions into a metro app as I mentioned in Comment 12, but we'd be missing out on a lot of important things.  I think we'll have to do the more complex solution of actually having a metro app which uses electrolysis.

I'll be posting all of the below into different bugs, but here is a rundown:

A big part of Metro apps is having a consistent standard UI.  This means that when you go to the left and right edge of the screen, the system controls come up (not something we can control).
The left side is used for going back, the right edge is used for charms (search, share, start, devices, settings).

The top and bottom edge should be used for things specific to our application like add tab.

Metro apps must also be able to support a snap state, so you can have another metro app next to it.

Metro apps can also consume or provide for the charms in the right edge system menu (contracts).
Things like sharing with other apps, using the picker charm to select things from other apps, ...

Metro apps also have popup notifications when you're in a different metro app.

Touch in Metro is not about what's possible but about consistency with the rest of the Metro interface.  There are 7 standard gestures for touch in Windows 8.  
I'll post details into the touch ticket.

Providing common controls and common animations is probably also desirable.

So in short, there is a lot to gain from having a metro interface, probably more than I isted above, but it is going to be a ton of work.

Comment 27

[Paul [pwd]]

(In reply to Brian R. Bondy [:bbondy] from comment #26)
This doesn't appear to leave much room for UI customisation. I guess what I'm asking is, could we not do what we do on Fennec (phone version) whereby we drag a page right for the tab bar and left for the navigation controls while still maintaining the overall metro experience?

Comment 28

[Brian R. Bondy [:bbondy]]

I have not tried a touch interface for it yet, but a tab bar would probably be possible as its only the edge to popup the system panels.  It is something the metro guidelines want on top or bottom though.  Metro programs are simple and consistent by definition.

Comment 29

[Robert O'Callahan (:roc) (email my personal email if necessary)]

(In reply to Jim Mathies [:jimm] from comment #25)
> Roc, you might take a look at the api restrictions via the link in comment
> 4, I'd be interested in your thoughts. I don't see support for what I'd
> consider key apis - no std lib support, no wsa socket calls which I think
> nspr uses, plus a lack of support for a number of win32 api calls we have
> spread around. A gecko port might be a pretty big undertaking.

Yeah, it would be; it would effectively be a new platform. TCP sockets are there in WinRT (Windows.Networking.Sockets). Sync I/O could probably be faked on top of async I/O. Not saying a WinRT port is the right approach though...

Comment 30

[Bas Schouten (:bas.schouten)]

Although I think a WinRT port -may- be the right approach long term. There's a lot of trouble with that (i.e. as roc says, it looks like a whole new platform), it looks nicer than Win32 though on a quick look!

A very simple Metro-style front-end app which shares a DXGI surface with a normal firefox back-end that pipes over any communication commands might indeed work though? Although there may be less obvious components in the system preventing that.

Comment 31

[Robert O'Callahan (:roc) (email my personal email if necessary)]

With the Metro front-end/Win32 back-end approach, there's the question of whether you use a completely Metro-native front-end, or actually pull over a XUL-rendered UI window from the Win32 process.

Comment 32

[Brian R. Bondy [:bbondy]]

> With the Metro front-end/Win32 back-end approach, there's the question of whether you use a completely Metro-native front-end, or actually pull over a XUL-rendered UI window from the Win32 process.

That's one of my concerns as well.

Comment 33

[Jim Mathies [:jimm]]

I'll admit I'm enthused by the idea of creating a metro port of the whole project. But it adds the overhead of maintaining two separate windows platforms where today we only have one. Getting away from win32 if in fact this is the direction windows is headed is intriguing.

I'm wondering about ARM - from what I understand, a metro port would allow us to run on new Windows based ARM tablets. If we limited ourselves to  win32 background process using win32 apis, don't we also give up ARM support?

Comment 34

[Emanuel Hoogeveen [:ehoogeveen]]

I believe so. From what I've heard, Win32 will continue to be available side-by-side with Metro on the desktop version of Windows 8, but only Metro will work on ARM.

Comment 35

[Brian R. Bondy [:bbondy]]

I think there's a misconception about ARM being metro only.  Microsoft said that x86 apps will not run on ARM.  This doesn't mean a Win API (Win32) Desktop environment will not be provided.  

Instead I think it just means that they will not offer something like WOW64 which they had for running x86 apps on x64 architectures.  

This means we will have to do ARM support for our desktop version independent of this ticket/task of a metro port.

Comment 36

[Brian R. Bondy [:bbondy]]

Here's a reference from PCWorld re ARM Desktop support:
http://www.pcworld.com/article/240125/windows_8_on_arm_processors_three_working_tablet_demos.html

> Yes, Ye Olde Windows Explorer was there, as was the Windows Desktop.

Comment 37

[Jim Mathies [:jimm]]

Found this - inter-process communication with a metro front end was brought up at BUILD:

http://stackoverflow.com/questions/7465517/how-can-a-metro-app-in-windows-8-communicate-with-a-backend-desktop-app-on-the-sa

Comment 38

[Jim Mathies [:jimm]]

http://channel9.msdn.com/Events/BUILD/BUILD2011/TOOL-789C

worth watching!

Comment 39

[Jim Mathies [:jimm]]

It sounds like the new metro container is pretty heavily locked down, cross process communication between metro and the desktop may not be supported.

Comment 40

[Brian R. Bondy [:bbondy]]

I'm guessing we could get around anything with a service running as the local system account but the more I hear about it the more I think we'll need a full port.

Comment 41

[Bas Schouten (:bas.schouten)]

From what I understood Win32 will be supported on ARM. That would make sense as well, it's just one of the subsystem and since the kernel works on ARM, I don't see why Win32 would not, unless they just -chose- to restrict it.

Comment 42

[Brian R. Bondy [:bbondy]]

> It sounds like the new metro container is pretty heavily locked down, cross
> process communication between metro and the desktop may not be supported.

They talk about communication across boundaries of Metro / Desktop in the video on Comment 38: 01:03:00 for a few minutes.  They seem to think it's a really bad idea to even try it.

Comment 43

[Bas Schouten (:bas.schouten)]

(In reply to Brian R. Bondy [:bbondy] from comment #40)
> I'm guessing we could get around anything with a service running as the
> local system account but the more I hear about it the more I think we'll
> need a full port.

I share that impression. Fwiw I've played with Windows 8 now on an x86 tablet I have and the IE10 metro integration is very smooth.

Comment 44

[Sav]

The IE metro app is a separate app from IE, and just embeds an IE frame in it. I'm not too familiar with gecko, but if it is able to be embedded as a frame, that's the way forward. Just have a simple app coded in C# or something, some fancy UI like IE10, and just package them together.
Obviously, addons won't work, but they don't in IE10, and it's just for casual browsing.

Comment 45

[Josh Tumath]

(In reply to Sav from comment #44)
> Obviously, addons won't work, but they don't in IE10, and it's just for
> casual browsing.

Actually, the Metro-style apps aren't designed just for casual browsing. It's expected that in the future, we will rarely ever need to go on the desktop, because it's only there for legacy apps.

A frame could be used initially, but -- going forward -- we will need to support add-ons and whatnot.

Comment 46

[Peter Henkel [:Terepin]]

(In reply to Sav from comment #44)
> and it's just for casual browsing.

No, it's meant as a fully featured web browser that relies only on HTML5. That's a difference.

Comment 47

[Bas Schouten (:bas.schouten)]

As far as we know a front-end/back-end combination also isn't going to work. Metro Apps are pretty tightly sandboxed, even through sockets for example you're not allowed to connect to localhost.

Comment 48

[Bas Schouten (:bas.schouten)]

Based on the wiki updates I'd like to note in the graphics department we're actually not in that bad of a state. Our rendering combination D3D10/D2D is pretty good for metro, however currently we're still blacklisting some cards and using GDI there. When I update Azure to D2D 1.1 I'll keep in mind we might need Direct2D software rendering support.

Comment 49

[fbrfa8.g]

https://wiki.mozilla.org/Firefox/Windows_8_Integration#Desktop
On this topic:
The start screen is the start menu.

Performing operations that would add an item to the start menu in W7 adds a tile to the start screen in W8.

Not having a tile is as simple as not creating a "start menu" shortcut.
(do not create a shortcut in C:\ProgramData\Microsoft\Windows\Start Menu\Programs)

Comment 50

[Brian R. Bondy [:bbondy]]

We're done with the elm branch now, I removed the info from the disposable branches wiki page.

Comment 51

[Benjamin Smedberg]

Will the people on the elm nightlies be updated to mozilla-central?

Comment 52

[Brian R. Bondy [:bbondy]]

That will be tracked in bug 840541, but I guess I should keep it checked out until we decide for sure if we'll do that task or not.  I'll revert that wikip age to re-check it out.

Comment 53

[Brian R. Bondy [:bbondy]]

If it's possible to serve a MAR without the branch we wouldn't need the branch anymore.