1687562 - Create API for Windows 10 (v1903+) Firefox to pin itself to taskbar

Status: RESOLVED FIXED

(Firefox :: Shell Integration, enhancement, P1)

Description

[Ed Lee :Mardak]

I found bug 1162562 for converting Firefox to UWP after mhowell's bug 1686343 comment 3:

I don't think the TaskbarManager interface is available to Win32 apps… I assume that means it can only be called from within a UWP app.

It doesn't seem easy, so trying to move forward without that dependency, is there a way to get specifically the pin-itself-to-taskbar functionality? I estimate most of our users are on Windows 10 a version newer than 2018, and having Firefox pinned should be key in improving momentum/days-of-use.

To be clear, this would be an API used from Firefox potentially as part of onboarding bug 1686343 with explicit user interaction as opposed to previous implementations where the windows installer would automatically pin.

mhowell, are there alternatives to Firefox-as-UWP, so we can experiment and get data sooner?

Comment 1

[Molly Howell (she/her) (no longer active)]

(In reply to Ed Lee :Mardak from comment #0)

[...] is there a way to get specifically the pin-itself-to-taskbar functionality?

Well, no, because that's one of the features of that interface; if we can't even get the interface, we can't call the API that's part of the interface.

To be clear, this would be an API used from Firefox potentially as part of onboarding bug 1686343 with explicit user interaction as opposed to previous implementations where the windows installer would automatically pin.

mhowell, are there alternatives to Firefox-as-UWP, so we can experiment and get data sooner?

It's kind of the same thing I was talking about with default browser, where the OS can't really tell what we're trying to do and it feels strongly that this is the user's decision, so there's not a lot available to us to get around that. And if we try, then we're doing something that the operating system not just doesn't support but has repeatedly put up roadblocks to prevent, and that's not a game I can personally support playing.

Comment 2

[(No longer employed by Mozilla) Aaron Klotz]

(In reply to Ed Lee :Mardak from comment #0)

I found bug 1162562 for converting Firefox to UWP after mhowell's bug 1686343 comment 3:

I don't think the TaskbarManager interface is available to Win32 apps… I assume that means it can only be called from within a UWP app.

Many UWP APIs are now available to standard C++ applications via the C++/WinRT language projection. Are we sure this cannot be called using that?

Comment 3

[Molly Howell (she/her) (no longer active)]

As I explained in the linked comment, that is exactly how I tried to call it.

Comment 4

[Masatoshi Kimura [:emk]]

FYI: https://docs.microsoft.com/en-us/windows/apps/desktop/modernize/grant-identity-to-nonpackaged-apps
Desktop Win32 apps need a package identity to make more UWP APIs available.

Comment 5

[Molly Howell (she/her) (no longer active)]

I wasn't aware of that sparse package feature, thank you for pointing that out. That is something that we could do, however I don't see a specific indication that it would enable the TaskbarManager API, so we wound need to test that that is actually the case.

Comment 6

[Ed Lee :Mardak]

Looking at the example of registering the sparse package, looks like main() checks if it's running with identity, and if not calls PackageManager.AddPackageByUriAsync() then restart with identity.

Notably, PackageManager class has [Windows.Foundation.Metadata.DualApiPartition(version=1)] attribute while TaskbarManager doesn't. Is that distinction what allows calling via C++/WinRT language projection? widget/windows/OSKInputPaneManager.cpp uses InputPane, widget/windows/ToastNotificationHandler.h uses ToastNotifier, and widget/windows/WindowsUIUtils.cpp uses UIViewSettings -- all of those classes indeed have DualApiPartitionAttribute.

Is there something I should be checking to see if TaskbarManager.RequestPinCurrentAppAsync() requires package identity which could indicate that registering a sparse package would then work for Firefox?

Comment 7

[Adam Gashlin (he/him) [:agashlin] (ex-moco)]

I tried using TaskbarManager from within Microsoft's SparsePackages sample, figuring that I should start from a working example. I'm able to get a TaskbarManager with TaskbarManager.GetDefault() once the app has Identity, but IsSupported is false (which shouldn't be the case on Desktop), IsPinningAllowed is false (which should only happen with an explicit policy), and RequestPinCurrentAppAsync() fails with 0x80040155 (interface not registered). I was able to manually pin the app and run it from the pin, though the pin is invisible (could this be a clue?), but even then IsCurrentAppPinnedAsync() returned false.

So It looks like we won't have the appropriate access even with a sparse package, unless I'm overlooking something. I cross checked with the sample UWP app, which does properly prompt, it doesn't look like there's anything special that needs to go into the manifest. A few things I tried were changing the TargetDeviceFamily from Windows.Desktop to Windows.Universal, adding the ShowNameOnTiles elements and ShortName property, and setting the AUMI to match Firefox with SetCurrentProcessExplicitAppUserModelID. My fork is here if anyone wants to try adjusting it.

Comment 8

[Ed Lee :Mardak]

Attachment: RequestPinCurrentAppAsync() sample

(In reply to Adam Gashlin (he/him) [:agashlin] from comment #7)

I tried using TaskbarManager from within Microsoft's SparsePackages sample… but IsSupported is false (which shouldn't be the case on Desktop)

Thanks for trying out the sparse package. I suppose if it believes IsSupported is correctly returning false, it somewhat makes sense why the other APIs don't really "work" (assuming you skipped the early return for "Taskbar not supported").

I found a video of someone using the API and this is what it would look like (attached). So even if we get say the defaultagent working as a full UWP with Firefox passing along a user initiated event to get WDBA to respond, is this the UX we want (where we would also probably need to figure out what launching WDBA to launch Firefox would look like)? (Where the premise is converting something smaller to UWP might be easier than all of Firefox as in bug 1162562.)

Where if the user needs to click something in Firefox to then need to confirm another windows prompt, maybe asking/showing the user would be a reasonable alternative along the lines of attachment 9197495 [details] from bug 1686343 comment 4.

Comment 9

[Adam Gashlin (he/him) [:agashlin] (ex-moco)]

(In reply to Ed Lee :Mardak from comment #6)

Notably, PackageManager class has [Windows.Foundation.Metadata.DualApiPartition(version=1)] attribute while TaskbarManager doesn't. Is that distinction what allows calling via C++/WinRT language projection? widget/windows/OSKInputPaneManager.cpp uses InputPane, widget/windows/ToastNotificationHandler.h uses ToastNotifier, and widget/windows/WindowsUIUtils.cpp uses UIViewSettings -- all of those classes indeed have DualApiPartitionAttribute.

Yeah it looks like DualApiPartition is the thing to look for, according to this document, "APIs that have the DualApiPartition are supported in all desktop apps, including those with package identity and those without package identity." That doc also lists APIs supported in desktop apps with package identity, it claims to be exhaustive and does not include TaskbarManager, so I think this is as close as we can get to documentation that it is not supported.

(In reply to Ed Lee :Mardak from comment #8)

I found a video of someone using the API and this is what it would look like (attached). So even if we get say the defaultagent working as a full UWP with Firefox passing along a user initiated event to get WDBA to respond, is this the UX we want (where we would also probably need to figure out what launching WDBA to launch Firefox would look like)? (Where the premise is converting something smaller to UWP might be easier than all of Firefox as in bug 1162562.)

Where if the user needs to click something in Firefox to then need to confirm another windows prompt, maybe asking/showing the user would be a reasonable alternative along the lines of attachment 9197495 [details] from bug 1686343 comment 4.

I've experimented with a few sample apps, it doesn't look like we're going to be able to set the AUMID in a UWP app. We'd need this in order to get Windows to treat a helper app as part of the same taskbar group as Firefox. Using SetCurrentProcessExplicitAppUserModelID in the SparsePackages sample only works when it isn't running with Identity, with Identity it always appears in a different group. This doc says explicitly "You can't define custom AUMIDs." Thus if we used a standalone launcher app that could request pinning, the Firefox that launches wouldn't be grouped with the pin, cluttering the taskbar and causing confusion.

It's conceivable that we could set Firefox to use the AUMID that the package would use, Microsoft does provide a mechanism for migrating pins when installing a package. But this would required changing the ID in a lot of other places as well, especially for existing installs, and it would make multiple installs on a system indistinguishable. It might not even work in simple cases.

In conclusion, I don't think that pinning Firefox programmatically is feasible using any of the mechanisms I've learned about. Perhaps the best option is to provide a tutorial explaining how to manually pin if the user expresses interest in pinning.

Comment 10

[Ed Lee :Mardak]

From the exploration so far, it seems like the converting Firefox and/or a helper application (e.g., default browser agent) to UWP or sparse package is likely not viable especially for running experiments to decide if we should even continue down this path. https://docs.google.com/document/d/1Op2-HCNe7qtVrz5ukgRoLtYeLvo4XjpG0wqu9apKGrc/edit#heading=h.qdndw0yzycur

rachel, could you help find out what it would take to get the remaining option of macros/ui automation working for an experiment? We can scope the experiment to run with various limitations, e.g., english only or specific windows versions, so we don't need a fully robust implementation that works for all users. Then depending on the result of the experiment, we can decide if we should figure out a more robust solution or a different path.

One thing I'm trying to explore right now is if I can get Firefox to launch a file/script that Windows already handles to simulate mouse clicks or keyboard presses not too different from Firefox launching a download. This is in attempts to avoid needing new C++ code for custom functionality.

Comment 11

[Ed Lee :Mardak]

Potentially with bug 1685213, even with a fragile macro temporary implementation, we can know the user clicked a button to have Firefox pinned and that our hacky solution failed to actually pin and show a message to the user that pinning wasn't successful. We would still be able run an experiment to measure that the user intended to get Firefox pinned (as well as how fragile/robust the approach turned out to be in the wild), but actual retention measurements probably wouldn't be as accurate overall.

Comment 12

[Rachel Tublitz [:rachel]]

Oops! Didn't mean to resolve my ni? just meant to link a related issue. Re-adding the ni? and will continue collecting info here and get back to you shortly, Ed.

Comment 13

[Rachel Tublitz [:rachel]]

(In reply to Ed Lee :Mardak from comment #10)

rachel, could you help find out what it would take to get the remaining option of macros/ui automation working for an experiment?

Resolving, as we've been discussing in slack. Adam will be working on researching additional options here.

Comment 14

[Adam Gashlin (he/him) [:agashlin] (ex-moco)]

Attachment: Bug 1687562 - Part 1: Implement Taskbar pinning. r?mhowell!

Comment 15

[Tim Spurway [:tspurway]]

[Tracking Requested - why for this release]: this is required for planned Pin To Taskbar experiments planned for 87

Comment 17

[Adam Gashlin (he/him) [:agashlin] (ex-moco)]

Attachment: Bug 1687562 - Part 2: Detect whether Firefox is pinned to the taskbar. r?mhowell!

Depends on D104779

Comment 18

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/integration/autoland/rev/0dd025a6da01de3b274b9f9309fe3cdcb238b736
https://hg.mozilla.org/integration/autoland/rev/3d42785f84cb4251378b1cd3583df18fd8aa8bd9
https://hg.mozilla.org/mozilla-central/rev/0dd025a6da01
https://hg.mozilla.org/mozilla-central/rev/3d42785f84cb

Comment 19

[Ed Lee :Mardak]

Attachment: prototype jsctypes IPinnedList3

Just stashing this file on the bug if I need to reference it in the future