1773509 - Allow use of windows-rs Rust crate

Status: RESOLVED FIXED

(Firefox Build System :: General, task, P3)

Description

[Kagami Rosylight [:saschanaz] (they/them)]

(Not sure this is the right component, but I know :glandium knows well about this at least)

This is intended to be a tracking issue since I think someone will have to take a look when the existing dependencies start embracing it. There are also some interests within Gecko for:

1. new API uses since windows-rs simply has better API coverage than winapi does
2. potential migration to Rust in widget component since the modern C++/WinRT is exception-based and thus does not fit in Gecko. (Using deprecated WRL makes the code dirty.)

A known issue is that windows-rs is BIG (200 MB).

Comment 1

[Mike Hommey [:glandium]]

It's been worked around in bug 1773189, and I'll be working with upstream to attempt to reduce its size and the version churn.

Comment 2

[Kagami Rosylight [:saschanaz] (they/them)]

Thank you!

Comment 3

[Kagami Rosylight [:saschanaz] (they/them)]

Fixed by which bug? 👀

Comment 4

[Alex Hochheiden [:ahochheiden]]

(In reply to Kagami :saschanaz from comment #3)

Fixed by which bug? 👀

Based on the title's (or workaround it somehow) and Glandium's comment It's been worked around in bug 1773189 I interpreted it as it being resolved by Bug 1773189. If that's not the case, I can re-open it. Though, from the rest of Glandium's comment it seems like the rest of the work to be done is upstream (which I presume to mean the rust repo outside of Bugzilla).

Comment 5

[Kagami Rosylight [:saschanaz] (they/them)]

By "workaround" I meant more like to use it without vendoring it like we do with Windows SDK. Glandium's workaround is about not using windows-rs. I believe those are two different thing, but anyway changing the title to reduce confusion.

A fix would be allowing use of windows-rs in whatever way.

Comment 6

[Erich Gubler [:ErichDonGubler] (he/him)]

It appears that windows upstream now has a stable way to filter generated bindings via the windows-bindgen crate. Based on my understanding of :glandium's revision message in D183826 (for bug 1844001):

The windows crate, however, is still too big, and upstream is working towards making it easier to generate adhoc bindings.

…it seems that generating filtered bindings would be an acceptable solution. So, I think we might be able to make forward progress now? Of course, this assumes that we can keep the disk size of vendored WinMD metadata and generated code within tolerance.

We in the WebGPU team are blocked on the consumption of several open-source components that improve WGPU's performance considerably, windows ecosystem, particularly for the gpu-allocator crate and making our D3D12 bindings much safer. We'd be happy to work on this under :glandium's guidance, if bandwidth is the only blocker here!

Comment 7

[Mike Hommey [:glandium]]

As kind of hinted by the recent addition of a dependency and the more recent fixing of said dependency, this is actually actively under work. @nrishel has been testing a proof of concept I provided. Nick, apart from the rust-analyzer completion problem, how has it been going?

Comment 8

[Nick Rishel [:nrishel]]

:glandium I found that development was going too slow without rust-analyzer for the PoC, so it's on the back burner. I was implementing a COM object with weird constraints and had a lot of copying/pasting from the docs since I couldn't relying on autocomplete. Things were compiling as expected once I worked through build errors so I suspect vendoring a crate like gpu-allocator that (I assume) abstracts over windows might be an easier path to a PoC.

Comment 9

[Erich Gubler [:ErichDonGubler] (he/him)]

:nrishel: It should be fairly easy to use windows-bindgen to generate the necessary COM APIs that gpu-allocator uses. If you also enable the windows_rs feature for wgpu-hal in gfx/wgpu_bindings/Cargo.toml, and things compile, I'd consider that a successful PoC (for our purposes)!

Comment 10

[Kagami Rosylight [:saschanaz] (they/them)]

FWIW, I was also testing on my own PoC on top of glandium's Jan-11 windows-rs stack, things build and functions run as expected. Somehow I'm getting memory access violation crash either from:

• jemalloc through windows_core::imp::heap::heap_free
• SHCore.dll with 0xc000374 which is reportedly heap corruption error code

and I'm unsure whether that's my fault or because of some kind of conflict between jemalloc and windows-rs.

Comment 11

[Kagami Rosylight [:saschanaz] (they/them)]

Or a MOZ_DIAGNOSTIC_ASSERT(arena->mMagic == ARENA_MAGIC); failure where arena is nullptr. 🤔

Comment 12

[Nick Rishel [:nrishel]]

:ErichDonGubler can confirm it builds and runs after vendoring gpu-allocator and enabling the windows_rs feature in wgpu-hal on top of :glandium's changes.

Comment 13

[Mike Hommey [:glandium]]

Attachment: Bug 1773509 - Vendor the windows-core crate.

Comment 14

[Mike Hommey [:glandium]]

Attachment: Bug 1773509 - Add a windows-rs toolchain that provides a copy of the windows-rs crate.

The windows-rs crate is too large to reasonably vendor in a normal
fashion. So we'll use a bootstrappable toolchain.

Comment 15

[Mike Hommey [:glandium]]

Attachment: Bug 1773509 - Add an in-tree windows crate that wraps a bootstrapped windows-rs.

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3c39f2b8ee28
Vendor the windows-core crate. r=supply-chain-reviewers

Comment 17

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/3c39f2b8ee28

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3b8bc578e6b6
Add a windows-rs toolchain that provides a copy of the windows-rs crate. r=firefox-build-system-reviewers,taskgraph-reviewers,bhearsum,ahochheiden
https://hg.mozilla.org/integration/autoland/rev/baf146e5aff6
Add an in-tree windows crate that wraps a bootstrapped windows-rs. r=firefox-build-system-reviewers,supply-chain-reviewers,ahochheiden

Comment 19

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/3b8bc578e6b6
https://hg.mozilla.org/mozilla-central/rev/baf146e5aff6