1843786 - Some Pipewire webcam / camera improvements

Status: RESOLVED FIXED

(Core :: WebRTC, enhancement)

Description

[Robert Mader [:rmader]]

Some improvements that came up during testing. Webrtc changes will also be proposed upstream.

Comment 1

[Robert Mader [:rmader]]

Attachment: Bug 1843786 - webrtc: Filter out non-camera nodes, r=grulja,@webrtc-reviewers

Comment 2

[Robert Mader [:rmader]]

Attachment: Bug 1843786 - Pipewire: Add videotransform meta, r=grulja,@webrtc-reviewers

Depends on D183650

Comment 3

[Robert Mader [:rmader]]

Attachment: Bug 1843786 - webrtc: Implement Pipewire camera rotation support, r=grulja,@webrtc-reviewers

Depends on D183651

Comment 4

[Robert Mader [:rmader]]

Context: https://fosstodon.org/@capyloon/110731752382326647

Comment 5

[[:fabrice] Fabrice Desré]

@rmader, with this patch set there is one remaining issue: the width and height are swapped when getting the webrtc track settings.

For instance I get that on the PPP:

{
  "deviceId":"6meR3wfblEzfaZwr5+FBRA2WVOtGrpgycgPioTFj/rg=",
  "frameRate":0,
  "groupId":"tImRBoCnx0o/1WCWi8RsmONAJzk0alUkawBsrp6GcyI=",
  "height":480,
  "width":640
}

Comment 6

[Robert Mader [:rmader]]

(In reply to [:fabrice] Fabrice Desré from comment #5)

@rmader, with this patch set there is one remaining issue: the width and height are swapped when getting the webrtc track settings.

Thanks - this might be an upstream bug in the rotation support but didn't dig yet.

Comment 7

[Andreas Pehrson [:pehrsons]]

(In reply to [:fabrice] Fabrice Desré from comment #5)

@rmader, with this patch set there is one remaining issue: the width and height are swapped when getting the webrtc track settings.

What are the dimensions of the rendered frames in this case?

The spec says

On systems where it's desirable to sometimes automatically flip the X and Y axis of the resulting captured video in response to ongoing environmental factors, the width, height and aspectRatio constraints and capabilities MUST remain unaffected in all algorithms and be considered in the primary orientation only, except for the getSettings() algorithm where settings for these constrainable properties MUST be flipped if necessary to match the returned dimensions of the captured video at any point in time.

and looking at code we seem to match that.

Comment 8

[[:fabrice] Fabrice Desré]

(In reply to Andreas Pehrson [:pehrsons] from comment #7)

(In reply to [:fabrice] Fabrice Desré from comment #5)

@rmader, with this patch set there is one remaining issue: the width and height are swapped when getting the webrtc track settings.

What are the dimensions of the rendered frames in this case?

They were 640x480 iirc. I would have to back out the patches from my fork to verify.

The spec says

On systems where it's desirable to sometimes automatically flip the X and Y axis of the resulting captured video in response to ongoing environmental factors, the width, height and aspectRatio constraints and capabilities MUST remain unaffected in all algorithms and be considered in the primary orientation only, except for the getSettings() algorithm where settings for these constrainable properties MUST be flipped if necessary to match the returned dimensions of the captured video at any point in time.

and looking at code we seem to match that.

Right, I think Robert's patch in https://phabricator.services.mozilla.com/D183652 addresses correctly the problem.

Comment 9

[Andreas Pehrson [:pehrsons]]

Great, thanks for confirming. Robert, are you pushing these patches upstream, or can I help in some way?

Comment 10

[Robert Mader [:rmader]]

(In reply to Andreas Pehrson [:pehrsons] from comment #9)

Great, thanks for confirming. Robert, are you pushing these patches upstream, or can I help in some way?

Yep, will do now.

(In reply to [:fabrice] Fabrice Desré from comment #5)

@rmader, with this patch set there is one remaining issue: the width and height are swapped when getting the webrtc track settings.

I somewhat suspect this to have to do with https://searchfox.org/mozilla-central/source/third_party/libwebrtc/modules/video_capture/video_capture_impl.cc#249-254 - we don't use it all all, but Chromium probably does? Anyhow, will submit the changes as is and wait for feedback.

Comment 11

[Robert Mader [:rmader]]

Opened

• https://webrtc-review.googlesource.com/c/src/+/318641
• https://webrtc-review.googlesource.com/c/src/+/318642

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4c2505f53328
webrtc: Filter out non-camera nodes, r=webrtc-reviewers,pehrsons,@webrtc-reviewers
https://hg.mozilla.org/integration/autoland/rev/214fd339fda5
Pipewire: Add videotransform meta, r=grulja,@webrtc-reviewers
https://hg.mozilla.org/integration/autoland/rev/1db7a958b945
webrtc: Implement Pipewire camera rotation support, r=webrtc-reviewers,pehrsons,@webrtc-reviewers

Comment 13

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/4c2505f53328
https://hg.mozilla.org/mozilla-central/rev/214fd339fda5
https://hg.mozilla.org/mozilla-central/rev/1db7a958b945