1759537 - PiP window size changes unexpectedly depending on the aspect ratio of the videos

Status: NEW

(Toolkit :: Picture-in-Picture, defect)

Description

[Daniel Bodea [:danibodea]]

Note

• When the user launches PiP on videos with different aspect ratios, he will notice that the size of the PiP is unexpectedly small or large.
• It appears that the PiP keeps its height as its fixed PiP size, not depending on the video aspect ratio. I suggest it should somehow take into consideration the aspect ratio of the video in order to avoid these situations.

Affected versions

• Nightly v100.0a1

Affected platforms

• all

Steps to reproduce
scenario1:
load 4:3 PiP ("width": "640", "height": "360")
load 21:9 PiP ("width": "1280", "height": "360")
-> too large

scenario2:
load a 21:9 PiP ("width": "640", "height": "180") kinda small
load a portrait PiP ("136", "height": "180")
-> MUCH too small

scenario3:
load a 1:1 PiP ("width": "640", "height": "640") -kinda big
load a 21:9 PiP ("width": "2276", "height": "640")
-> does not fit on screen; spills out.

scenario4:
load a 21:9 PiP ("width": "640", "height": "180") kinda small
load a 1:1 PiP ("width": "180", "height": "180")
-> MUCH too small

Expected result

• The PiP window should have approximately the same size when launching it on videos with different aspect ratios.

Actual result

• The size of the PiP window can have unexpected sizes.

Regression range

• Not a recent regression.

Additional notes

• Test YT videos for different aspect ratios:

4:3 https://www.youtube.com/watch?v=kaUemcqIQ-k

close to 4:3 https://www.youtube.com/watch?v=Eh0Cx9yWUO0

close to 4:3 https://www.youtube.com/watch?v=cULQhvuq1Zc

1:1 https://www.youtube.com/watch?v=L0wusEVceek

portrait https://www.youtube.com/watch?v=-MkClg7XgRI

superwide https://www.youtube.com/watch?v=wo2HTSxcvpU

1.85:1 https://www.youtube.com/watch?v=hKpfxu0Y9Yo

2.39:1 https://www.youtube.com/watch?v=tkAEBBaYZ2Q

portrait https://www.youtube.com/watch?v=wTLIpIQghcM&list=PLj6XzcqwRpN73wfRp1w7j_PK2OLrbsR-A

Comment 1

[Molly Howell (she/her) (mostly inactive)]

Can I ask: what would you have expected should happen when switching from a 1:1 video to a 21:9 video? It's kind of hard for me to see what "approximately the same size" would even mean in that context.

That said, preventing our automatic resizing from exceeding the display dimensions does seem like an unambiguous improvement.

Comment 2

[kpatenio]

Attachment: 4-3__superwide__1-1__portrait.mov

I recorded how the pip window currently acts when opening it for several video dimensions (just so we know how it looks). Here is a recording of opening the pip window from 4:3 -> superwide -> 1:1 -> portrait.

Comment 3

[kpatenio]

Attachment: superwide__21-9__1-1.mov

Here is superwide -> 21:9 -> 1:1. Notice the size difference compared to starting with 4:3 video.

Comment 4

[kpatenio]

Attachment: portrait__superwide_1-1.mov

And finally, from portrait -> superwide -> 1:1.

---

In fitToScreen(), when not using a new default position + size, we try to check the previously loaded pip window location and size. It seems we try to read the previously loaded pip dimensions, and then base our new pip dimensions from that, even if the new video is a completely different aspect ratio.

In the case of 4:3 -> superwide, we keep the pip window height saved from the 4:3 video, and instead adjust the width so that we respect the 4:3 dimensions. Hence this explains why the superwide video's pip window is big.

Comment 5

[Daniel Bodea [:danibodea]]

In response to comment 1: I would expect it to calculate more appropriate sizes even in these weird scenarios and have a more constant behavior.

Comment 6

[Molly Howell (she/her) (mostly inactive)]

Well, that's actually why I asked, I'm not sure what would be appropriate. It seems like it would get tricky to work out all the different scenarios that can happen, while keeping reasonable behavior in all of them.

Comment 7

[Daniel Bodea [:danibodea]]

Based on Katherine Patenio's response in my PiP QA Questions document:
"That's the right idea. So we actually set max values for width and height when it comes to the default pip window size (i.e. if there is no data about a previously saved pip window location and/or size). Those max values are 1/4 of screen height and 1/3 of screen width.

For the most part, we set the pip window size according to whichever dimension is largest (ex. if vertical video, height is the largest). We set the largest dimension to the imposed max value, then we adjust the size of the smaller dimension such that the original aspect ratio is respected.

Oh another thing probably worth mentioning: the pip window size and location on the screen are preserved once resized (I think it's per profile? Not exactly sure atm). So it's possible that the pip window size will be bigger or smaller the next time we try to toggle pip."

A personal suggestion would be to create the PiP window based on previously set MIN and MAX values like in the case when there is no data about a previously saved PiP window location/size. The PiP window should not open as too small or too large. They should be around the desired size in ratio with the screen's resolution; i.e. about 1/3 of the screen's width and about 1/4 of its height; If I am to suggest further, I'd say that the PiP's width should be between 1/2 and 1/4 of screen resolution and the height should be between 1/3 and 1/5 of its height.
All this, only in the case when it hasn't been resized manually before. A UX developer should have a more detailed opinion on this.

Considering my lack of technical knowledge relevant to development, this is probably my best opinion.