1797556 - [WebExtension] browser.windows.create() hangs

Status: NEW

(WebExtensions :: General, defect, P3)

Description

[shadedblink]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:107.0) Gecko/20100101 Firefox/107.0

Steps to reproduce:

Method browser.windows.create() is used to open background window in non-visible area(outside of display bounds). Windows type is normal, state is normal.
OS: Windows 10
Not device-specific, tested on 3 different PC.
Started approximately since firefox 102-103 or earlier.

Use-case: bunch of urls are being subsequently rendered in single background window to get their preview images.

Actual results:

Physical window is created, but promise is not fulfilled unless windows is explicitly requested to render(e.g. by hovering over it on taskbar to force render its preview).

browser.windows.getAll() returns this new window successfully, it's tab url is 'about:blank'.

browser.tabs.warmup() on given tab completes successfully but makes no effect at all.

When window completed render, it is snapshoted by browser.tabs.captureVisibleTab(). First tab to be proccessed in this tab is always snapshoted successfully, while every subsequent tab, that was opened in this window, may fail browser.tabs.captureVisibleTab() with an unexpected error occured. If window is forced to render for all this time(by keeping mouse over it in taskbar as forementioned), then all snapshots are made right and no error occurs.

Expected results:

Promise should be completed the moment the window is created. Tab should begin to load requested url instantly.
browser.windows.create() and browser.tabs.captureVisibleTab() should not depend on whether window is viewed or not.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Widget: Win32' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Stephen A Pohl [:spohl]]

Could you run mozregression to see when this started happening?

A number of Firefox versions will open in succession to narrow down when this started occurring. Simply answer "good" or "bad" based on whether or not a build reproduces the bug. Once finished, please post the output from the last run. It should give a last good and first bad revision as well as a link to look at the changesets in that range. Thank you!

Comment 3

[shadedblink]

I don't see there any option to add attachment, so there are some last lines of logs from regression. Note that result may be inaccurate since there may be false-good results(like window opened right just because something else triggered render or last render focus is still not worn-off; as I said earlier, render can continue normally for some subsequently open-closed windows after first render).

2022-10-28T00:08:47.535000: INFO : b"[fluent] Couldn't find a message:"
2022-10-28T00:08:47.536000: INFO : b'[dom/l10n] Errors during l10n mutation frame.'
2022-10-28T00:08:48.858000: INFO : Narrowed integration regression window from [b7f6557a, 49a271c4] (4 builds) to [663615ef, 49a271c4] (2 builds) (~1 steps left)
2022-10-28T00:08:48.864000: DEBUG : Starting merge handling...
2022-10-28T00:08:48.864000: DEBUG : Using url: https://hg.mozilla.org/mozilla-central/json-pushes?changeset=49a271c42001f7bc5b9efadfc3e0b66dc0b67777&full=1
2022-10-28T00:08:48.864000: DEBUG : redo: attempt 1/3
2022-10-28T00:08:48.864000: DEBUG : redo: retry: calling _default_get with args: ('https://hg.mozilla.org/mozilla-central/json-pushes?changeset=49a271c42001f7bc5b9efadfc3e0b66dc0b67777&full=1',), kwargs: {}, attempt #1
2022-10-28T00:08:48.865000: DEBUG : urllib3.connectionpool: Resetting dropped connection: hg.mozilla.org
2022-10-28T00:08:49.773000: DEBUG : urllib3.connectionpool: https://hg.mozilla.org:443 "GET /mozilla-central/json-pushes?changeset=49a271c42001f7bc5b9efadfc3e0b66dc0b67777&full=1 HTTP/1.1" 200 None
2022-10-28T00:08:49.816000: DEBUG : Found commit message:
Backed out changeset e8354032aaf6 (bug 1787274) for being the likely cause of Fenix & Focus UI test bustage.

2022-10-28T00:08:49.816000: DEBUG : Did not find a branch, checking all integration branches
2022-10-28T00:08:49.817000: INFO : The bisection is done.
2022-10-28T00:08:49.819000: INFO : Stopped

Comment 4

[shadedblink]

By the way, command used to open new window:
browser.windows.create({
left: -100000,
top: -100000,
width: 1920,
height: 1080,
url: 'https://microsoft.com'
}).then(console.log)

Comment 5

[shadedblink]

Hello, any progress, ETA or anything on this task?

Comment 6

[Ray Kraesig [:rkraesig]]

This looks to be WebExtensions-specific. Bouncing over to them for retriaging and analysis.

Comment 7

[Alex Cornestean]

Hello,

I’m from Webextensions QA and I’m attempting to reproduce the issue in order to confirm it.
Would you be so kind as to attach a test extension so I can try and reproduce the bug? Any video or screenshot of the issue occurring is also welcome.

Thank you !

Comment 8

[shadedblink]

Hello.
I don't see any buttons to attach files to the message, so I will instead explain a reproduction steps.

[browser.windows.create() stuck]

1. Create any webextension. My permissions set:
   ["<all_urls>","storage","unlimitedStorage","bookmarks","webRequest","webNavigation","tabs","activeTab","tabHide"]
2. Open it's debug console. Write and execute:
   browser.windows.create({
   left: -100000,
   top: -100000,
   width: 1920,
   height: 1080,
   url: 'https://microsoft.com'
   }).then(console.log)
3. As you can see, new window has appeared on your Windows taskbar, but promise is not fulfilled yet. You can wait either for a minute or a hour, but nothing will happen, unless you hover your mouse over this new window in the taskbar so that it's preview is displayed. It forces this window to render and thus promise is fulfilled this very moment. In the same time you may notice that window has started to load document itself only after your requested this window to render, not when window was created.

[browser.tabs.captureTab() throws "unexpected error"]

1. After step 3 you have to execute following script and repeat it for some times(may be 1, 20 or even more iterations, pretty random, took 4 repeats for me last time). For all these repeats long you have to avoid hovering over taskbar.
   browser.tabs.create({
   windowId: 12345, // INSERT WINDOW ID FROM STEP 2 THERE
   url: 'https://microsoft.com',
   active: false
   }).then(x => setTimeout(() =>
   browser.tabs.captureTab(x.id).then(console.log)
   , 2000));
2. On random iteration you will see an exception in console instead of data url result:
   Uncaught (in promise) Error: An unexpected error occurred

Comment 9

[Alex Cornestean]

Hello and thank you for the extension contents !

I followed the STR and managed to reproduce the issue on the latest Release (107.0/20221110173214) and Beta (108.0b5/20221122190120) only, on Windows 10 x64. The promise does get fulfilled only after hovering over the taskbar as you explained in Step 3 from Comment 8 and as such after running the second code I got the error.

For further details, please see the attached screenshot.

On the latest Nightly 109.0a1 though, the promise gets fulfilled immediately after running the snippet in Step 2 from Comment 8 and after running the second code more than 40 times I could not get the error in the console to show up.

I also tried reproducing the issue on Ubuntu 16.04 LTS on the same browser versions but did not succeed. On Linux, the browsers behave like Nightly on Windows in the sense that the promise is immediately fulfilled and I cannot trigger the error when proceeding with the STR.

Comment 10

[Alex Cornestean]

Attachment: 2022-11-24_09h59_34.png

Comment 11

[Rob Wu [:robwu]]

Alex, are there any other errors in the global browser console?

Comment 12

[Alex Cornestean]

Hey Rob,

I reproduced the issue on the latest Release (107.0.1/20221128144904) on Windows 10 x64 and this is what I got in the browser console (see attachment).

As a dissection of the actions I performed and the results in the console:

• the first line is the result of Step 3 from Comment 8
• data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAB3IAAAPHCAYAAADdCQilAAAgAElEQVR4XuzdC7gtZV0 … is the result of a successful execution of the script from Step 1 in the second part of Comment 8
• and finally, Uncaught (in promise) Error: An unexpected error occurred is an unsuccessful execution of the script from Step 1 in the second part of Comment 8.

Comment 13

[Alex Cornestean]

Attachment: 2022-12-06_09h37_45.png

Comment 14

[shadedblink]

Hello again. Here I am with some additions and new bugs.

1. Meanwhile I noticed that page which was already in background loading(after preview display) may stop its loading immediately or some time after the preview is closed.
2. Long before this story(more than a half a year ago) I noticed that when the window is loaded in forementioned way, it may look different for some sites. It is not random, each site has exact difference. As I understand, it is some kind of optimization when browser decides not to load ot suspend to load some parts of page. For example some scripts may not run(occured for discord home page if I remember it right, it lacked bottom half of page on screenshots; youtube home page had all video preview pictures as solid gray), some images may not load or load partially(I noticed some of them were shaded unless you force window render, occured for toonily.com).
3. Some days ago I had my browser updated to 109.b2 and now forementioned way of opening a window doesn't work at all, even with sequential browser.windows.update() to move that window of the way. I assume that this is a bug since this behaviour violates documentation(which says that these fields are ignored only for panels and popups, not for normal windows) and not throws any exception when specified top/left is negative, these fields are just simply ignored in such case. Hope this is not how you are going to fix forementioned bug - just forbid any background windows.

Currently all these bugs made background tabs rendering impossible and actual behaviour is hella annoying and heavily interrupting any workflow on PC(just imagine that you have popups with focused: true that appear on your PC every 5min over any other window) so I am going to revert my instances of firefox to older version.
For the God's sake, I had heavy time to decide whether to move for chrome when you disabled XUL and lost support of original SpeedDial(what's the point of firefox when it decides to be same as chrome with every update?), then I moved first to FVD Speed Dial which had annoying mandatory focus switch when it opened it's background windows(since you don't support unfocused windows anymore), then I decided to move to my own extension for background page preview rendering. All this story just looks like a story of how firefox is trying to eradicate any possible speed dial compatibility. I hope you are gonna fix these bugs.

Comment 15

[Alex Cornestean]

Seeing your comment about Beta 109.0b2, I tried the STR you initially posted to see if anything is different from before.

Beta now behaves as Nightly 109 did initially (Comment 9). A new window is created when performing Step 2 from Comment 8, the page contents are rendered without me having to hover over the preview in the taskbar. Performing Step 1 from the second part of Comment 8 multiple times, does not produce the mentioned error.

This report has been set to New already and I’m not entirely sure what you are asking of me (QA).

If you want me to test out something different regarding this, please provide some steps to reproduce for this new scenario and a test extension in case the older one you provided is no longer relevant.

Thank you !

Comment 16

[shadedblink]

I wonder how you are checking it and get such results. That STR is not working anymore for me since window is created in visible area(i.e. left&top are ignored, new window is created over existing window in user-viewable area. I tried two different Win10 environments and both are getting same results.
I.e. I can't reproduce this bug anymore since now I am unable to open window in invisible area.

Comment 17

[Alex Cornestean]

My previous comment stated that on the current Beta, I get the results I initially got on Nightly i.e. the issue does not reproduce.

After running the below script in the debug console, I get a window which renders without me hovering over the taskbar. I’m using a multi monitor setup and the window appears split between my second and third monitor (anchored in the upper area, a bit on the second monitor and a bit on the third monitor). See screenshot.

browser.windows.create({ left: -100000, top: -100000, width: 1920, height: 1080, url: 'https://microsoft.com' }).then(console.log)

Comment 18

[Alex Cornestean]

Attachment: 2022-12-16_10h00_38.png

Comment 19

[shadedblink]

Is this window placement for you the same as in previous version(107 for example)? I am not sure on how this is treated on multimonitor setups, but original intention was for this window to be opened in totally non-viewable area, i.e. that's why left and top are negative - window should appear totally outside of display bounds.
I feel that on previous version you got this window outside of display bounds(as I intended). If window is present in viewable area, it will be rendered of course since OS requests this window to be rendered same way as it does when you hover over it on taskbar.

So I want to say that this bug is not just about new window not being rendered, but exactly about "background" windows not being rendered. By background window I mean the window that is not displayed to user in any way and that is not drawing any attention/focus. And I feel that this bug is not affected by v109 at all since only change was to forbid background windows - you can't even check this bug anymore. I.e. instead of fix you just removed this functionality.

Comment 20

[Alex Cornestean]

Indeed, using Beta 107.0b9 and performing the steps, will open the window in a non-viewable area i.e. the window cannot be seen on screen(s) anywhere. It’s only clear from the taskbar that a new window is opened though.
The promise resulted from running the snippet is only fulfilled after hovering over the preview in the taskbar and then page renders (as seen from the preview).

So apparently, this functionality has been indeed removed from 109.

Comment 21

[shadedblink]

So what will you do with this bug? Will you return and fix this functionality? I would say that this is core functionality for SpeedDial-like extensions since there is no other way to refresh dial previews.

Comment 22

[Alex Cornestean]

As I mentioned in a previous comment, I’m QA and do not have the authority to decide what changes are implemented or not. At this point there is nothing more I can do.
The issue has been brought to the attention of developers and we have to wait for them to make a decision on this matter.

Comment 23

[shadedblink]

Yes, I understand, I just have nobody else to ask for this. Notify me on their answer please.

Also I can offer a workaround way for this case to avoid encounter this bug, this is an API proposal. Add a method to get a snapshot for given url in background mode, i.e. like in good old times with XUL browser just renders requested url in background without actual creation of a new window or a tab. Result image may be encoded in some protected way so it will be only available to be displayed to user in <image/>, but in case of browser.tabs.captureTab() it won't be rendered in result image(like solid black or etc.). Also this image should throw an exception if script attempts to access it. I mentioned these restrictions only if you have security concerns for these images since they should use user's cookies during loading. Mandatory part of this proposal is only a method that gets a snapshot for an url without any window or tab.

However, fixes for existing API are prefered over this workaround.

Comment 24

[shadedblink]

Good news, everyone, I managed to break that again.
If window is left in minimized state or is just not focused beneath other window, it will stop to load the same way as in original case with opening out of the display box. Also it works when window is directly visible, but operation system session is in blocked state(like block with Windows+L).
Thus this bug is definitely still present, no changes in behaviour.
By the way, I reverted all other firefox instances to v93, everything works smooth and fine like originally intended. Why 93 - I just don't want to test every version prior to current to find last working version.

Comment 25

[Alex Cornestean]

In reply to Comment 23, as the issue has been set to New, it’s already at the attention of the developers. When the issue will be fixed or when there any changes pertaining to this, there will updates on the report you’ll be able to see.
Thank you !

Comment 26

[shadedblink]

Some additions, just noticed.
I launched firefox and switched to another fullscreen app before firefox window appeared. When I switched back(in about 2-3mins), I noticed that my home page animation behaved like extension just opened(i.e. it keeps loading screen before IndexedDB is opened which happens on background page when extension is started(not background window - I mean extension singleton background page)).
At the same time my extension internal timer that I use to postpone rendering operations only started on this 2-3min delay, not when I originally launched firefox.
Thus it seems that this bug has deeper impact on firefox than just browser.window API.

Comment 27

[Alex Cornestean]

Hey shadedblink,

Thank you for the additional information, but please refrain from NeedInfo-ing me with every update.

Regarding Comment 26, this appears to be a new bug? In case it is, please report it as such and provide the necessary steps to reproduce and extension (in case it differs from the one from this report).

Thank you !

Comment 28

[shadedblink]

Sorry for flag, just didn't see anything else to directly mention a person.
About that comment - I think that it has same nature with this bug, just witnessed different area of influence. At first I thought that only browser.window.create() API was affected, but it seems that either core(main background script) extension loading or even whole browser loading are affected, not only that method.

Comment 29

[Rob Wu [:robwu]]

(In reply to shadedblink from comment #24)

Good news, everyone, I managed to break that again.

At this point it is unclear what this bug report is about. Is it a bug in extensions? Is it a bug in Firefox? We cannot reliably reproduce this issue.

Could you report whether this works on the latest Firefox versions?
And if it still happens on your system, can you try to run mozregression again? The previous output in comment 3 looks unlikely, so that could have been a faulty mozregression session.

Comment 30

[shadedblink]

(In reply to Rob Wu [:robwu] from comment #29)

(In reply to shadedblink from comment #24)

Good news, everyone, I managed to break that again.

At this point it is unclear what this bug report is about. Is it a bug in extensions? Is it a bug in Firefox? We cannot reliably reproduce this issue.

Could you report whether this works on the latest Firefox versions?
And if it still happens on your system, can you try to run mozregression again? The previous output in comment 3 looks unlikely, so that could have been a faulty mozregression session.

Actually I am not sure myself whether it is extension-only or core firefox bug. What I know for sure is that all that misbehaviour revolves around the fact that firefox doesn't proccess invisible windows at certain circumstances. I.e. like I dont see it so it doesn't exist.
It was most easily tested in v108 and older versions since api for window placement was working right so you could have just instantly create it in invisible area and get that effect. But I have told you about this bug and we got v109, where window placement api was broken to avoid reproduction of this bug.
However, there are still ways to reproduce it, but they are more tricky. Main condition is that window should either be created without call for render from OS or be in invisible state long enough for that render-effect to fade off, like it was in part 2 of STR in comment 8.

For the first way you have to open your browser and then immediately switch to other full screen app BEFORE browser window appears. Thus you avoid it being rendered. In this case loading of extensions is postponed before your first focus switch on the browser window(like alt+tab or close your current fullscreen app). I.e. firefox won't continue to load unless you look at its window. According to my logs it is not like some certain API is in sleeping state, but no scripts on background page are started at all.

For the second case you have to open a window in minimized state. It works same way as window in invisible area, i.e. it is not rendered unless hovered over in taskbar to show its preview. Keep in mind, that when you open window in minimized state, firefox first opens it as a normal-state window, but then immediately changes its state to minimized. Thus this windows takes render request and keeps its effect for sometime, so part 1 in STR from comment 8 is NOT reproducible that way.

You better start with fixing the window placement API. This restriction just makes you more problem with reproduction of this bug.

Comment 31

[shadedblink]

Agh, it seems I missed your statement against comment 3. I will repeat it, but I don't think the result is gonna be different. You may get better results with STR I provided in comment 8.

Comment 32

[Tomislav Jovanovic :zombie]

Can you still confirm using mozregression?

Comment 33

[shadedblink]

Hello again, finally found some time for this.
This time I splited search between parts 1 and 2 of STR. Results for part 2 seems to be pretty accurate, it gave me this link: https://phabricator.services.mozilla.com/D151822
However, for the part 1 everything goes pretty unpredictable. I noticed that sometimes windows open normally while others are stuck. I.e. I open 5 windows with intervals of 1sec: first 3 opens normally, 5th open normally, 4th stucks. I.e.:

1 open
1 load //instant
2 open
2 load //instant
3 open
3 load //instant
4 open
5 open
5 load //instant
4 load // only after hover in taskbar.

Thus results for part 1 may be very random, because sometimes I encountered even 10x combo success before first fail. Whatever, this is what I got by regression:

2023-02-23T13:36:00.805000: INFO : Narrowed integration regression window from [1c3c5be9, 6bd345bd] (3 builds) to [31214a3a, 6bd345bd] (2 builds) (~1 steps left)
2023-02-23T13:36:00.805000: DEBUG : Starting merge handling...
2023-02-23T13:36:00.805000: DEBUG : Using url: https://hg.mozilla.org/mozilla-central/json-pushes?changeset=6bd345bd74c69508640678b78be826f8f138421b&full=1
2023-02-23T13:36:00.805000: DEBUG : redo: attempt 1/3
2023-02-23T13:36:00.805000: DEBUG : redo: retry: calling _default_get with args: ('https://hg.mozilla.org/mozilla-central/json-pushes?changeset=6bd345bd74c69508640678b78be826f8f138421b&full=1',), kwargs: {}, attempt #1
2023-02-23T13:36:00.805000: DEBUG : urllib3.connectionpool: Resetting dropped connection: hg.mozilla.org
2023-02-23T13:36:01.868000: DEBUG : urllib3.connectionpool: https://hg.mozilla.org:443 "GET /mozilla-central/json-pushes?changeset=6bd345bd74c69508640678b78be826f8f138421b&full=1 HTTP/1.1" 200 None
2023-02-23T13:36:01.915000: DEBUG : Found commit message:
no bug - Bumping Firefox l10n changesets r=release a=l10n-bump DONTBUILD

es-AR -> 6a7f9a43281890ba5cfe8bb3bcafda4966f373d8
it -> 007c027398b011e012ad9f2a03a3723263d3f664
nn-NO -> 8b0525028a979b4198cec6ee63eb8b45ef41cdfb
si -> b277e52b4db7aac5641e2f66d6fce94a3465a20a

2023-02-23T13:36:01.915000: DEBUG : Did not find a branch, checking all integration branches
2023-02-23T13:36:01.915000: INFO : The bisection is done.
2023-02-23T13:36:01.915000: INFO : Stopped

Comment 34

[shadedblink]

Hello, are there any news on this bug? May you atleast revert blocking of moving windows outside of display bounds? As you can see, it doesn't fix anything, it just breaks firefox API even more. Constant window flickering due to opening of minimized windows is pretty annoying.