Open Bug 1375585 Opened 7 years ago Updated 9 months ago

Support WebGL in headless mode

Categories

(Core :: Graphics: CanvasWebGL, defect, P2)

Product:
Core
Component:
Graphics: CanvasWebGL
Type:
defect

Tracking

()

People

(Reporter: mismith, Unassigned)

References

(Regression)

Details

(Keywords: regression)

Attachments

(1 file)

Bug 1375585 - Support WebGL in headless mode. r?jgilbert
47 bytes, text/x-phabricator-request
Details | Review 
At the moment WebGL tries to interact with a display at various points (see gfx/gl/GLContextProviderGLX.cpp, for example). This causes a crash in headless mode. I'm disabling WebGL in headless for the time being, but it'd be nice to support it down the road.

Ideas:
- Use a software GL implementation like Mesa? (what about non-Linux platforms?)
- Detect if a display server is actually running and use system GPU if available?
- Provide an option to disable WebGL in headless, for performance?
Priority: -- → P3
Hi,
I'm experimenting the same problem. It would be interesting to have a minimal WebGL as chromium does at least on GNU/linux and mac os x https://chromium-review.googlesource.com/c/chromium/src/+/522068.
Confirming the same issue as the above commenters; on CentOS 7 with Firefox 60.1.0, and on Ubuntu 16.04 with Firefox 61.0.1. As mismith says, a command-line argument to run headless with WebGL disabled would be great.
+1
Experiencing the same problem here, Windows 10 Home/Pro 64bit, Firefox 60.0.3 64bit, Selenium 3.114.0, GeckoDriver v0.23.0-win64.
Status: UNCONFIRMED → NEW
Ever confirmed: true

I am developing a wasm application and I have lots of WebGL2 code I would like to test on headless Firefox, but canvas.getContext("webgl2") is failing (now I know why). It would be highly appreciated to get WebGL support back to headless mode.

As another use case for webgl in headless.

We use headless browsers to do tracking research. In particular we use the OpenWPM framework (https://github.com/mozilla/OpenWPM/) which is a webextension that gathers data about what's happening on the page instrumenting both web requests and js apis.

Last summer I was hoping to do crawling on more platforms and advocated for the switch we made from using firefox with xvfb on linux to using firefox in headless mode.

However, that was probably premature on my part. The lack of webgl support affects us in two ways.

Firstly, WebGL fingerprinting (https://browserleaks.com/webgl) is a real concern and something that I'm going to start trying to measure (to do this for now I will add back in xvfb support).

Secondly, the lack of webgl support increases the risk that our crawlers are identified as a bot (https://github.com/mozilla/OpenWPM/issues/448)

+1
Having same issue using playwright firefox headless.
I would love this fixed.

Component: Headless → Canvas: WebGL
Priority: P3 → P2
Product: Firefox → Core

This Should Just Work, oops!

Although I feel like I have seen a crash in the past as per the original reporter, I cannot reproduce a crash now. However WebGL does not work.

If I run the following:

firefox https://get.webgl.org -P headless --screenshot out.png

(note that -P headless is just using my headless profile, --screenshot automatically runs the browser in headless mode)

I get the following screenshot.
source: http://www.sarahbird.org/out.png
Edited. This screenshot is not accurate its an artifact of the screenshot being taken before the page finishes running all the JS. See below.

If I remove --screenshot and run

firefox https://get.webgl.org -P headless

My browser opens the webgl page correctly and I see the following.

source: http://www.sarahbird.org/screenshot-webgl.png

:jgilbert - is there any other information I can get you?

Flags: needinfo?(jgilbert)

That's weirder. If you disable WebGL for that page, you normally get:

Hmm. While your browser seems to support WebGL, it is disabled or unavailable. If possible, please ensure that you are running the latest drivers for your video card.

Instead, you see it saying JS is disabled. Someone will need to poke at this, but that doesn't sound like a WebGL problem anymore. Possibly the profile has JS disabled?

Flags: needinfo?(jgilbert)

Agree. That's super wierd. I did check and JS is not disabled. I got a lead this morning that there's a console log that webgl gets preffed off in headless mode. I will setup remote debugging and check more deeply.

Here's the console output, from running the command:

 firefox -P headless --start-debugger-server 8899 --headless https://get.webgl.org

Content Security Policy: Couldn’t process unknown directive ‘image-src’

WebGL warning: <Create>: Can't use WebGL in headless mode (https://bugzil.la/1375585). get.webgl.org:193:23

Failed to create WebGL context: WebGL creation failed: 
* Can't use WebGL in headless mode (https://bugzil.la/1375585). get.webgl.org:193:23

WebGL warning: <Create>: Can't use WebGL in headless mode (https://bugzil.la/1375585). get.webgl.org:197:27

Failed to create WebGL context: WebGL creation failed: 
* Can't use WebGL in headless mode (https://bugzil.la/1375585). get.webgl.org:197:27

I have just checked the webgl prefs from the remote debugging e.g. Services.prefs.getBoolPref('webgl.disabled') and the response was false.

So the pref is not being flipped in the headless session. Something lower level is.

Looked it up on dxr and here's the result:

https://dxr.mozilla.org/mozilla-central/source/dom/canvas/WebGLContext.cpp#314-322

  // Can't use WebGL in headless mode.
  if (gfxPlatform::IsHeadless()) {
    FailureReason reason;
    reason.info =
        "Can't use WebGL in headless mode (https://bugzil.la/1375585).";
    out_failReasons->push_back(reason);
    GenerateWarning("%s", reason.info.BeginReading());
    return false;
  }
Flags: needinfo?(jgilbert)

I'm going to try a build locally with that if clause removed and see what happens.

Flags: needinfo?(jgilbert)
Regressed by: 1373739
Keywords: regression
See Also: → 1632948

This wasn't a regression. At the time (2017) we had no way of creating a GL context from headless mode, so any WebGL access caused a hard crash. It'd be cool if this has changed and things now "just work", with all that's gone into the rendering backend since then.

See Also: → 1632951

(In reply to Michael Smith [:mismith] (non-bmo mail: mds009@eng.ucsd.edu) from comment #14)

This wasn't a regression. At the time (2017) we had no way of creating a GL context from headless mode, so any WebGL access caused a hard crash. It'd be cool if this has changed and things now "just work", with all that's gone into the rendering backend since then.

Did I actually confirm that headless wasn't viable? That would surprise me! You can see in even code back then that most of WebGL was designed to operate as "headless" as far as its GL context was concerned. Canvases that aren't inserted into the DOM are effectively headless anyways! I don't think my "it should just work" expectation has changed since then, but rather it's likely there was pressure to just disable it in order to ship. It should always have just worked, but I can understand cutting that corner at the time.

Bug 1373739 is the time this code changed to the current behavior, even if it's not a regression from a previously-known-working state.

Assignee: nobody → birdsarah

Not unviable, or else I wouldn't have opened this ticket :)

Pointing headless mode at a web page which used WebGL caused a crash, when (as I wrote up at the top) GLContextProvider tried to access a display. This occurred very early on in initialization. It was particularly problematic for users applying headless Firefox for web crawling. It was decided that we would temporarily disable WebGL in headless mode so that those pages could at least be loaded without crashing the browser. (Note this included the surprising amount of the web making non-user-visible calls into WebGL for, e.g., fingerprinting purposes.)

At the same time, I filed this ticket as a todo item to route around anything in WebGL that needed access to a display in order to create the context. Eventually that internship ended and it never got picked up, I guess!

Naively commenting out the if clause results in the following error.

(/home/bird/Dev/firefox/src/mozilla-central/obj-x86_64-pc-linux-gnu/dist/bin/firefox:29688): Gdk-CRITICAL **: 14:20:04.748: gdk_x11_display_get_xdisplay: assertion 'GDK_IS_DISPLAY (display)' failed
JavaScript warning: https://get.webgl.org/, line 193: Failed to create WebGL context: WebGL creation failed: 
* tryNativeGL
* Exhausted GL driver options.

(/home/bird/Dev/firefox/src/mozilla-central/obj-x86_64-pc-linux-gnu/dist/bin/firefox:29688): Gdk-CRITICAL **: 14:20:04.750: gdk_x11_display_get_xdisplay: assertion 'GDK_IS_DISPLAY (display)' failed
JavaScript warning: https://get.webgl.org/, line 197: Failed to create WebGL context: WebGL creation failed: 
* tryNativeGL
* Exhausted GL driver options.
See Also: → 1632971

If I setup xvfb on display 99, I get this different error

*** You are running in headless mode.
JavaScript warning: https://get.webgl.org/, line 193: WebGL warning: <Create>: getContext: Disallowing antialiased backbuffers due to blacklisting. (FEATURE_FAILURE_GLXTEST_FAILED)
JavaScript warning: https://get.webgl.org/, line 193: WebGL warning: <Create>: Refused to create native OpenGL context because of blacklist entry: FEATURE_FAILURE_GLXTEST_FAILED
JavaScript warning: https://get.webgl.org/, line 193: Failed to create WebGL context: WebGL creation failed: 
* Refused to create native OpenGL context because of blacklist entry: FEATURE_FAILURE_GLXTEST_FAILED
* Exhausted GL driver options.
JavaScript warning: https://get.webgl.org/, line 197: WebGL warning: <Create>: getContext: Disallowing antialiased backbuffers due to blacklisting. (FEATURE_FAILURE_GLXTEST_FAILED)
JavaScript warning: https://get.webgl.org/, line 197: WebGL warning: <Create>: Refused to create native OpenGL context because of blacklist entry: FEATURE_FAILURE_GLXTEST_FAILED
JavaScript warning: https://get.webgl.org/, line 197: Failed to create WebGL context: WebGL creation failed: 
* Refused to create native OpenGL context because of blacklist entry: FEATURE_FAILURE_GLXTEST_FAILED
* Exhausted GL driver options.
Assignee: birdsarah → nobody

I'm guessing this is not an easy fix, but if it is I'm happy to take a crack :jgilbert. Unassigning myself for now.

Given that it works on chromium, presumably there is a way of calling x11 appropriately. I wonder if it "just works" on windows and mac?

Or maybe we need to have it not route through the "get me a display" code path if headless mode is enabled.

The following

@@ -461,6 +463,9 @@ bool WebGLContext::CreateAndInitGL(
     return gl;
   };
 
+  if (gfxPlatform::IsHeadless()) {
+    tryNativeGL = false;
+  }
   const auto newGL = [&]() -> RefPtr<gl::GLContext> {
     if (tryNativeGL) {
       if (useEGL)

updates the output error situation slightly, to:

*** You are running in headless mode.
JavaScript warning: https://get.webgl.org/, line 193: Failed to create WebGL context: WebGL creation failed: 
* Exhausted GL driver options.
JavaScript warning: https://get.webgl.org/, line 197: Failed to create WebGL context: WebGL creation failed: 
* Exhausted GL driver options.

(In reply to Sarah Bird from comment #20)

Given that it works on chromium, presumably there is a way of calling x11 appropriately. I wonder if it "just works" on windows and mac?

Or maybe we need to have it not route through the "get me a display" code path if headless mode is enabled.

Chromium falls back to a software GL implementation in headless mode (before they used Mesa, now SwiftShader): https://bugs.chromium.org/p/chromium/issues/detail?id=617551

As of 2019 you can manually force hardware GPU usage via EGL on some platforms (Linux, macOS): https://bugs.chromium.org/p/chromium/issues/detail?id=765284

(In reply to Sarah Bird from comment #21)

const auto newGL = & -> RefPtr<gl::GLContext> {
if (tryNativeGL) {
if (useEGL)

I see "EGL" in there, so maybe we can similarly force that here?

I just tried that with following patch

@@ -463,7 +465,7 @@ bool WebGLContext::CreateAndInitGL(
 
   const auto newGL = [&]() -> RefPtr<gl::GLContext> {
     if (tryNativeGL) {
-      if (useEGL)
+      if (useEGL || gfxPlatform::IsHeadless())
         return fnCreate(&gl::GLContextProviderEGL::CreateOffscreen, "useEGL");
 
       const auto ret =

and it failed for me

*** You are running in headless mode.

(/home/bird/Dev/firefox/src/mozilla-central/obj-x86_64-pc-linux-gnu/dist/bin/firefox:13349): Gdk-CRITICAL **: 15:01:57.137: gdk_x11_display_get_xdisplay: assertion 'GDK_IS_DISPLAY (display)' failed
JavaScript warning: https://get.webgl.org/, line 193: Failed to create WebGL context: WebGL creation failed: 
* useEGL
* Exhausted GL driver options.

(/home/bird/Dev/firefox/src/mozilla-central/obj-x86_64-pc-linux-gnu/dist/bin/firefox:13349): Gdk-CRITICAL **: 15:01:57.139: gdk_x11_display_get_xdisplay: assertion 'GDK_IS_DISPLAY (display)' failed
JavaScript warning: https://get.webgl.org/, line 197: Failed to create WebGL context: WebGL creation failed: 
* useEGL
* Exhausted GL driver options.

The error is coming from: gdk_x11_display_get_xdisplay: assertion 'GDK_IS_DISPLAY (display)' presumably we need to call / setup x differently? On my machine I have webgl drivers, so it's not that they're absent it's the x interaction. If they were absent e.g. in a docker container in the cloud, then the error may be different.

In headless mode we shouldn't be calling into x11/gdk. Most of what headless does on Linux is stub out or mock the behavior of gdk. Ultimately, we wanted to have a build without x11/gdk, but I don't foresee that happening anytime soon.

I have good news. webgl and headless appears to "just work" on my windows box.

It's a windows laptop, has an nvidia graphics card.

I simply commented out

  // Can't use WebGL in headless mode.
  if (gfxPlatform::IsHeadless()) {
    FailureReason reason;
    reason.info =
        "Can't use WebGL in headless mode (https://bugzil.la/1375585).";
    out_failReasons->push_back(reason);
    GenerateWarning("%s", reason.info.BeginReading());
    return false;
  }

Then ran: ./mach run --start-debugger-server 8899 --headless get.webgl.org.

Then attached to debugging console. As would be expected, I did not see the console messages "Failed to create WebGL context".

Finally, :screenshot from js console to get a screenshot and it has webgl cube.

note that ./mach run --screenshot get.webgl.org does not produce a screenshot with the cube because the screenshot is so quick that it hasn't had time to load. (See Bug 1216731).

I don't know whether it will "just work" for OSX.

Any updates on this?
It has been 3 years since this issue was opened...

Maybe this issue I created one year ago can help.
https://github.com/mozilla/geckodriver/issues/1520

I found that I needed the WebGL to render Mapbox map on Firefox headless mode (You can see the screenshot in issue 1520)

So far, I can use pyvirtualdisplay==1.3.2 to render Mapbox map on Firefox 79.0 and geckodrive is 0.27.0.

By the way, I ran this in the docker, OS is ubuntu16.04 stable version. Remember to set --shm-size 2g and --memory 1024mb as you run the docker container, otherwise, you will encounter other issue such as Failed to decode response from marionette with Firefox >= 65

Assignee: nobody → birdsarah

WIP for review and assistance.

Notes

I previously tried the modification to dom/canvas/ on windows and it just worked
(https://bugzilla.mozilla.org/show_bug.cgi?id=1375585#c27). I have
not tried it on windows this time round.

I have not tried this patch on OSX.

On Ubuntu, X11, this works (can screenshot the cube on get.webgl.org in a remote session)

But, I get the following errors, which I think should be cleaned-up / handled:

Can't find symbol 'eglGetNativeClientBufferANDROID'.
Can't find symbol 'eglQuerySurfacePointerANGLE'.
Can't find symbol 'eglCreateStreamProducerD3DTextureANGLE'.
Can't find symbol 'eglStreamPostD3DTextureANGLE'.

(mozilla-central/obj-x86_64-pc-linux-gnu/dist/bin/firefox:276021): Gdk-CRITICAL **: 21:54:45.232: gdk_x11_display_get_xdisplay: assertion 'GDK_IS_DISPLAY (display)' failed

ToDos and Questions

  • The main chunk in nsAppRunner needs dedenting, but I didn't do that yet to keep things simple
  • Test on Wayland
  • Write a test / enable some webgl headless tests if some exists already
  • Understand the source of the four can't find symbol errors (I don't get them in non-headless mode)
  • It's not surprising that we're getting an xdisplay error, but should fix something so it doesn't fire.
    Not quite sure where the right place is.

Any updates or progress on the above discovery?

I opened a work in progress PR (https://phabricator.services.mozilla.com/D88945), that was waiting on input from jgilbert on the following questions:

ToDos and Questions

  • Test on Wayland (I don't have a way to do this)
  • Write a test / enable some webgl headless tests if some exists already
  • Understand the source of the four can't find symbol errors (I don't get them in non-headless mode)
  • It's not surprising that we're getting an xdisplay error, but should fix something so it doesn't fire. Not quite sure where the right place is.

That PR was accepted by jgilbert. But not merged. So I'm not sure what the next steps are.

Flags: needinfo?(jgilbert)

Sarah as it looks like your patch got approved but since then no further activity happened. As such is there more work to do, or could the patch get landed? Thanks.

Flags: needinfo?(jgilbert) → needinfo?(birdsarah)

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #33)

Sarah as it looks like your patch got approved but since then no further activity happened. As such is there more work to do, or could the patch get landed? Thanks.

The PR was intended as a WIP PR and I left a number of questions on it and was hoping to get support to see it through to conclusion. Unfortunately that wasn't available at the time when I had time to work on the issue and I no longer have time to work on it.

Others are welcome to pick up the work if they'd like.

Flags: needinfo?(birdsarah)

Thanks for the update Sarah. As such I will remove you as assignee from this bug. Now reading your last comment on this bug it would indeed be good to know what would be next. Maybe Jgilbert could give some feedback. This would be kinda helpful, especially when someone else wants to pick it up.

Assignee: birdsarah → nobody
Flags: needinfo?(jgilbert)
Severity: normal → S3
Flags: needinfo?(jgilbert)
Has Regression Range: --- → yes

There's a r+ patch which didn't land and no activity in this bug for 2 weeks.
:birdsarah, could you have a look please?
If you still have some work to do, you can add an action "Plan Changes" in Phabricator.
For more information, please visit auto_nag documentation.

Flags: needinfo?(jgilbert)
Flags: needinfo?(birdsarah)

This bug has the keyword regression, so its type should be defect.

Type: enhancement → defect

I don't know if the patch is still acceptable. That's for Kelsey to decide. But I can say that if it's not, then I don't currently have bandwidth to get back into this issue.

Flags: needinfo?(birdsarah)

This is not actionable for me until I am told how to start Firefox in headless mode.
Does it still work?

Flags: needinfo?(jgilbert) → needinfo?(birdsarah)

Kelsey, if you're on linux you can do MOZ_HEADLESS=1 ./firefox or ./firefox --headless.

You can see notes here: https://bugzilla.mozilla.org/show_bug.cgi?id=1375585#c27

./mach run --start-debugger-server 8899 --headless get.webgl.org.

Flags: needinfo?(birdsarah)

On the emscripten project this bug currently blocks us from running any of our graphics tests under firefox: https://github.com/emscripten-core/emscripten/blob/main/.circleci/config.yml#L368-L370

Just FYI in case it encourages anyone to work in this..

Is there any progress on this issue - or a workaround folks can use to enable WebGL in headless mode?

I am trying to test the WebGL browser fingerprint of a Selenium-driven Tor Browser in headless mode (in a Docker container) and this issue seems to be a barrier. It seems to persist with FF 120 as firefox --headless still gives [GFX1-]: RenderCompositorSWGL failed mapping default framebuffer, no dt with FF v120.0.1 (on Debian 12).

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: