1526822 - Headless doesn't work when redirect is involved

Status: RESOLVED FIXED

(Firefox :: Headless, defect)

Description

[Thomas Deutschmann]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0

Steps to reproduce:

1. firefox --no-remote --headless -createProfile "headless-test1"
2. firefox --no-remote --headless -P "headless-test1" --screenshot "http://www.gentoo.org/"

Actual results:

Nothing will happen:

Firefox will start and consume CPU cycles but no screenshot will be written.

If you change URL from "http://www.gentoo.org/" to "https://www.gentoo.org/" instead it will work.

Tested on Debian, Arch Linux and Gentoo. With Firefox 65 build from source and official binary provided by Mozilla.

You can use any other URL redirecting. For example http://www.debian.org/ will trigger same problem but https://www.debian.org/ will work. When using http://www.mozilla.org/ which won't work, you have to use https://www.mozilla.org/en-US/ for a working URL, just using https://www.mozilla.org/ won't work (I guess due to the redirect to "/en-us/").

Comment 1

[Thomas Deutschmann]

I can also reproduce on Windows using official builds.

Still happening with latest Nightly (67.0a1 (2019-02-10) (64-bit)).

Comment 2

[Luis]

I can reproduce it on Ubuntu 18.04 and Ubuntu 18.10 with the Firefox from Ubuntu repository (v65.0) and developer edition (v66.0b7) manually downloaded :

1. firefox --screenshot test1.png https://www.google.es/ -> OK
2. firefox --screenshot test1.png http://www.google.es/ -> Only shows "*** You are running in headless mode." and hangs

And this happens with any kind of redirect, not only http to https redirects.

Comment 3

[Daniel Ritz]

Can confirm on Ubuntu 16.04 and macOS.

This is probably what breaks all our CI builds with Selenium. The funny thing is that while FF hangs on macOS too with the above two commands, the headless tests run fine. But only on macOS, hangs on Linux.

Comment 4

[Daniel Ritz]

Is there anything we can do to help this get fixed? I would really hate to drop FF from CI, but Chrome works while FF doesn't...

Comment 5

[Daniel Ritz]

btw. on one occasion is saw this message repeat endlessly:

###!!! [Parent][MessageChannel] Error: (msgtype=0x9E0001,name=PVsync::Msg_Notify) Channel error: cannot send/recv

Also, I did some strace -f -p <pid> for firefox, the one content process with high CPU, geckodriver. That might help. Files in a minute or so...

Comment 6

[Daniel Ritz]

Attachment: strace of firefox main process

Comment 7

[Daniel Ritz]

Attachment: strace of firefox content process

Comment 8

[Daniel Ritz]

Attachment: strace of geckodriver

Comment 9

[Daniel Ritz]

(In reply to Daniel Ritz from comment #5)

Also, I did some strace -f -p <pid> for firefox, the one content process with high CPU, geckodriver. That might help. Files in a minute or so...

forgot to mention: this is from a Ruby on Rails test suite with Capybara and Selenium on Ubuntu 16.04.6 with plain Ubuntu built FF. In another env, a test suite with Java driving Selenium fails in the same way (don't know much details there as it's not my setup).

Comment 10

[Brendan Dahl [:bdahl]]

Attachment: Bug 1526822 - Fix headless screenshot that has redirects. r=mossop

The changes in bug 1507352 caused onLocationChange events to be ignored
when there was an http redirect. This caused the screenshot command to hang
since it never attached an event listener on the page it was redirected to.

Comment 11

[Brendan Dahl [:bdahl]]

Try run: https://treeherder.mozilla.org/#/jobs?repo=try&revision=6794d493316606aa289503633e8bd3edd0d90d72

Comment 13

[Daniel Ritz]

Thanks. I tried patching my FF 65 on Ubuntu with this. Screenshots are indeed working again, that's good. But my Selenium tests still hang. But when executed locally on macOS, they run just fine. Weird and hard to debug :(

Comment 15

[Daniel Ritz]

can we get at least the screenshot fix from above for FF66? that would already fix some of the use cases...thx

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

There's a r+ patch which didn't land and no activity in this bug for 2 weeks.
:bdahl, could you have a look please?

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/47064bce8693
Fix headless screenshot that has redirects. r=mossop

Comment 18

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/47064bce8693

Comment 19

[Luis]

This bug isn't fixed! Keeps falling with simple redirects :

firefox --screenshot x.jpg http://bibliotecadigital.jcyl.es --window-size=800,1000

Hangs with 66.0.3, firefox beta or firefox nightly!

index.html :

<HTML>
<HEAD>
<meta http-equiv="refresh" content="0;url=es/inicio/inicio.do"/>
</HEAD>
<BODY>
<SCRIPT>
<!--
top.location='es/inicio/inicio.do';
//-->
</SCRIPT>
</BODY>
</HTML>