1503241 - The CloudShell on Google Cloud Platform does not load (prompt missing)

Status: RESOLVED FIXED

(Core :: DOM: Core & HTML, defect, P2)

Description

[Hussain]

Attachment: Screen recording of the CloudShell that is not working

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:65.0) Gecko/20100101 Firefox/65.0

Steps to reproduce:

Click on the Activate Cloud Shell on the Google Cloud Platform console.


Actual results:

The Cloud shell opens, loads but then does not show the shell prompt.


Expected results:

Shell prompt should work as it does on Firefox 63.

CloudShell breaks in both Firefox 64 and 65

Comment 1

[Adrian Florinescu [:aflorinescu]]

Regression range:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=982f91f69b39773ac302d4d01610de1414894529&tochange=70f63648a0f6cf738d5555eb347986d7323ef16e

Seems like bug 1497126 introduced this regression.

Comment 2

[Marion Daly [:mdaly]]

Baku, can you take a look at this?

Comment 3

[Andrea Marchesini [:baku]]

> Click on the Activate Cloud Shell on the Google Cloud Platform console.

I don't have a cloud google service. I don't have any shell to activate. Can somebody give me valid account or tell me the fastest way to reproduce this issue? Adrian, can you help me here?

Comment 4

[Adrian Florinescu [:aflorinescu]]

Attachment: console.jpg

Sure, you can use your mozilla account - just ignore all try for free messages and proceed to the login)

https://accounts.google.com/signin/v2/identifier?passive=true&continue=https%3A%2F%2Fcloud.google.com%2Fcloud-console%2F%3Frefresh%3D1&service=cloudconsole&flowName=GlifWebSignIn&flowEntry=ServiceLogin

Comment 5

[Andrew Overholt [:overholt]]

Thanks, Adrian! baku, this seems like a not great regression so I'll assign it to you and mark it P1.

Comment 7

[Nathan Froyd [:froydnj]]

[Tracking Requested - why for this release]: Bad regression, major site.

Comment 8

[Andrea Marchesini [:baku]]

adrian_sv, I'm not able to reproduce the issue anymore with the latest nightly. Can you confirm this?

Comment 9

[[Back August 16] Ryan VanderMeulen]

Please bisect the fix too if indeed this is fixed.

Comment 10

[Hussain]

This bug, as has been reported, has been fixed. I can now see work with the cloud shell but I think the underlying problem is still present because when I try the ssh connect to an instance from the web it has the same symptoms which are reported in https://bugzilla.mozilla.org/show_bug.cgi?id=1513568


To reproduce, create an instance on GCP (would need to add billing info for that I guess) and click the ssh button of that instance. I have the video of it but can't upload it here.

Comment 11

[Andrea Marchesini [:baku]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #9)
> Please bisect the fix too if indeed this is fixed.

It seems been fix by the server side. I cannot reproduce this issue also using a 2 months old nightly.

Comment 12

[Hussain]

baku, you could still reproduce the problem in an instance's ssh console as I have described above (comment 10).

If creating a billing account on gcp is not convenient, I could try to create a project and invite a developer to it so that they can easily work on this.

Comment 13

[Andrea Marchesini [:baku]]

> If creating a billing account on gcp is not convenient, I could try to
> create a project and invite a developer to it so that they can easily work
> on this.

If this is OK for you, I just need to check why this happens to propose a solution. Yes, please.

Comment 14

[Hussain]

:baku please check your email.

Comment 15

[Adrian Florinescu [:aflorinescu]]

Removing the NI, since I see Hussain already helped on the required additional investigation and it seems you need billing info for GCP.

Comment 16

[Hussain]

Please somehow snapshot the page so that it's reproducible even if the server response is changed. Would love to know how the snapshot was take too :D

Comment 17

[Hussain]

The above mentioned (ssh through web console) is now fixed as well. I hope that too was not a server-side fix.

Comment 18

[Hussain]

More info: Although the ssh console now work. But it's very unstable e.g. if we paste some command (git clone ...) in the terminal the terminal blacks out. Some random keystrokes bring it back and you can we that the command hasn't been executed.

Comment 19

[Andrea Marchesini [:baku]]

Hussain, can you check if the issue goes away when disabling anti-tracking?
You can do it on about:preferences#privacy or setting network.cookie.cookieBehavior to 0.
Thanks for helping with the debugging of this bug.

Comment 20

[Hussain]

Just checked. Disabling anti-tracking didn't help.

Comment 21

[Julien Cristau [:jcristau] (away, back August 16)]

It's too late for 64 now.

Comment 22

[[Back August 16] Ryan VanderMeulen]

Seems very unlikely to make 65 at this point, though I'd consider a low-risk patch as a dot release ride-along if one is available.

Comment 23

[Tim Spurway [:tspurway]]

Hey :baku, how's this looking? should we still be tracking for 66?

Comment 24

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

While this isn't a new problem in 66 I notice we have at least one duplicate report. It might be affecting many users, so, if we do land a fix I'd be inclined to uplift as far as we think is safe.

Comment 25

[Andrea Marchesini [:baku]]

I have no news here. Is there any chance that somebody else jumps on this bug? It's a P1 but I don't know exactly when I have time to work on it.

Comment 26

[Marcia Knous [:marcia]]

Hsin-Yi - Based on Comment 25, can we try to find a new owner? Thanks in advance.

Comment 27

[Hsin-Yi Tsai (Fx104 REO) [:hsinyi]]

I'm exploring the alternative. Will get back here.

Comment 28

[Andrea Marchesini [:baku]]

Karl, can you help me to debug this issue? I don't understand why enabling 2 nested iframes with the same URL breaks that website. I'm NI-ing you because this could be a webcompact issue. Thanks.

Comment 29

[Karl Dubost💡 :karlcow]

We have a lot of issues related to this site not all the same, often closed because lack of details or fixed in release. The site seems also very sensitive to CSP, and any kind of blockers.

This looks like the same issue which was reported on October 2018 in https://webcompat.com/issues/20551

We had also for this site.

SSH blocking

• https://webcompat.com/issues/20551
  Dupe of https://webcompat.com/issues/20505
  which was itself dupe of https://webcompat.com/issues/15089 closed as worksforme by denschub
• https://webcompat.com/issues/21295

We had also
https://webcompat.com/issues/9716 (with a lot of duplicates) which was investigated by Thomas Winiewski
The issue was for this one an incorrect use of the Performance Observer API and it has been fixed.
Google Engineers were super responsive in that bug.
Was related to Bug 1398477 probably
Maybe also https://webcompat.com/issues/20551

Form password issue

• https://webcompat.com/issues/15406 Dupe of Bug 1043118

Text input

• https://webcompat.com/issues/22025

I will transfer this issue to Thomas or Dennis, who have already investigated it in the past.

Comment 30

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

Following up in email.

Comment 31

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

Tom is going to look into this, but it seems unlikely we'll be aiming to fix this for 66. We could still take a patch for 68 and potentially uplift to 67, though.

Comment 32

[Mike Taylor [:miketaylr]]

Attachment: working now?

Hussain, can you verify that this is working now? I've tested in 67 Beta and 68 Nightly.

Comment 33

[Mike Taylor [:miketaylr]]

Aw crud, I didn't follow the STR correctly. I still get "Connecting: Provisioning your Google Cloud Shell machine...".

Back over to you Tom.

Comment 34

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

Tom is about to be on pto so I'm marking this fix-optional for 67 and we can aim for a fix in 68.

Comment 35

[Reuben Morais [:reuben]]

Huh, I just tried this again and it worked. Then I tried one more time and it failed with a blank screen and a JS error in the console: "TypeError: this.screen_ is undefined". So it seems the problem is intermittent.

Comment 36

[Reuben Morais [:reuben]]

The error seems to be in the hterm.js component, and I can reproduce it by simply going the project page at https://hterm.org/ which is a live instance of the terminal emulator. Sometimes it works, sometimes it doesn't. Enabling network throttling in devtools seems to make it happen more often.

Comment 37

[Firefox Bug Husbandry Bot]

Migrating Webcompat whiteboard priorities to project flags. See bug 1547409.

Comment 38

[Firefox Bug Husbandry Bot]

See bug 1547409. Migrating whiteboard priority tags to program flags.

Comment 39

[Thomas Wisniewski [:twisniewski]]

I just tested this on Firefox 64.0.2 on Linux and OSX (also the current nightly on Linux) and I was able to get into my shell console just fine. I suspect that whatever broke it has since been fixed on Google's end. Mike, can you try again?

Comment 40

[Mike Taylor [:miketaylr]]

Tom, can you reproduce the issue described by Reuben in comment #36?

If I visit that site and refresh 10 or 15 times, it will reproduce. I see

TypeError: this.screen_ is undefinedhterm_all.js:10871:3

Breaking on hterm.ScrollPort.prototype.setBackgroundColor (when it's not broken), this.screen_ is a custom element x-screen.

Comment 41

[Thomas Wisniewski [:twisniewski]]

On my nightly build I get a completely different an unrelated error on hterm.org which breaks the keyboard functionality (it doesn't happen in Chrome):

TypeError: this.scrollPort_.getDocument(...) is null hterm_all.js:12756:50
    installKeyboard https://hterm.org/dist/1.84/hterm_all.js:12757
    setupHterm https://hterm.org/index.js:157
    initNext https://hterm.org/dist/1.84/hterm_all.js:172
    wrapperGenerator https://hterm.org/dist/1.84/hterm_all.js:1755
    initOs https://hterm.org/dist/1.84/hterm_all.js:4782

That happens here:

hterm.Terminal.prototype.installKeyboard = function() {
  this.keyboard.installKeyboard(this.scrollPort_.getDocument().body);
};

hterm.ScrollPort.prototype.getDocument = function() {
  return this.document_;
};

Sure enough, the ScrollPort only seems to set its document_ in a function named paintIframeContents_, which is being called too late in Firefox compared with Chrome (I've snipped some irrelevant lines from this code-fragment):

hterm.ScrollPort.prototype.decorate = function(div, callback) {
  this.div_ = div;
  this.iframe_ = div.ownerDocument.createElement('iframe');
  div.appendChild(this.iframe_);

  const onLoad = () => {
    this.paintIframeContents_();
  }

  // Insert Iframe content asynchronously in FF.  Otherwise when the frame's
  // load event fires in FF it clears out the content of the iframe.
  if ('mozInnerScreenX' in window) { // detect a FF only property
    this.iframe_.addEventListener('load', () => onLoad());
  } else {
    onLoad();
  }
}

I've confirmed that they need that work-around in Firefox, but doing so causes the ScrollPort's document_ to be not be set in time for the installKeyboard call. This error hits Firefox as far back as version 56, according to mozregression.

They would need to delay calls to installKeyboard until after the ScrollPort has been set up. For example, this quick hack lets me get to a usable state:

hterm.Terminal.prototype.installKeyboard = function() {
  let doc = this.scrollPort_.getDocument();
  if (!doc) {
    setTimeout(() => { this.installKeyboard(); }, 100);
  } else {
    this.keyboard.installKeyboard(doc.body);
  }
};

But it's still a different error than the screen_ one others were getting. However, this is the line of code that was being a problem for others:

hterm.ScrollPort.prototype.setBackgroundColor = function(color) {
  this.screen_.style.backgroundColor = color;
};

And screen_ is also defined in paintIframeContents_, so I suspect that it was breaking for similar reasons (the iframe wasn't loaded, so some of the ScrollPort's state wasn't set yet).

As such I wouldn't recommend they simply apply my above hack, but rather improve their loading code to wait for the iframe to be loaded in Firefox's case.

(For the record, I think they're working around bug 1493878).

Comment 42

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

At this point, is this actually a wontfix/incomplete that we could split into several different issues? It sounds like there may be several (smaller) problems than what the bug was originally filed for.

Comment 43

[Mike Taylor [:miketaylr]]

Yeah, we probably need a bug open against hterm before this one becomes actionable for Google. Tom, would you mind filing one?

Comment 44

[Thomas Wisniewski [:twisniewski]]

Done. https://bugs.chromium.org/p/chromium/issues/detail?id=980671

I wonder if we should also try to bump the priority of bug 1493878?

Comment 45

[Mike Taylor [:miketaylr]]

ni? myself to email Google about this, thanks Tom.

Comment 46

[Mike Taylor [:miketaylr]]

Sent an email on our partner list pointing to this bug and the crbug.

Comment 47

[Adam Stevenson [:adamopenweb]]

Followed up on the mailing list.

Comment 48

[Hussain]

please

Comment 49

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

This is now filed internally with Google (as of 9/26), so hopefully there will be some progress soon.

Comment 50

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

Followed up again on the moz-google discuss list.

Comment 51

[Liz Henry (:lizzard Please n-i to RyanVM, jcristau, or pascal)]

The google dev is able to reproduce this on 60.9.0esr but not on 70.0.1. Adrian, would you mind testing this again with 70.0.1 and with the current ESR (68.2)?

Comment 52

[Mike Taylor [:miketaylr]]

I'm not able to reproduce on https://console.cloud.google.com/home/dashboard?project=tsci-241420&cloudshell=true anymore (I think? I reloaded 10 times and it worked each time, but I can reproduce the bug on the https://hterm.org/ home page (reproduced on the 7th load). Will follow up on the mailing list.

Comment 53

[Hussain]

uBlock origin breaks the console otherwise it works on 72.

Comment 54

[Adrian Florinescu [:aflorinescu]]

(In reply to Liz Henry (:lizzard) from comment #51)

The google dev is able to reproduce this on 60.9.0esr but not on 70.0.1. Adrian, would you mind testing this again with 70.0.1 and with the current ESR (68.2)?

It is WFM on Windows 7 Pro x32 , Windows 10 x64, Mac 10.14 with:
72.0a1 2019-11-06
70.0.1 2019-10-30
68.2.0esr 2019-10-16

FYI, this issue was initially covering the fact that the google console doesn't load at all, so that's what I've checked.
Also, I still see a CSP warning spamming the browser console:

Content Security Policy: The page’s settings observed the loading of a resource at eval (“script-src”). A CSP report is being sent. 11 m=cloudshell:45:55

AFAIK, seems to me this issue has been fixed - guessing on the google side? Should we mark it as such?

Comment 55

[Mike Taylor [:miketaylr]]

Yeah, let's close -- they said they would work on a workaround for 68. It seems like they have. Thanks!