733534 - Make VNC work properly on OS X Mountain Lion

Status: RESOLVED WONTFIX

(Infrastructure & Operations :: RelOps: Puppet, task, P4)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

John left this at "it's complicated", I think because of questions about where the X server might run -- within or outside the mock.

This is not a B2G priority.

Comment 1

[Kim Moir [:kmoir] ET]

I've read through all JohnFord's documentation about the Mock environment.  However, I'm not sure that I have enough context to make a decision on where the vnc server should reside.  Thoughts?

Comment 2

[Kim Moir [:kmoir] ET]

I talked to coop about this and he thought catlee would have some insight on the best location for vnc in puppet so I'm cc'ing him for his insight :-)

Comment 3

[Chris AtLee [:catlee]]

Currently the mock slaves aren't running X at all, so nothing to VNC into.

Once we move linux desktop builds over to these slaves we'll need Xvfb running for PGO builds. I'm not convinced we need vnc running all the time on the build slaves though. We rarely use VNC for linux builders. So I'd leave the mock slaves as-is for now.

Once we move other platforms like OSX or test slaves over, we will need VNC running.

The VNC package itself should be installed via a 'packages::vnc' class implemented in modules/packages/manifests/vnc.pp. The VNC service could be run via the supervisord module I'm working on in bug 759974.

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

So let's call this WONTFIX.  We can re-open for new platforms, or just implement it.

Comment 5

[Dustin J. Mitchell [:dustin] (he/him)]

We should ensure this on Darwin.

Comment 6

[Dustin J. Mitchell [:dustin] (he/him)]

The kickstart command for what we want is

  /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -configure -activate -access -on -restart -agent -privs -all -allowAccessFor -allUsers

..but that's a Perl script, so I bet we can do that more efficiently.

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

-activate:
 enable service com.apple.screensharing

-access -on - included in ARD_AllLocalUsersPrivs, below

-privs -all:
 defaults write ${TargetDisk}Library/Preferences/com.apple.RemoteManagement ARD_AllLocalUsersPrivs -integer 0x400000ff

-allowAccessFor -allUsers:
 defaults write /Library/Preferences/com.apple.RemoteManagement ARD_AllLocalUsers -boolean TRUE

Comment 8

[Dustin J. Mitchell [:dustin] (he/him)]

(In reply to Dustin J. Mitchell [:dustin] from comment #7)
> -privs -all:
>  defaults write ${TargetDisk}Library/Preferences/com.apple.RemoteManagement
> ARD_AllLocalUsersPrivs -integer 0x400000ff

kickstart actually uses the decimal equivalent, but after a bit of testing, kickstart's settings don't have any effect.  I'm guessing this is untested and represents a bug in Mountain Lion.  So I just won't touch this key.

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: bug733534.patch

Comment 10

[Dustin J. Mitchell [:dustin] (he/him)]

This still isn't working.  I don't recall all of the history yet, but we should finish it up.  This is what the relevant portion of modules/vnc/manifests/init.pp looks like right now:

        Darwin: {
            osxutils::defaults {
                # kickstart -configure -allowAccessFor -allUsers
                ARD_AllLocalUsers:
                    domain => "/Library/Preferences/com.apple.RemoteManagement",
                    key => "ARD_AllLocalUsers",
                    value => "1";
                # kickstart -configure -privs -all -access -on
                # (this doesn't work; see https://bugzilla.mozilla.org/show_bug.cgi?id=733534#c8)
            }
            service {
                # kickstart -activate
                'com.apple.screensharing':
                    enable => true,
                    ensure => running,
            }
            $kickstart = "/System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart"
            exec {
                #allow builder user to connect via screensharing
                "enable-remote-builduser-access":
                 command => "$kickstart -configure -allowAccessFor -specifiedUsers; $kickstart -activate -configure -access -on -users $::users::builder::username -privs -all -restart -agent -menu",
                 refreshonly => true;
            }
        }

you'll note that the kickstart command never runs -- it's refreshonly, but nothing refers to it.  So that might be part of the problem.

kickstart -activate -configure -access -on -restart -agent -privs -all -clientopts -setvnclegacy -vnclegacy -yes -setvncpw -vncpw ...

seems to work properly when run from the command line, although once you authenticate it does drop you into a GUI login screen.

Comment 12

[Kim Moir [:kmoir] ET]

I tried removing the refreshonly yesterday in bug 848148 but it still didn't work.  I was able to authenticate via vnc when running the command from the ssh session, but not when this was updated via puppet.

Comment 13

[Dustin J. Mitchell [:dustin] (he/him)]

##osx-server:

16:42 < rtrouton> Ah, VNC is broken in Mountain Lion.
16:43 < rtrouton> If you need VNC, as opposed to using the Apple Screen Sharing client, I recommend installing RealVNC.

I like that idea

Comment 16

[Armen [:armenzg]]

I made this work for me by manually enabling legacy VNC and setting the correct password:
https://wiki.mozilla.org/User:Armenzg/VNC#For_Linux_clients

That page is WIP and only verified some cases by Linux and some with Mac.

Comment 17

[Armen [:armenzg]]

I believe that using Apple's Remote Desktop is behind all of our issues.

https://bugzilla.mozilla.org/show_bug.cgi?id=1035311#c8

Comment 18

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Just to add what I also send to the release mailing list when I was asked for...

> In case of our Mozmill-CI machines we can perfectly login via VNC and
> screen sharing after running the following commands on the slave machines:
>
> 1. Enable Remote Management (with user password)
>
>     sudo
> /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart
> -activate -configure -access -on -clientopts -setvnclegacy -vnclegacy
> yes -clientopts -setvncpw -restart -agent -privs -all -vncpw %password%
>
> 2. Allow VNC to directly connect to the console without any extra
> password check (10.7 and higher)
>
>     sudo defaults write /Library/Preferences/com.apple.RemoteManagement
> VNCAlwaysStartOnConsole -bool true

Comment 19

[Dustin J. Mitchell [:dustin] (he/him)]

So, I think releng has survived with this for long enough that we don't really care to fix it, but it's important for QA.  So, Henrik, if you can figure out a fix that works with bot screen sharing and VNC, for all OS X versions, then more power to ya!  Otherwise, WONTFIX is fine.

Comment 20

[Justin Wood (:Callek)]

fwiw I *do* get bit by this ALL the time, releng has only survived without because everytime we have needed screen-access to an OSX host we have bugged the people who have had a Mac to use screen sharing to do so.

This is something I feel we do need, but as of yet hadn't figured out, even when I did spend a few days on it. So c#18 does sound promising.

Comment 21

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dustin, do you have a releng machine for me for testing? I kind wanna make sure that I don't regress anything. Thing is that nowadays the ping times here from Europe are pretty bad, and working with VNC in general is far away from being ideal.

Comment 22

[Dustin J. Mitchell [:dustin] (he/him)]

We can put one of the relabs minis into the QA VLAN for a bit, sure.

Comment 23

[Henrik Skupin [:whimboo][⌚️UTC+2]]

That would be great. I will be around the next 3 days before I have to leave again for a week.

Comment 24

[Dustin J. Mitchell [:dustin] (he/him)]

Amy points out that doing so would require a physical move of the mini, which is a good bit of work.  We might be able to reimage the a mini in relabs, but I had trouble trying to do so a while back.

Comment 25

[Henrik Skupin [:whimboo][⌚️UTC+2]]

And I assume the Mini we currently have in use for deploystudio testing, can't be re-imaged with a releng system? Something else what we could do is that I prepare the patch, and one of you is testing in on a releng box. I can make sure it works for our machine. How does that sound? Lesser work from you all involved.

Comment 26

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Ok, so we wont be able to make it properly working. Connecting to a mac machine via VNC will still be way slower as compared to a mac machine. We have seen constant decreases of speed from 10.6 -> 10.7 -> 10.8 -> 10.9. The latter is really the worst.

So with my patch will allow us at least to connect to mac machines from Windows and Linux machines. You can control the mac, and I think that is what you cannot do at the moment, right? Or what is the exact current state we have to get fixed?`

Comment 27

[Henrik Skupin [:whimboo][⌚️UTC+2]]

So I'm a bit buffled now. Please check the following code:
http://hg.mozilla.org/build/puppet/file/03b2a5c6fe7f/modules/vnc/manifests/init.pp#l43

As you can see the kickstart command only gets executed when called via notify. BUT I cannot find any line of code in the whole repository which runs 'notify => Exec["enable-remote-builduser-access"]'.

I'm not sure how far I will come before I have to leave for PTO. But I will try to do some tests until then and will let you know about the status. Maybe Dustin can pick up the bug later.

Comment 28

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Enable VNC v1

Given that I will be on PTO for a full week, I can't come up with the final patch right now. So this is the first patch of a series of two, which we need.

What it is changing is the exec command. Now it gets executed if the ARDAgent process is not running. This is the one for Remote Management. I think we can safely get this landed. It will enable you all to use VNC on a non Mac host.

A follow-up patch will be necessary to simplify all the configuration steps. But that's too much for me today. I think that with part 1 you will be more than happy! :)

Comment 29

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Enable VNC v1.1

Now with a commit message.

Comment 30

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8473740 [details] [diff] [review]
Enable VNC v1.1

Review of attachment 8473740 [details] [diff] [review]:
-----------------------------------------------------------------

It looks like the old version of this had refreshonly => true, but that nothing ever referred to it, so it never ran.  Kim, do you have any recollection of why that was the case?

I don't see how this patch can hurt, anyway!

::: modules/vnc/manifests/init.pp
@@ +44,5 @@
>              exec {
>                  #allow builder user to connect via screensharing
>                  "enable-remote-builduser-access":
>                   command => "$kickstart -configure -allowAccessFor -specifiedUsers; $kickstart -activate -configure -access -on -users $::users::builder::username -privs -all -restart -agent -menu",
> +                 unless => "/bin/ps aux | /usr/bin/grep -v grep | /usr/bin/grep -q 'ARDAgent'";

Another good tactic for omitting the grep process is something like 'grep -q [A]RDAgent' -- the [A] only matches 'A', but doesn't match itself.

Comment 31

[Kim Moir [:kmoir] ET]

My recollection of this bug is that the command would work via the command line and enable vnc access but it didn't work when invoked via puppet. So I'm not sure if this is why it was disabled.

Comment 32

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Before I can re-test in regards of puppet and a restart, I'm blocked with any OS X work by bug 1058559.

Comment 33

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Thanks Kim. So I think that I will get this landed with the proposal from Dustin included. Then it makes it easier to spot for issues during startup, as you mentioned above.

Comment 34

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Enable VNC v1.2

Comment 35

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8481320 [details] [diff] [review]
Enable VNC v1.2

https://hg.mozilla.org/qa/puppet/rev/de963d30f6b7

I hope that I will find the time early next week, to do the remaining work.

Comment 36

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dustin, one question which came up for me last night - yes, I dreamed about puppetagain! :) - was if we probably should behave the same as on Linux, and only enable screensharing if really needed. So we could have two commands like start_vnc and stop_vnc, which configure the ARDAgent and start or stop it. That way we do not check/run that code for each puppet agent invocation.

The above mentioned scripts we could also enable for Linux (and maybe Windows later) so that we have a unique way to enable/disable VNC.

Comment 37

[Dustin J. Mitchell [:dustin] (he/him)]

I like that idea.  Maybe not for Windows (since SSH isn't so reliable there), but definitely for OS X.  It's worked fine in releng for the Ubuntu systems.

Comment 38

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I'm not actually working on that bug anymore. Maybe someone else can finish it up.

Comment 39

[Amy Rich [:arr] [:arich]]

Mountain Lion is going away when we move to cross compiling.