1306989 - Generic Worker task user temp and appdata environment variables are incorrect.

Status: RESOLVED FIXED

(Taskcluster :: Workers, defect)

Description

[Rob Thijssen [:grenade (EET/UTC+0300)]]

The generic-worker should explicitly set the values of APPDATA, LOCALAPPDATA, TEMP, TMP and USERPROFILE to directories under the users home path as configured under generic-worker.config.usersDir and create those directories so that they are available to tasks.

Comment 1

[Pete Moore [:pmoore][:pete]]

Note, to solve this properly I'll need to get to the bottom of why these env vars get set incorrectly. Currently they are not explicitly set, they are set as a result of syscalls that are made. Presumably one or more syscalls has a problem, and I think it would be best if possible to correct those, rather than hardcode these env vars, just because there might be subtle differences (like the env vars might be correct, but maybe registry entries are wrong).

Bug 1306992 offers a workaround though, until this bug is solved.

Comment 2

[Rob Thijssen [:grenade (EET/UTC+0300)]]

I think for our purposes (speed/cost) it's enough that we just set the variables and create the directories. The only problem we're trying to solve here is that we read and write temporary data from and to the ephemeral volume during the course of the build/test.

From what we've seen, in-tree code uses the values of the variables listed above to decide where to read and write temp data. If we can get those variables set to folders on the ephemeral drive, we should be good.

For the time being, I will set the variables in the task definition, but it would be great if the generic worker could do this under the covers.

Comment 3

[Rob Thijssen [:grenade (EET/UTC+0300)]]

these builds should demonstrate that having the vars set and directories created puts temp data on the correct drive: https://treeherder.mozilla.org/#/jobs?repo=try&revision=26774c817292e739ea5838be18f74e1255ecd2fe

Comment 4

[Pete Moore [:pmoore][:pete]]

I've established that this is caused by the user creation procedure.

Originally we were calling:

net user <user> <password> [options...] /homedir:<homedir> /profilepath:<homedir>

and then we were using LogonUser system call.

A while later, we added a system call to LoadUserProfile.

It turns out, now that we are calling LoadUserProfile, we no longer need to specify a homedir and profilepath when calling `net user`. Doing so causes the system to think that the profile already exists locally, and then it fails to find it. By not specifying a homedir or profilepath, the loaduserprofile system call recognises that the profile does not exist locally, and creates it.

Testing locally on windows 2008 I was able to reproduce the issue when including the homedir and profilepath parameters in the `net user` call, and able to fix the env vars by not including them.

Note, this does mean that the user home directory will resolve under the system default location (typically C:\Users). However, we are not forced to use the user home directory for the task directory - so even though the user's registry hive and other files may be created under the home directory, we can continue to use another drive / location for task.

It would have been nicer to have this all in one place, but it doesn't seem easy to switch. Changing the default user profile path is only supported when changing at installation time of the operating system, which we do not do. See https://support.microsoft.com/en-us/kb/949977

Therefore I think the best is to have the user profile in the default location, and the task directory in a custom location, and accept that the user might be writing to more than one drive, and that we will have two drives to clean up between tasks.

Comment 5

[Pete Moore [:pmoore][:pete]]

Note, CreateProfile also does not let you specify a location, so it isn't like we can actively create a profile, and then continue to specify the location in the `net user` command.

https://msdn.microsoft.com/en-us/library/windows/desktop/bb762271(v=vs.85).aspx

Comment 6

[Pete Moore [:pmoore][:pete]]

This test demonstrates that a temp profile is no longer created / used:

https://public-artifacts.taskcluster.net/b6-z8P0HQt2WwBZ-EbzHZw/0/public/logs/live_backing.log

We now allow the user home directory to be created on C: drive, but the task is executed on Z: drive. This is because the only supported way of setting the user profile directory not to be on C: drive, is to do so during system installation, which we are not in a position to do. See, for example https://support.microsoft.com/en-us/kb/973289:

> The only supported method for customizing the default user profile is by using
> the Microsoft-Windows-Shell-Setup\CopyProfile parameter in the Unattend.xml
> answer file. The Unattend.xml answer file is passed to the System Preparation
> Tool (Sysprep.exe).

Also in this article (Article ID: 973289 Revision 16):

> If you try to use other methods to customize the default user profile, it may
> result in extraneous information being included in this new default user
> profile. Such extraneous information could lead to serious problems with
> applications and system stability.

So I think this is the only feasible solution - to separate home directories from the task directories (so a given task user can write both to its own home directory on C: drive, and also typically a task directory on the Z: drive).