916499 - [jsdbg2] "Debugger scope is not live" error when doing fct.environment.getVariable("arguments")

Status: NEW

(Core :: JavaScript: Debugger API, defect)

Description

[David Bruant]

* Create a script with a function
* Find this function in a debugger context
* Notice that "arguments" is listed in fct.environment.names()
* Call fct.environment.getVariable("arguments")

Actual:
Error thrown with message "Debugger scope is not live"

Expected:
not sure... I feel this happens because the function is not live. Maybe the debugger should consider that the "arguments" implicit variable only exists (and is listed in .names()) when this function is called and has a corresponding live frame?

Comment 1

[Jim Blandy :jimb]

I think this is a case of a confusing design operating as intended.

In general, inspecting environments may throw errors; for example, if one attempts to access a variable whose value was not retained when the closure was constructed, that should throw as well (although it currently returns 'undefined'). In this case, 'arguments' is a variable whose value can only be retrieved while the call for which the environment was created remains on the stack.

So, code using getVariable should always do so in a 'try' block, in case the variable of interest is not available. getVariableDescriptor (not implemented) might be able to provide a less clumsy interface that offers the same information.

Comment 2

[Jim Blandy :jimb]

I should document this better.

Comment 3

[David Bruant]

(In reply to Jim Blandy :jimb from comment #1)
> I think this is a case of a confusing design operating as intended.
> 
> In general, inspecting environments may throw errors; for example, if one
> attempts to access a variable whose value was not retained when the closure
> was constructed, that should throw as well (although it currently returns
> 'undefined')
I don't understand this. Can you provide a debuggee code example?
I (apparently wrongly) assumed the only time getVariable could throw was when a getter was defined on the global object and it would throw DebuggeeWouldRun.


> In this case, 'arguments' is a variable whose value can only
> be retrieved while the call for which the environment was created remains on
> the stack.
I'm not sure I understand the expression "the call for which the environment was created". According to the ES5 spec, a function gets a [[scope]] as soon as it's created (http://es5.github.io/#x13.2 step 9). I imagined it would be the same for environment. I understand the optimization that consist in creating this scope lazily only if the function is called at least once, but that's not something that should be observable by the Debugger API.

Regarding 'arguments', if its value can only been retrieved if the corresponding environment is on the stack, then... wait a minute...

    function f(x){
        if(x.length === 0)
            return '';
        else
            return arguments[0][0] + f(x.substr(1));
    }
    f('whatever');

f is a recursive function. At some point, several frames are created based on calls to f. Each of these frames has arguments. However, there is a unique f.[[Scope]]. Right before "return '';" what is f.environment.getVariable('arguments') supposed to be?
Each frame arguments can be inspected via Debugger.Frame instances, but it looks like it is an abstraction violation to expose "arguments" as an environment implicit variable (regardless of being how authors have the impression to use it).

Based on this analysis, I would propose to not include "arguments" in Debugger.Environment.prototype.names() as well as always returning undefined for Debugger.Environment.prototype.getVariable('arguments'). Alternatively, throw an error suggesting to go look at Debugger.Frame instances .arguments instead.


> So, code using getVariable should always do so in a 'try' block, in case the
> variable of interest is not available.
My use case looks like (inspected is a Debugger.Environment):
    var variableNames = inspected.names();
    variableNames.forEach(v => {
        var value = inspected.getVariable(v);
        // ...
    });
So it was hard to imagine that .getVariable could throw (expect maybe for getters on global)

> getVariableDescriptor (not implemented)
I figured indeed :-p Is there a bug to follow?
No rush anyway, e.names().forEach(v => {... e.getVariable(v) ...}) works fine for what I need to do for now.

I've fixed my code to ignore the "arguments" variable based on the above analysis.

(In reply to Jim Blandy :jimb from comment #2)
> I should document this better.
Don't worry too much, it's already well-documented enough that I can work around limitations as I find them ;-)

Comment 4

[Jim Blandy :jimb]

(In reply to David Bruant from comment #3)
> (In reply to Jim Blandy :jimb from comment #1)
> > I think this is a case of a confusing design operating as intended.
> > 
> > In general, inspecting environments may throw errors; for example, if one
> > attempts to access a variable whose value was not retained when the closure
> > was constructed, that should throw as well (although it currently returns
> > 'undefined')
> I don't understand this. Can you provide a debuggee code example?
> I (apparently wrongly) assumed the only time getVariable could throw was
> when a getter was defined on the global object and it would throw
> DebuggeeWouldRun.

I think that, at the moment, that is true. In the code below, the getVariable('y') call will return 'undefined':

var g = newGlobal();
var dbg = new Debugger;
var gw = dbg.addDebuggee(g);

g.eval('function f(x, y) { gg = y; return function g() { return x; } }');
g.eval('var gg = f(3, 4);');
print(gw.getOwnPropertyDescriptor('gg').value.environment.names())
print(gw.getOwnPropertyDescriptor('gg').value.environment.getVariable('x'))
print(gw.getOwnPropertyDescriptor('gg').value.environment.getVariable('y'))

but I think that is poor behavior; the fact is that SpiderMonkey cannot find the value of 'y'; getVariable should throw an error indicating this.

 
> > In this case, 'arguments' is a variable whose value can only
> > be retrieved while the call for which the environment was created remains on
> > the stack.
> I'm not sure I understand the expression "the call for which the environment
> was created". According to the ES5 spec, a function gets a [[scope]] as soon
> as it's created (http://es5.github.io/#x13.2 step 9). I imagined it would be
> the same for environment. I understand the optimization that consist in
> creating this scope lazily only if the function is called at least once, but
> that's not something that should be observable by the Debugger API.

A function gets a scope when it's created --- but when is that scope created? Each function call creates an environment; see ES5 10.4.3 step 5. Not all environments are created for function calls; calls to eval and with statements create them too. But the environments created for calls can only access their 'arguments' array while that call is actually on the stack.

> Regarding 'arguments', if its value can only been retrieved if the
> corresponding environment is on the stack, then... wait a minute...
> 
>     function f(x){
>         if(x.length === 0)
>             return '';
>         else
>             return arguments[0][0] + f(x.substr(1));
>     }
>     f('whatever');
> 
> f is a recursive function. At some point, several frames are created based
> on calls to f. Each of these frames has arguments. However, there is a
> unique f.[[Scope]]. Right before "return '';" what is
> f.environment.getVariable('arguments') supposed to be?

There is a unique f.[[Scope]], but there is also a fresh enviroment created for every call to the function. Each of those refers to the 'arguments' of the corresponding call.

> Each frame arguments can be inspected via Debugger.Frame instances, but it
> looks like it is an abstraction violation to expose "arguments" as an
> environment implicit variable (regardless of being how authors have the
> impression to use it).

Well, the spec says that it's a variable.

> Based on this analysis, I would propose to not include "arguments" in
> Debugger.Environment.prototype.names() as well as always returning undefined
> for Debugger.Environment.prototype.getVariable('arguments'). Alternatively,
> throw an error suggesting to go look at Debugger.Frame instances .arguments
> instead.

That alternative won't work. If you can get the arguments from a Debugger.Frame, then the frame must be live, and then Debugger.Environment would work just as well.

I think you may be under the impression that 'arguments' can never be accessed in a function's captured enviroment. This is not true; as I said originally, the call for which the environment was created must still be live.

var g = newGlobal();
var dbg = new Debugger;
var gw = dbg.addDebuggee(g);

g.eval('function f(x, y) { function h() { }; debugger }');
dbg.onDebuggerStatement = function (frame) {
  let h = frame.eval('h').return;
  print(h.environment.getVariable('x'));
  print(h.environment.getVariable('y'));

  let args = h.environment.getVariable('arguments');
  print(args.getOwnPropertyDescriptor(0).value);
  print(args.getOwnPropertyDescriptor(1).value);
}
g.f(42, 1729)

For me, this prints:

sergei:dbg$ js/src/obj~/js -f ~/moz/foo.js
42
1729
42
1729
sergei:dbg$ 

> I figured indeed :-p Is there a bug to follow?

Unfortunately, no. It isn't needed at the moment. If we do need it, we'll file the bug.

Comment 5

[David Bruant]

(In reply to Jim Blandy :jimb from comment #4)
> In the code below, the getVariable('y') call will return 'undefined':
oh ok. Thanks for the code snippet!


> A function gets a scope when it's created --- but when is that scope
> created? Each function call creates an environment; see ES5 10.4.3 step 5.
:-s I was confusing environment and scope. Sorry about that. Thanks for the clarification.


> There is a unique f.[[Scope]], but there is also a fresh enviroment created
> for every call to the function. Each of those refers to the 'arguments' of
> the corresponding call.

> Well, the spec says that it's a variable.
Makes sense now that I understand environment and scope are 2 different things...

In the end, is this bug INVALID?

Comment 6

[Jim Blandy :jimb]

No, you were right originally: "environment" and "scope" are essentially synonyms. But environments are objects with dynamic lifetimes, and a function's [[Scope]] property is set to point to an existing environment.

Have you seen my slides at https://github.com/jimblandy/JSTools ? (I guess you'll have to check it out and visit presentation.html on the local filesystem...) That has a walk-through of the evaluation of some code that creates a closure that captures an interesting environment --- and the diagrams are drawn to operate exactly the way ECMAScript says execution should proceed: the environments are exactly ES5 Lexical Environments; the stack frames are exactly ES5 Execution Contexts; and so on.

In ES5, fresh environments are allocated by calls to NewDeclarativeEnvironment or NewObjectEnvironment:  for function *calls* (not creation; but see later), strict mode direct eval, 'with' statements, and catch blocks. 

The only time we create a fresh environment when we're creating an actual function object is when evaluating function expressions, and that's a weird special case. When we're evaluating an expression like '(function f() { ... })', we want the resulting function to capture the enclosing (previously allocated!) environment --- but we also need to ensure that 'f' is in scope in the body. So we allocate a fresh environment, with the enclosing environment as its "outer" environment, and add the single identifier 'f' to it. But again, this is just a minor extension to the enclosing environment, which was allocated for a call or eval or catch block.

Comment 7

[Jim Blandy :jimb]

In ES5, note that the semantics of FunctionDeclaration does *not* create a new environment. Only FunctionExpression with an Identifier does --- and the newly created environment contains only one identifier.

Comment 8

[Jan de Mooij [:jandem]]

Moving bugs to Debugger API component. Use castle-terms-potato to filter out.