55863 - xpidl/xpconnect needs support for indexed attributes

Status: RESOLVED WONTFIX

(Core :: XPConnect, defect, P4)

Description

[John Bandhauer]

In order to better support arbitrary properties on xpcom objects we should add 
support for indexed attributes. These would be indexed either by wstring or 
PRUint32. They would be declared (something) like:

  [string_indexed] attribute anytype anyname;
or
  [int_indexed] attribute anytype anyname;

They can optionally be [readonly].

The generated C++ header would look like:

  NS_METHOD GetStringIndexedAnyname(PRUnichar* _index, anytype** _retval); 
  NS_METHOD SetStringIndexedAnyname(PRUnichar* _index, anytype* _arg);

The count of '*' is dependent on the type of 'anytype'. We generate 
'GetStringIndexed' just like we generate 'Get'.

I propose that we might support only wstring and PRUint32 indexing. If we 
want support for other types (e.g. string) then we better decide that upfront.

The xpconnect mapping can use the "reference object" stuff. see bug 44997.

Given...

   [string_indexed] attribute wstring bar;

...you would be able to say in JS...

  foo.bar('fred') = 'whatever';
and
  var whatever = foo.bar('fred');

xpconnect can implement the former by using JS_SetCallReturnValue2 to set an id 
like "__xpc_[...method index here...]_bar" (e.g. "__xpc__12_bar"). Then when the 
setter is called it can parse that to figure out which method is to be called 
with what index string (or number as appropriate).

The latter is a simpler and straightforwardd mapping.

The xpidl changes will be easy enough I think. We'll need to dig into the 
typelib spec again and see where we can steal some bits.

Thoughts?

Comment 1

[Mark Hammond [:markh] [:mhammond]]

Is there any reason we can't be more generic, and support arbitary indexes:

[indexed] anytype anyname(in anytype indexname, ...);

This would make the IDL declaration looks more like a function call, borrowing 
its syntax.  You would only support "in" params to such an "indexed" object, 
and insist on at least 1 param

This would allow a spreadsheet:
  [indexed] nsICell cells(in int x, in int y);

And js to say:
  spreadsheet.cells(1,1) = "Hello from js"

If the main motivation for bug 44997 is interop with MS JScript, then I believe 
this will be a requirement anyway.  I know JScript is hooked up to IE via 
ActiveScripting, and I understand that well (I did the Python implementation)

It doesnt seem to be inventing much new IDL syntax, and seems to capture a 
broader range of semantics?

Comment 2

[Arthur Barrett]

Forgot to add myself to the cc: list!

My vote is for a string and numeric index as minimum.  Upwards from there is 
better, but optional, but if Mark has the skills and time to implement the 
other bits....

Comment 3

[John Bandhauer]

Mark, what I'm suggesting is a simple way to support flat named/indexed 
properties.

I think what you are suggesting is a whole 'nother animal.

As far as JS goes, (as I understand it) the engine support for reference type 
access really takes...
  foo.bar('fred') = 'something';
... and interprets it to mean: "tell me the id on foo which I can use to do a 
setProperty". The system I suggest would not actually call the native foo object 
until that setProperty call happens. That is why I suggested encoding the 
'index' param into the 'temporary' property name. Doing this with properties of 
arbitrary types would be messy.

In your example (I think) you are suggesting that this would call 
spreadsheet.cells(1,1) and return an nsICell object to which it would assign the 
string "Hello from js". Or, does this assume that there is some ctor that takes 
a string and creates an nsICell and that we set that? Or am I missing it?

I'm pretty luke warm on the goals of bug 44997. I don't intend to bend over very 
far backwards. What I am interested in is cases where people have some large 
(potentially finite and known) flat set of properties that they want to get/set 
on the interface. These people have the option of custom coding huge interfaces 
with a bunch of methods that look the same but each manipulate one property, or 
writing some indexing scheme of their own and having to use get/set method call 
style in their JS code. I'm really just trying to suggest something with a 
prettier syntax to entice the "huge interface" crowd.

For cases with a higher dimemsionality (or arbitrary param types as indexes) 
then I don't have a problem with people having to use function call syntax...

  spreadsheet.cells(1,1).setWhatever("foo");
or
  spreadsheet.setCell(1,1, new MyCellCtor("foo"));

Comment 4

[Mark Hammond [:markh] [:mhammond]]

Note: My proposal was flawed.  I did not mean to imply a magic call of the 
nsICell interface.  My apologies.  I meant something like:

[indexed] string Cell(in int x, in int y) - implying:

GetCell(int x, int y, char **_result);
SetCell(int x, int y, char *_value);

So I see my proposal as not too far removed from yours at all.  You already 
introduce the new "get/set" style methods, and xpcom already has the syntax and 
mechanism for handling multiple args.  Inventing the new syntax etc almost 
seemed _more_ work - but obviously I don't understand things as well.

I am very luke-warm in the intended goals of this, if restricted to your 
previous comment.  It means that interfaces with a large number of properties, 
all of the same type, can be shoved together this way.  I fear seeing lots of 
JS code:

  // Note the type is defined in the property - hence different
  // indexed properties even for different basic types
  ob.IntProperty("foo") = 3
  ob.StringProperty("bar") = "hi"

When really they are just trying to say:

  ob.foo = 3
  ob.bar = "Hi"

However, if we do support arbitary indexes, I think it would allow capturing 
some nice semantics beyond the hack to squeeze under todays arbitary vtable 
limitation.

Comment 5

[Brendan Eich [:brendan]]

FYI: The JS extension I perpetrated in bug 44997 allows any number of arguments
to the function that returns a reference.  So you can say obj.foo(i) = bar; or
obj.foo(i,j) = bar; or even obj.foo() = bar.  The type of i, j, etc. are not
restricted.

/be

Comment 6

[John Bandhauer]

brendan: I got that. For me *a* factor is that when called it is not clear if 
the function call represents a lhs or rhs. So, the best thing (I think) is to 
encode the params into the id and then do the real work in getProperty or 
setProperty. Restricting type and count simplifies this a lot. A string id is 
simple for this. We *could* root a copy of argv and have the id reference that 
for later use. But what is the lifetime model?

Nevertheless, I think flat property lists add enough to the functionality. I 
think they do 90% of what is missing. I don't agree with Mark's assertions about 
'ob.StringProperty("bar") = "hi"' and 'ob.bar = "Hi"'. I don't think that 
interfaces are likely to want a top level bag of unrelated properties. I think 
that named lists are good:

foo.addresses('Uncle Jim') = new AddressRecord('123 Street','town','ca',90001);
...

foo.colorPrefs('toolbarBorder') = Color.red;
... 

Comment 7

[Alec Flett]

this is cool, I like the prefs usage already.

Comment 8

[Mike Shaver (:shaver emeritus)]

In terms of not knowing whether a given call is to be the lhs or rhs of an
assignment, we have JS_IsAssigning.  Is that not sufficient?

It almost seems like the engine might want to keep the stack-end-as-seen-by-GC
artificially low, until the SETCALL's rval2 has been stored through, so that the
lvalue function doesn't have to explicitly root all of argv.  Blah.

I think I'm with jband, again, though I wonder why we don't just use
INT_TO_JSVAL(methodIndex), rather than consing up and then parsing some magic
string.

Comment 9

[Brendan Eich [:brendan]]

shaver: argv[i] for i in [0,argc) is rooted by definition.

jband: why did you worry about rooting argv?

I think the methodIndex as id is a great idea (of course, I perpetrated DOM 
level 0).  Interfaces cannot have members named by integers (indexes), so there 
is no conflict.

/be

Comment 10

[Mike Shaver (:shaver emeritus)]

But the call to the lvalue function has completed before the
set-through-rval2-named-property occurs, which should unroot argv[[0,argc)] by
adjusting the current stack depth, no?

And when the set _does_ occur, how is the setter to know what arguments were
passed to the lvalue function, to distinguish
  obj.cell(1, 1) = 5;
from
  obj.cell(2, 2) = 5;
?

Comment 11

[John Bandhauer]

I forgot about JS_IsAssigning.

I didn't like the magic string either, but its what I thought of at the time.

I'm realizing now that for "foo.bar(12)" bar already is bound to foo (i.e. foo 
is findable by xpconnect given only bar), so I could just set the return value 
be bar and the ret2 to be argv[0] (aka '12'). That way the property lookup is on 
bar not foo and this leverages the stuff mentioned in bug 41830 to make this 
work the same as foo.bar[12]. Right?

Comment 12

[Brendan Eich [:brendan]]

Duh!  Forgot my own handiwork there for a minute; thanks, shaver.

Recollecting myself...

If we're going to do the fully general, N-ary form, then we do need to make all 
the arguments available, rooted, as the "identifier" of the property being set. 
It would be better to avoid converting the args to a string.  So why not store 
the arguments object in the JSObject wrapper's i'th slot, where i is the method 
index associated with the GetIndexed or SetIndexed C++ method?  Then we can do 
the equivalent of Function.prototype.apply on the native method from the JS get 
or setProperty hook.  INT_TO_JSVAL(i) would be rval2, of course.

There is no pigeon-hole (reentrancy) problem in using the JSObject's i'th 
element, because the JSOP_SETCALL or JSOP_CALL is followed immediately by the 
set or get property op.

In this case, [parameterized] seems like a better IDL "property" than [indexed].

/be

Comment 13

[John Bandhauer]

I think that MarkH set us off to talking not about a more general version of the 
same thing, but about something else entirely. 

What we started talking about is:

 [int_or_string_indexed] attribute type name;

 assignment with foo.bar(x) and now foo.bar[x]

This leverages xpidl attributes and array notation. I think this will be in and 
of itself a *big* win.

The arbitrary parameterized ideas are a completely different thing. They don't 
fit into xpidl attribute declaration syntax and won't map to array notation.

I think we sould do the former and consider the latter as something different 
that we might do in the future. I think this more arbitrary stuff is just not 
that big a win.

Comment 14

[Mark Hammond [:markh] [:mhammond]]

I didnt mean to hi-jack the discussion!  I will refrain from trying to turn 
this into something else.

However, as the proposal stands, I am afraid I dont see any significant win.  
jband's example:

foo.colorPrefs('toolbarBorder') = whatever

Doesnt seem to buy much more than an nsIPreference with a "value" property:

foo.colorPrefs('toolbarBorder').value = whatever

I note that my proposal suffers the same "problem" - it is really just 
syntactic sugar for something that can be fairly reasonably expressed now.  So 
while I can understand your reluctance to expand this to my proposal, the lack 
of generality to this proposal as it stands means I can't see sufficient 
justification for the change.

Whatever you decide, I can promise Python will fully support it tho!  I'm 
having loads of fun playing with this stuff - thanks!

Comment 15

[John Bandhauer]

Mark, I think the mapping to array syntax really pulls this together and makes 
it generally useful. It gives us a well understood and common syntax for 
croos language flat property lists. More importantly, it gives us a simple way 
to both call *and* implement the pattern in the scripting language mappings. 
Instead of having to implement getBar(i)/setBar(i,val) in script to support an 
interface, the script code can just expose a 'bar' property on 'foo' and layers 
like xpconnect can just do an indexed property access on bar.

Also, I'm glad to hear you're enjoying this. It is not always clear if people 
are happy with where this is going or if they are quietly annoyed at the 
decisions made to date.

Comment 16

[Brendan Eich [:brendan]]

I think there is a big win to this [string_or_int_indexed] proposal, now that it
stands alone from any N-ary function call returning a reference type idea (which
I doubt we'll never need).  Instead of foo.bar(i).value = j, which requires that
foo.bar(i) returns an object, we can store a scalar type in foo.bar[i].  Arrays,
what a concept.

Sorry for adding confusion earlier.  Not quite awake today.

/be

Comment 17

[Mark Hammond [:markh] [:mhammond]]

I would just like to be clear:
> we can store a scalar type in foo.bar[i].  Arrays,

Where I am really coming from is that for many languages, this will seem a 
little limiting.  In many scripting languages, it is quite natural to be able 
to index an object with arbitary numbers and types of args.  Indeed, MSCOM 
provides semantics for the same.  Thus:

  ob[x,y]

on either the LHS or RSH is a common idiom in Python, both for native Python 
objects, and for using arbitary MSCOM objects that support these semantics.

So don't get me wrong - indexing is a _great_ idea.  I just have a problem with 
limiting it to a single string_or_int index!

This is my last word of advocacy on the matter, I promise :-)

Comment 18

[John Bandhauer]

mass reassign of xpconnect bugs to [e-mail]

Comment 19

[timeless]

*** Bug 122864 has been marked as a duplicate of this bug. ***

Comment 20

[(not currently active) Ted Mielczarek]

See also bug 677329

Comment 21

[Kyle Huey (Exited; not receiving bugmail, old account, do not use)]

If you want this use WebIDL.