394732 - Create js-friendly API for mozstorage

Status: RESOLVED WONTFIX

(Toolkit :: Storage, defect)

Description

[Axel Hecht]

I'd like to see an object-relational-mapping for js and mozstorage. Say, something like sqlachemy does. Using JS1.7 iterators and generators, that should be totally feasible. That is, for someone that knows SQL, which is not me.

Rich showed interest in my blog, I hope that's the right Rich I'm CCing. CCing jresig for some js library API insight.

This might start out as an extension-provided jsm, given our current 1.9 timeline.

I'm not sure I care about C++, I wouldn't want to do compromises for that, though.

As for the actual API, sqlalchemy uses a lot of named params, would js objects as params be a good replacement? John?

Comment 1

[[e-mail]]

At the very least we should be able to use classinfo to allow access to parameters and results using normal JS object syntax instead of xpcom methods, even if it's only for numeric indices.

Comment 2

[Shawn Wilsher :sdwilsh]

Neil - you already can.  See http://mxr.mozilla.org/seamonkey/source/storage/test/unit/test_storage_statement_wrapper.js

Comment 3

[Rich Dougherty]

Attachment: Translation of SQLAlchemy tutorial v1

Thanks Axel - you got the right Rich.

The attached JavaScript code shows what a fairly faithful port of SQLAlchemy might look like. I've translated the SQLAlchemy tutorial's example Python code into JavaScript code, making changes only where necessary. You can see the original here: http://www.sqlalchemy.org/docs/04/ormtutorial.html.

Some notes:
- As suggested by Axel, objects with properties are passed as a substitute for keyword arguments.
- I haven't used SQLAlchemy before, so I may have misunderstood how some things work. :-)
- SQL relations use explicit functions rather than operator overloading. eg jsa.eq(a, b) rather than a == b.
- Please excuse syntax errors, etc - I'm just posting this code to get the ball rolling.

Axel, is this what you want the ORM library to look like?

Comment 4

[Axel Hecht]

Comment on attachment 279545 [details]
Translation of SQLAlchemy tutorial v1

Sorry, lost track of this in my inbox.

Yes, this is pretty much what I had in mind. I just ran across elixir, too, but that may be (as with sqlachemy) be a solution on top.

I'm really interested in the table creation part and the query_by.

I'd like jresig to look at this, too, from an js-api point of view. John, I'd think that filter_by with kwargs is fine if filter() works like js array extras? That'd be the kind of wisdom I'd like you to provide.

Basically, we should divert from http://www.sqlalchemy.org/docs/04/sqlalchemy_orm.html#docstrings_sqlalchemy.orm_Query wherever js 1.6-8 array extras suggest so. Right?

Comment 5

[Shawn Wilsher :sdwilsh]

This is on my radar for moz2

Comment 6

[Andreas M.]

Wouldn't it make sense, to add something on top of it like Elixir? 

Also, while doing Django development, I found the Django's query_sets to be very sexy. To get a list of all the items in a table named "News", sorted by the date "created" one creates a (Python)dictionary á lá:

latest_news_list = {
	'queryset'   : News.objects.all(),
	'date_field' : "created"
}

See http://www.djangoproject.com/documentation/db-api/ for documentation on the API, that produces News.objects.all()

I think it may be nice to offer a layer, that takes away verbosity.

Comment 7

[Shawn Wilsher :sdwilsh]

Note: Bug 429986 is being designed such that JS is much nicer to work with (see test cases for sample code).

Comment 8

[Shawn Wilsher :sdwilsh]

bug 452987 helps a bit

Comment 9

[Magnus Melin [:mkmelin]]

sdwilsh: it does? ;)

Comment 10

[David Dahl :ddahl]

Attachment: WIP django.db style javascript ORM

This is no doubt very naive and bug ridden, just trying to flesh out some ideas and functionality I miss from django.db

Joins are not even supported yet, etc... I would like some feedback.

Comment 11

[David Dahl :ddahl]

Attachment: XPC shell test file for django-style js orm

a couple of tests are broken and disabled. very much a WIP.

Comment 12

[Shawn Wilsher :sdwilsh]

(In reply to comment #10)
> Created an attachment (id=366632) [details]
> WIP django.db style javascript ORM 
API seems a bit foreign/strange to me.  Especially the constraints for selects; those seem really unnatural.

I also think it might be more interesting to have Orm (which should be ORM) to take an object, so you can do something like:
new ORM({database: nsIFileRepresentingDB, models: allModels});
- or -
new ORM({inMemory: true, models: allModels});
- or -
new ORM({connection: mozIStorageConnectionToRepresent});

Comment 13

[David Dahl :ddahl]

It is foreign, somewhat Pythonic, very truncated. The docs at djangoproject.com are a quick read:
http://docs.djangoproject.com/en/dev/topics/db/queries/

I like the conciseness, however, the main issue is fast and easy CRUD. I have been writing a lot of Python in the past few years. It does make sense to make this more "Javascripty". I think the onject as argument is smart as well, and I was planning on supporting that as well. If we can only support a single approach, lets use objects as they are more self-explanatory. 

I'll keep hacking on this - I want to add JOINS including 'many to many'.

Comment 14

[David Dahl :ddahl]

Attachment: ORM ideas WIP patch - refactored Field instanciation

Still have NOT addressed sdwilsh's naming conventions - posting for dietrich to check out.

Comment 15

[David Dahl :ddahl]

Attachment: forgot the makefile

Comment 16

[David Dahl :ddahl]

Attachment: Very much WIP, test failures posting for Shawn to peep

sdwilsh: Just curious if there is anything I need to do for the database connection to work in a storage xpcshell test where the sqlite db is on the filesystem. If i run this test with my connection to an "In Memory" database, it works.

Btw: how is the naming and the location in storage?

Comment 17

[David Dahl]

Attachment: WIP - posting latest for adw to peruse

The weird thing is that when you run the test file test_storage_JSORM_field_model.js, certain tests fail that instantiate a new Model(). Usually, it is a test that creates a new Model with a different table and columns as the previous test.  If you run the test by itself, it passes. The "fields" property seems to live on in future objects. SO WEIRD. perhaps I am doing something totally wrong, I do not know.

Comment 18

[Marco Bonardo [::mak]]

Let's be realistic, this is unlikely to happen. The best thing we have is Sqlite.jsm, and if we can get bug 512761 it won't be that bad. We can still improve that clearly.