666613 - Tamarin Foundation Utilities (part 2)

Status: RESOLVED WONTFIX

(Tamarin Graveyard :: Virtual Machine, enhancement, P3)

Description

[Stan Switzer]

Continuing work on the foundation utilities.

Where we are now:

- Platform/vmpi/vmbase can be built and used as a library with no runtime dependencies on the MMgc or AVM core

- Introduced gtest test framework

- "assert" and "static_assert" are now supported as the preferred assertion notations

- Scope of VMPI:
  - threads
  - synchronization
  - thread local store
  - libc abstraction
  - memory
  - logging -- but this really belongs in an instrumentation module
(Basically, core OS system "kernel" functionality)

Cheats:

- Still references the AVM feature system for VMCFG_SAFEPOINTS.
(Besides being a structural dependency outside the module, ifdefs interfere with the ability to deliver the module as a library.)

To Do:

- "Platform" directory properly belongs within VMPI, I believe. Or vice-versa. But they represent the same basic idea. In fact, some VMPI definitions are currently in the platform directory.

- Renaming, perhaps, once the to what and how agreed to

- Public/private header segregation (details to follow in the notes)

- Run gtests tests in the smokes

- Current gtest tests duplicate tests in the older test framework; remove the original (now) redundant tests

- Further gtest tests for VMPI/platform functionality

- remove avmfeature system dependency

----------------------------------------------------------------
+++ This bug was initially created as a clone of Bug #645878 +++

This bug proposes to carve out out a "Foundation Utilities" component, similar to vmbase (or perhaps a refinement to it) so that a common infrastructure for threading, synchronization, etc. can be shared between the VM and clients of the VM.

It is duplicative and probably dangerous for Tamarin and its clients to have different platform implementations of threading and synchronization, so we need to find a supportable way to share these common foundation components.

Nothing currently prevents clients from just calling into vmbase or VMPI, but at present these are understood as components internal to the VM. There is an effort underway to make the VM interface narrower and more encapsulated, so if this is to be allowed the interface should be formalized and not ad-hoc.

Additionally, in order to facilitate better separation between client code that is VM-involved and GC-involved, this set of utilities should be more primitive than and independent of both the GC (and safe points) and of the VM. Specifically, we do not want to force VM or GC dependencies on code which does not need to have any.

The component would include:

   threads (pretty much as-is, but probably renamed)
   locks, monitors, condition variables (pretty much as-is, but probably renamed)
   thread local storage
   assert and static assert

Additionally, since the foundation will serve as a sort of inter-component substrate or vocabulary, it will be very useful to have a generic (non-VM) abstractions for strings and arguably, but far less urgently, vectors and hash maps.

It will probably also be necessary to abstract a “panic abort” protocol from the MMGC out-of-memory machinery.

Comment 1

[Stan Switzer]

Carrying forward some comments from bug 645878, comment 18
    https://bugzilla.mozilla.org/show_bug.cgi?id=645878#c18

> I suppose all of that is pretty uncontroversial except for segregated headers
> and the include pattern. We want to do those things in order to make the
> modular structure evident in the code, to minimize conflicts, to be able to
> enforce the API perimeter, to help enforce proper dependencies, and to make
> {.lib,header} packaging feasible.

Here's what I was thinking (since so far nobody's objected):

 - clients include by writing #include <vmbase/VMThread.h>
 - compiler -I flags should be able to select in-situ set of headers for the vmbase module or a copy thereof.
 - With suitable -I flags, clients should be able to include public headers but not private headers
 - With suitable -I flags, clients should be able to select just vmbase and not other tamarin components so that attempts to include them would fail.

Unfortunately doing all of that leads to some funny looking directory structures. This isn't the only conceivable organization, but it fills the bill:

root/
  include/  -- LLVM does something like this
     vmbase/  -- So that you can specify this dir on the compiler's -I directive
        vmbase/  -- what, again? Yeah, because the module name's in the #include
           VMThread.h  -- etc.

OK, so this has issues: The apparent stutter in the file paths. That you have to (and get to) specify each module in the compile command. That's so that the compiler can enforce your intended dependencies. And, finally, well... we use so many inlines, etc. that there might not be many definitions that can even be left out of the public header files. They might all end up over there anyway, and that would defeat the point of separating public and private headers.

I want to keep this as simple as possible. Thoughts?

 Edwin had comments:
  https://bugzilla.mozilla.org/show_bug.cgi?id=645878#c20
  https://bugzilla.mozilla.org/show_bug.cgi?id=645878#c21

Comment 2

[Stan Switzer]

Actually, the structure I meant to suggest was:

root/
  modulename
    sources and private headers in a structure I don't care about
    include/     -- this is used in the -I options
      modulename/    (yeah, again)
        a_public_header.h
        another_public_header.h

But anyway, the point is to have a directory you can point -I to that contains only module/header.h elements. How this is achieved is a matter of pragmatics.

Comment 3

[Stan Switzer]

I view this as part of the effort to set the pattern for ANI, which shares the goal of enforceably distinguishing public interfaces from private ones. It would (eventually) be a pattern used throughout AVM and elsewhere.

Comment 5

[Dan Smith]

Retargeting to Brannan.

Stan, what's the ETA for landing?