Closed Bug 435814 Opened 12 years ago Closed 10 years ago

Create Mozilla source code xref using dehydra

Categories

(Firefox Build System :: Source Code Analysis, enhancement)

x86
Linux
enhancement
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: humph, Assigned: humph)

References

()

Details

Attachments

(7 files, 5 obsolete files)

I have been working for a while now on creating a complete cross-reference for the Mozilla tree using dehydra--I got the idea from Dave's blog.  My eventual goal is to do full cross-referencing, but for now I'm trying to build the back-end for a complete class/type browser.  

Here is my dehydra script (moz_types.js), a shell script to drive it and do post-processing of the data (build_db.sh), and a sqlite database schema (class_schema.sql):

http://cs.senecac.on.ca/~david.humphrey/tmp/dehydra-xref-scripts-0.1.tgz

If you don't want to build yourself, you can also download a complete database, with error info (more on that below) here (5.5MB):

http://cs.senecac.on.ca/~david.humphrey/tmp/dehydra-xref-db.tgz

I've overcome many edge cases, but I'm hitting a wall now.  I don't know if these are poor assumptions or errors on my part, limitations of dehydra, or something else.  At any rate, I'd be glad for feedback and help moving forward.

The basic algorithm I use is to process each file and generate a .sql file.  These files contain insert and update statements to generate type, member, and inheritance data.  I rely on documented and observed behaviour of deyhdra to sift out types not defined in the source tree (e.g., /usr/..).  At the end of a build, I assemble all the .sql files and |sort -u| them in order to get rid of all the duplicate SQL statements (e.g., inserting type Foo will happen dozens or hundreds of times, but always be the same).

When I'm done, I'm left with a file containing all the lines that didn't work because of primary key duplication (i.e., lines that differ only in terms of loc--something that should happen).  I'm seeing errors in these general families:

* Same name used in more than one type locally (e.g., <anonymous struct>, AutoFD).  I'm not sure how to deal with this, since the type is not in conflict at the level of *this* compilation unit, but is a problem on a global scale.

* Things defined in more than one place (e.g., toolkit and elsewhere)

* CTOR/DTOR macro craziness (e.g., CERTCertificateCleaner(CERTCertificate*&))

* NS_ macro trouble (e.g., NS_IMPL_ISUPPORTS1 and AddRef() defs)

* Static members are messing up my logic about decl/def (e.g., nsCSSRendering.h/cpp).  I see a decl and a def in process_type(), but not for the same file.  For example, I'll see many references to nsCSSRendering::MakeBevelColor(PRIntn, PRUint8, nscolor, nscolor) at /home/dave/mozilla-central/src/layout/base/nsCSSRendering.h:290 when it's being included and used by another file (e.g., nsPresShell, nsPresContext, etc.).  But I'll only see 1 reference to the def in the file where it's actually implemeneted (i.e., /home/dave/mozilla-central/src/layout/base/nsCSSRendering.cpp:395).  I'm not sure how to tell these apart solely on the data available in process_type().

cf.

/home/dave/mozilla-central/src/xpinstall/src/nsJSInstallTriggerGlobal.cpp:158
c.name == "InstallTriggerCheckLoadURIFromScript(JSContext*, const nsAString_internal&)"
c.isStatic == true
c.fieldOf == undefined
c.methodOf == undefined

Taras, Dave, and anyone else who would care to lend a hand, how do I deal with these issues?  Also, what other things do you see in my method that could be improved?
Could you distill these into testcases? Completely minimal testcases would be nice, but if you want, you can just make a minimal dehydra script and we'll run it on moz.
> * Same name used in more than one type locally (e.g., <anonymous struct>,
> AutoFD).  I'm not sure how to deal with this, since the type is not in conflict
> at the level of *this* compilation unit, but is a problem on a global scale.
> * Things defined in more than one place (e.g., toolkit and elsewhere)

You could probably key on a name/location pair, instead of just the name: if two anymous classes are declared in different locations, just treat them as different classes.

Alternately if you encounter a conflict, throw both of them away and give up.

> * CTOR/DTOR macro craziness (e.g., CERTCertificateCleaner(CERTCertificate*&))
> * NS_ macro trouble (e.g., NS_IMPL_ISUPPORTS1 and AddRef() defs)

Catch me on IRC: I'm not sure what this means.

> * Static members are messing up my logic about decl/def (e.g.,

Sounds like a bug... can you file a testcase?
I'm with Taras, we can work through these one by one, but we need more detailed explanations in order to understand the problems. I have a few thoughts right away, though:

> * Same name used in more than one type locally (e.g., <anonymous struct>,
> AutoFD).  I'm not sure how to deal with this, since the type is not in conflict
> at the level of *this* compilation unit, but is a problem on a global scale.

I concur with bsmedberg: easy answer is to throw away anonymous structs, more complete solution is to give them unique names, based on location or contents.

> * Things defined in more than one place (e.g., toolkit and elsewhere)
> * CTOR/DTOR macro craziness (e.g., CERTCertificateCleaner(CERTCertificate*&))

Not sure what these two mean: please post more details.
 
> * NS_ macro trouble (e.g., NS_IMPL_ISUPPORTS1 and AddRef() defs)

I think you are saying that the macro expansion contains method definitions? Personally, I think it's OK to count them as method definitions. A really advanced version would count them as definitions, show in the UI that they are in a macro expansion, and be able to show the results of the expansion. 

Knowing that they're part of a macro expansion is hard. I think you would have to do something like use libcpp to find where the macro invocations are located, and cross-reference that with the method locations. You could also try something simpler for now, like grepping for a few known macros.

> * Static members are messing up my logic about decl/def (e.g.,
> nsCSSRendering.h/cpp).  I see a decl and a def in process_type(), but not for

I think I know the answer to this one, because I ran into a related problem with GCC attributes a few weeks ago. As GCC parses a file, it "merges" matching declarations and definitions. (Recall that there for a given function, there can be 0 or more declarations and 0 or 1 definition.) So in the typical case of a declaration in a .h file and a definition in a .cpp file, you will not see more than one record for a given method, even if both are in the compilation unit (as with nsCSSRendering.cpp in your example). 

Clearly, for our applications we would like to see all the decls and defs, but I think what GCC gives us is good enough. In your example, you can collect all the Dehydra records for nsCSSRendering::MakeBevelColor. If any two records have the same location, you know they refer to the same decl/def. For a given location, if it is ever referenced in a record from process_function, you know that location is a definition. If none of references to that location were seen through process_function, then that location is a declaration (actually, I believe it is possible but unlikely for that to be wrong--we can fix it if it proves necessary.) By collecting and classifying all records from every compilation unit in this way, I think you can get everything.

If that doesn't work--let me know. I've already patched the merge code, so I have a fair idea of how to make all the decl/defs visible.

The above may not be super-understandable; I may have been too concise. If so, it's my fault--just ask more questions.
* repeated type names in different places in the tree (e.g., AutoFD, <anonymous struct>).

I can't key on loc, since I rely on the fact that |sort -u| can throw away all the garbage (i.e., repetitious uses/callers).  As soon as I start saving loc, I'll get type foo occurring in hundreds of places.

For <anonymous *> stuff, I can throw things away, sure.  Should I also be throwing things like this away then?

Assertion::<anonymous union>::as

With real named types, where the type is used in more than one place, is throwing it away smart?  Here's a typical case:

testing/mochitest/ssltunnel/ssltunnel.cpp
.name=AutoFD  .loc=/home/dave/mozilla-central/src/testing/mochitest/ssltunnel/ssltunnel.cpp:89 .kind=class

toolkit/mozapps/update/src/updater/updater.cpp
.name=AutoFD .loc=/home/dave/mozilla-central/src/toolkit/mozapps/update/src/updater/updater.cpp:165 .kind=class



* Macro Hell.  Consider the case of security/manager/ssl/src/nsCertOverrideService.cpp

I see Duplicate type entries based on loc:

insert or abort into types ('tname','tloc','tkind') values('CERTCertificateCleaner','/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64','class');
insert or abort into types ('tname','tloc','tkind') values('CERTCertificateCleaner','/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertTree.cpp:65','class');
...x17 more

and duplicate members, again, based on loc:

insert or abort into members ('mtname','mname','mdecl') values('CERTCertificateCleaner','operator=(const CERTCertificateCleaner&)','/home/dave/mozilla-central/src/security/manager/ssl/src/nsCMS.cpp:57');
insert or abort into members ('mtname','mname','mdecl') values('CERTCertificateCleaner','operator=(const CERTCertificateCleaner&)','/home/dave/mozilla-central/src/security/manager/ssl/src/nsIdentityChecking.cpp:64');

Dehydra sees this:

process_function() -  .name=CERTCertificateCleaner::CERTCertificateCleaner(CERTCertificate*&) .loc=/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64 .isStatic=true .isConstructor=true
process_function() -  .name=CERTCertificateCleaner::~CERTCertificateCleaner() .loc=/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64 .isStatic=true
process_function() -  .name=CERTCertificateCleaner::detach() .loc=/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64 .isStatic=true
process_type()     -  .name=CERTCertificateCleaner .loc=/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64 .kind=class
...print_members() -  .name=CERTCertificateCleaner::CERTCertificateCleaner(const CERTCertificateCleaner&) .loc=/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64 .isConstructor=true
...print_members() -  .name=CERTCertificateCleaner::CERTCertificateCleaner() .loc=/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64 .isConstructor=true
...print_members() -  .name=CERTCertificateCleaner::operator=(const CERTCertificateCleaner&) .loc=/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64
...print_members() -  .name=CERTCertificateCleaner::CERTCertificateCleaner(CERTCertificate*&) .loc=/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64 .isStatic=true .isConstructor=true
...print_members() -  .name=CERTCertificateCleaner::~CERTCertificateCleaner() .loc=/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64
...print_members() -  .name=CERTCertificateCleaner::detach() .loc=/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64
...print_members() -  .name=CERTCertificateCleaner::object .loc=/home/dave/mozilla-central/src/security/manager/ssl/src/nsCertOverrideService.cpp:64


The reason is the macro used here:

nsCertOverrideService.cpp:64
NSSCleanupAutoPtrClass(CERTCertificate, CERT_DestroyCertificate)

which is...

from security/manager/ssl/src/nsNSSCleaner.h
/*
  NSSCleanupAutoPtrClass(CERTCertificate, CERT_DestroyCertificate)

  will produce:

  class CERTCertificateCleaner
  {
  private:
    CERTCertificateCleaner(const CERTCertificateCleaner&);
    CERTCertificateCleaner();
    void operator=(const CERTCertificateCleaner&);
    CERTCertificate *&object;
  public:
    CERTCertificateCleaner(CERTCertificate *&a_object)
      :object(a_object) {}
    ~CERTCertificateCleaner() {
      if (object) {
        CERT_DestroyCertificate(object);
        object = nsnull;
      }
    }
  };
...
*/
#define NSSCleanupAutoPtrClass(nsstype, cleanfunc) \
class nsstype##Cleaner                             \
{                                                  \
private:                                           \
  nsstype##Cleaner(const nsstype##Cleaner&);       \
  nsstype##Cleaner();                              \
  void operator=(const nsstype##Cleaner&);         \
  nsstype *&object;                                \
public:                                            \
  nsstype##Cleaner(nsstype *&a_object)             \
    :object(a_object) {}                           \
  ~nsstype##Cleaner() {                            \
    if (object) {                                  \
      cleanfunc(object);                           \
      object = nsnull;                             \
    }                                              \
  }                                                \
  void detach() {object=nsnull;}                   \
};



* Static members.... Dave, you were right.  I did some more debugging, and realized that if I ignore the DECL seen in the compilation unit (which will be true only when .isStatic), and instead take one of the ones I get from files that link to it (where .isStatic is missing), it works.  So statics are working now, where type:: name exists.

Having said that, I still need to figure out how to deal with the case that type:: is absent, as in:

/home/dave/mozilla-central/src/xpinstall/src/nsJSInstallTriggerGlobal.cpp:158
c.name == "InstallTriggerCheckLoadURIFromScript(JSContext*, const
nsAString_internal&)"
Having fallen victim to bsmedberg's soothsaying wrt keying on name/loc and the use of .methodOf, I did a run across the tree and found that 35% of members (1650072) report no .methodOf.  Some of these are data members, where .methodOf makes no sense (I need something equivalent, though), others are statics, structs etc.  I've filed 436511 in order to address this.
Depends on: 436511
As per bug 436511#c6, I need to come up with a solution for file-scope static functions, for example:

layout/base/nsCSSRendering.cpp:1817
static void
MakeBorderColor(gfxRGBA& color, const gfxRGBA& backgroundColor,
BorderColorStyle bpat)
{

If I key on name/loc, I have ~7,000 cases like MakeBorderColor that have no containing member info; and yet I need something to key on.  Should I use sys.aux_base_name, which would be confusing in the case where the filename and a class collide (happens often).

I have another problem to solve.  I have added post-processing to follow links from dist/include in order to map .h files back into the source (which works).  I do a number of tricks to end-up with source vs. dist/include loc.  However, I need to deal with .h files autogenerated from .idl.

I could just take dist/include paths for idl generated types.  However, this is somewhat problematic as the .h is not what is really in the tree.  I'm wondering if hacking xpidl to generate mappings back onto loc info in the idl is something I should explore?  In this case I'm not necessarily looking to reproduce what mxr does; I'm wondering if we can do better.

Otherwise, I'm getting pretty close now.  With bsmedberg's patched dehydra, I no longer have to worry about type collisions except in rare cases (4 to be exact).  For example, more than one anonymous inner type:

insert or abort into members ('mtname','mtloc','mname','mdecl') values('IconDirEntry','/home/dave/mozilla-central/src/modules/libpr0n/decoders/bmp/nsICODecoder.h:62','<anonymous>','/home/dave/mozilla-central/src/modules/libpr0n/decoders/bmp/nsICODecoder.h:70');
insert or abort into members ('mtname','mtloc','mname','mdecl') values('IconDirEntry','/home/dave/mozilla-central/src/modules/libpr0n/decoders/bmp/nsICODecoder.h:62','<anonymous>','/home/dave/mozilla-central/src/modules/libpr0n/decoders/bmp/nsICODecoder.h:74');

based on...

struct IconDirEntry
{
  PRUint8   mWidth;
  PRUint8   mHeight;
  PRUint8   mColorCount;
  PRUint8   mReserved;
  union {
    PRUint16 mPlanes;   // ICO                                                                                                                                                  
    PRUint16 mXHotspot; // CUR                                                                                                                                                  
  };
  union {
    PRUint16 mBitCount; // ICO                                                                                                                                                  
    PRUint16 mYHotspot; // CUR                                                                                                                                                  
  };
  PRUint32  mBytesInRes;
  PRUint32  mImageOffset;
};
Dave, what table are you trying to insert these non-member functions into? It looks like you're trying to insert them into 'Members' which doesn't make much sense. But if you really want to insert them into members...

There are two types of non-member functions: static (file-scope) and normal. I think static functions should be

INSERT INTO members ('mtname', 'mtloc', 'mname', 'mdecl')
VALUES ('layout/base/nsCSSRendering.cpp', NULL, 'MakeBorderColor(gfxRGBA& color, const gfxRGBA& backgroundColor, BorderColorStyle bpat)', 'layout/base/nsCSSRendering.cpp:1817')

and non-static functions should be

INSERT INTO members ('mtname', 'mtloc', 'mname', 'mdecl')
VALUES (NULL, NULL, 'MakeBorderColor(gfxRGBA& color, const gfxRGBA& backgroundColor, BorderColorStyle bpat)', 'layout/base/nsCSSRendering.cpp:1817')

This assumes that your DB can index on NULL. If it can't, you'd need to insert a dummy value such as '-'.
Attached patch xpidl-xref patch (obsolete) — Splinter Review
Part of a hack to add an xref backend to xpidl.
Attached file the actual backend file (obsolete) —
Apply the previous patch, then drop this file in xpcom/typelib/xpidl as xpidl_xref.c, and rebuild xpidl. You can then use xpidl -m xref to generate output like this:
/Users/luser/build/mozilla-central/xpcom/base/nsILeakDetector.idl:50 nsILeakDetector::dumpLeaks
Adds a build system hack so you can do mk_add_options MOZ_GENERATE_XREF=1, and you'll wind up with _xpidlgen/nsIFoo.xref files containing the xref output.
Attachment #323376 - Attachment is obsolete: true
(In reply to comment #9)
> Created an attachment (id=323378) [details]
> the actual backend file
> 
> Apply the previous patch, then drop this file in xpcom/typelib/xpidl as
> xpidl_xref.c, and rebuild xpidl. You can then use xpidl -m xref to generate
> output like this:
> /Users/luser/build/mozilla-central/xpcom/base/nsILeakDetector.idl:50
> nsILeakDetector::dumpLeaks
> 

I'm a little puzzled by the need for such functionality. Shouldn't xpidl just put #line markers into generated .h files? Then even gcc would get the correct positions.
Right, but whoever wrote xpidl_header wimped out on that:
http://hg.mozilla.org/mozilla-central/index.cgi/annotate/9b2a99adc05e/xpcom/typelib/xpidl/xpidl_header.c#l1151

So writing that little mapping bit was easier for the time being. :)
> I'm a little puzzled by the need for such functionality. Shouldn't xpidl just
> put #line markers into generated .h files? Then even gcc would get the correct
> positions.

As Ted says, "skipping for now" is the current state of affairs, and this method was within reach.  Having said that, I'm (really) open to suggestions on how to achieve this, as I only know what I want to achieve, but not always how to get there.

With Ted's patch, I essentially have decl.name and decl.loc for all members.  I'm putting these into the db alongside the other types, so that nsIFoo will appear along with nsFoo, and in addition, all its members.  I think it is more helpful to ignore the generated .h completely, since it doesn't exist in the tree, and replace it with links back to the idl--my code uses only source locations, jumping through a few hoops to follow things back from dist/include.  

The only thing Ted's patch doesn't give me now is the name/loc of the interface type itself.  It would be good (especially in idl files that declare more than one interface) to have xpidl emit something like this (all but the first line is what happens now with Ted's patch):

/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:62 nsIAccessible
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:68 nsIAccessible::parent
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:73 nsIAccessible::nextSibling
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:78 nsIAccessible::previousSibling
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:83 nsIAccessible::firstChild
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:88 nsIAccessible::lastChild
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:93 nsIAccessible::children
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:98 nsIAccessible::childCount
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:104 nsIAccessible::indexInParent
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:109 nsIAccessible::name
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:115 nsIAccessible::value
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:120 nsIAccessible::description
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:127 nsIAccessible::keyboardShortcut
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:133 nsIAccessible::defaultKeyBinding
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:141 nsIAccessible::getKeyBindings
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:149 nsIAccessible::role
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:156 nsIAccessible::finalRole
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:168 nsIAccessible::getFinalState
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:173 nsIAccessible::help
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:178 nsIAccessible::focusedChild
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:183 nsIAccessible::attributes
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:194 nsIAccessible::groupPosition
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:204 nsIAccessible::getChildAtPoint
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:209 nsIAccessible::getChildAt
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:214 nsIAccessible::getAccessibleToRight
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:219 nsIAccessible::getAccessibleToLeft
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:224 nsIAccessible::getAccessibleAbove
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:229 nsIAccessible::getAccessibleBelow
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:235 nsIAccessible::getAccessibleRelated
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:240 nsIAccessible::relationsCount
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:247 nsIAccessible::getRelation
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:252 nsIAccessible::getRelations
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:258 nsIAccessible::getBounds
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:263 nsIAccessible::setSelected
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:269 nsIAccessible::extendSelection
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:274 nsIAccessible::takeSelection
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:284 nsIAccessible::takeFocus
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:289 nsIAccessible::numActions
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:294 nsIAccessible::getActionName
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:299 nsIAccessible::getActionDescription
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:305 nsIAccessible::doAction
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:311 nsIAccessible::getNativeInterface

It would also be cool, and I'm not sure how to show it exactly, to list the base interfaces, in this case nsISupports.  Maybe:

loc name [base] ...

/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:62 nsIAccessible nsISupports
(In reply to comment #7)
> Dave, what table are you trying to insert these non-member functions into? It
> looks like you're trying to insert them into 'Members' which doesn't make much
> sense. But if you really want to insert them into members...

Agreed.  For now, and this is easily fixed later, I'm dumping these functions into a funcs table, keeping track of name/loc.

I need another few days to run some more tests, and deal with a few other edge cases, and I'll be putting my script and post-processing stuff up here, before writing a simple front-end so it's easier for people to give feedback.  I really appreciate all the feedback and support.
Ted's xpidl_xref.c backend plus interface and base info.
Attachment #323378 - Attachment is obsolete: true
Attached file build_db.sh
Main script for driving dehydra, generating output, and post-processing results into sqlite.  Requires class_schema.sql, fix_paths.pl, process_xref.pl, dehydra 0.9 (for .memberOf), and Ted's xpidl_xref.c.
Attached file class_schema.sql
SQLite schema for xref db.
Attached file fix_paths.pl
Script to unlink and "genericize" paths returned from dehyrda.
Attached file process_xref.pl
Script to convert xpidl_xref.c output into SQL.
Attached file moz_types.js
Dehydra script to generate sql for xref, including an initial attempt at caller data.
Minor feedback on http://zenit.senecac.on.ca/wiki/dxr/ It would be cool if one could get the full ui when using a direct link to something, e.g. compare http://zenit.senecac.on.ca/wiki/dxr/source.cgi/mozilla/xpcom/build/nsXPComInit.cpp#697 versus http://mxr.mozilla.org/firefox/source/xpcom/build/nsXPComInit.cpp#697
(In reply to comment #22)

It works now, just with a different syntax:

http://zenit.senecac.on.ca/wiki/dxr/?s=mozilla/xpcom/build/nsXPComInit.cpp&l=697

Added method signature info the same as dehydra creates:

foo(bar*, baz**)
Attachment #324231 - Attachment is obsolete: true
I think I'm done getting xpidl to spit out complete dehydra style info from idl.  After talking on #mmgc, I could also have added custom gcc attributes and picked them up via gcc+dehydra; however, I was already done this code, so here it is.  

As an example, this is what I get for nsIAccessible.idl using it (showing type, attrib, method with rest removed):

/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:62 nsIAccessible : nsISupports
...
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:109 nsIAccessible::GetName(nsAString&)
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:109 nsIAccessible::SetName(const nsAString&)
...
nsIAccessible::GetAccessibleToRight()
/home/dave/mozilla-central/src/accessible/public/nsIAccessible.idl:219
Attachment #327054 - Attachment is obsolete: true
fyi, I'm actively working on this and will try to get updated scripts/files up soon.  I've attached some instructions on what I do (slightly out of date, but will work and I'll fix).
I think this bug is no longer useful, and would instead direct people to follow or participate in dxr development, see:

http://vocamus.net/dave/?p=557

Any further discussion on things related to the development of dxr should go to mozilla.dev.static-analysis, or if bug worthy, get added to bug 489834.

Thanks for all the help here as I got this rolling.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.