Closed Bug 524725 Opened 15 years ago Closed 14 years ago

Get DXR hosted by Mozilla IT

Categories

(mozilla.org Graveyard :: Server Operations, task)

Product:
mozilla.org Graveyard
Component:
Server Operations
Platform:
All
Other
Type:
task
Priority:
Not set
Severity:
minor

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: taras.mozilla, Assigned: justdave)

Details

DXR is a replacement for MXR, it's currently hosted on the developer's box at http://australia.proximity.on.ca/dxr/

What do we need to do to get this deployed on a production server?
https://wiki.mozilla.org/IT/Production_Acceptance
Assignee: server-ops → nobody
Component: Server Operations → Webdev
QA Contact: mrz → webdev
I updated above wiki link with production acceptance info, waiting on IT to come up with any further requirements or a deployment timeframe.
Guess this needs some direction from Matthew at this point (sufficient information supplied, go, no go, where to put, etc?)
Component: Webdev → Server Operations: Projects
QA Contact: webdev → mrz
Group: infra
Assignee: nobody → mrz
Even if this is just hosted on a VM, we need somewhere permanent to put DXR. It's important and very useful.
Next couple weeks work for a time line?
Component: Server Operations: Projects → Server Operations
(and that's mostly because we have limited resources for the next week or week and a half)
(In reply to comment #6)
> Next couple weeks work for a time line?

Waiting a few weeks is ideal for me, since I have another release I need to get out, and I won't have time to do it until into Jan.
dave, can this be hosted on the same MXR hardware?  If not, suggest where.
Assignee: mrz → justdave
Can you point me at a page or some info so I know what "MXR hardware" means?  From a processing point of view, DXR is a superset of MXR since I build a full MXR-style glimpse index as part of it.  Basically there are 2 parts to this:

1) glimpse + python-cgi + sqlite + static html.  I pre-generate all the source files' html, and just query the db for searches of semantic data.  I use glimpse exactly like MXR does for plain text searches.  This part doesn't need any sophisticated hardware.

2) The indexer that builds the db, static html, and glimpse index needs a lot of stuff.  It's essentially a modified mozilla build environment using dehydra/treehydra, custom xpidl, etc. with a bunch of post-processing done in bash/python/perl.  This doesn't necessarily need to go on the same box--it could just rsync to the web machine when indexing is done.  On the machine where I do indexing now it takes 2-3 hours for a full run, and it can be CPU and memory intensive.  It might make sense to change my setup so it works like a tinderbox and builds on check-in, then syncs the resulting web output with the web box.

I'm assuming it will need to be modified to fit nicely in your environment (I'm still developing it, and it's not anywhere near version 1.0 yet), and I'm willing to do that.  If an email/irc/call is necessary, I'm happy to do that.
The indexer should run on a dedicated VM starting with the build refplatform image. The webhost can probably share a server with MXR.
sorry, my comments were directed to justdave, Dave Miller, not you.
If it's indexing the same code that MXR currently is, I'm not sure a VM would be enough.  MXR is on a physical box, and it barely keeps up with the indexing as it is.  Then again, it's got about a billion (exaggeration) repositories it's indexing, too.
DXR has to actually do a build in order to index. I expect that it will just sit in a loop, pulling whatever changes are latest in mozilla-central, and building. This will likely take hours. I think the current plan was only to do this for mozilla-central, or perhaps 1.9.2 and -central, not all the trees currently managed by MXR.
Whiteboard: [investigating hardware]
(In reply to comment #6)
> Next couple weeks work for a time line?

Matthew do you have an updated ETA for looking into this bug?
This may or may not be gated on needing hardware and I've asked justdave to figure that out.  We're working on racking another 16 blades this week.

However, this is below getting some of the infrastructure up in Phoenix which should be "real soon now".
We need an update on ETA here, so we can plan DXR work going forward.  Does it need new hardware?
Should have the new chassis up this week and can focus on it next week.  Will that work?
Was hoping to attempt deploying this over the weekend but ended up screwing around with a misbehaving backup server instead.  Hopefully I can get this in today.
Over to justdave - 

dm-dxr01 / 10.2.74.25 is up.
Whiteboard: [investigating hardware]
(In reply to comment #20)
> Was hoping to attempt deploying this over the weekend but ended up screwing
> around with a misbehaving backup server instead.  Hopefully I can get this in
> today.

Database backups continued to misbehave.  The repairs there are mostly done, and probably at a point where someone else can continue if they need too much more, so I'll be back on this later today.
I did get started on this last night, I'm probably halfway to having it working at the moment.  Continuing to work on this today.  Attempting to set up puppet to be able to deploy it (makes it easier to move it around in the future if we need to) while I'm at it.
Current status: I have DXR itself in place, but not hooked into Apache yet. Working on getting Dehydra set up right now.
Took most of the night to compile GCC with the plugin patches.  Got dehydra itself built this morning, but it's failing make check.  People on IRC tell me Taras is the guy to ask about that.  I'm also told it'll probably mostly work despite the test failures, so I'm continuing on working on getting DXR wired into the build now.
Dave, I'm just working now to add some callgraph bits to the dxr repo (they are what I've been using for months in trial on our test installation, but just never committed).  When I'm done you'll be able to update a few of the files and it will do a lot more.  I'll update again soon.
OK, done updating.  The dxr repo contains all the callgraph bits now.
successfully got dehydra built and passing tests by using 1.9.2 branch spidermonkey instead of trunk.
Got slightly sidetracked on this because of the graph server and getpersonas issues (and the usual release stuff because of the major update offer).  Still have some cleanup to do on getpersonas yet, hopefully I will get some more DXR work done tomorrow at some point.
Of course, many other exciting things happened over the weekend and Monday to keep the IT folks busy.  I'm still trying. :)
(bunch of end-of-q1 stuff pushed this back a bit, picking back up)
I'm out on vacation until April 12, this will be on hold until I get back, unless you convince someone else to pick up where I left off.  To the best of my knowledge all the pre-reqs are in place, all that's left is to get it actually building the mozilla code and indexing it, and getting apache to talk to it.

The host box is named dm-dxr01, and almost the entirety of the setup has been getting pushed via puppet, so a good list of where everything is located on the box can be figured out by looking at manifests/classes/dxr.pp in the puppet repo.
Any update on the ETA?  (We might squeak in under the 4-month mark!)
Entirely dropped this coming back from vacation, my apologies :(

I was told humph offered to help out, and I'd love to have it so I'm not driving quite so blind (it seems well documented, it's just a lot to learn from scratch since I don't typically do my own Mozilla builds and so forth), so I've put his ssh key from LDAP on the mozbuild@dxr.mozilla.org account.  He should be able to log in and poke around at what's there.  Then he can perhaps help me out a little instead of me trying to learn the whole thing from scratch as I go (and probably show me what I'm doing while he's at it).
Great!  I have end of term craziness this week, and am travelling/speaking next week.  After that I can focus on this.
(In reply to comment #35)
> Great!  I have end of term craziness this week, and am travelling/speaking next
> week.  After that I can focus on this.

Are you back now?
Back but slammed with a few things.  Maybe we can connect on irc next week and discuss?
david & dave - can you two sync up on irc or phone or whatever to get this wrapped up?  Feels stalled.
FYI, I'm on this box now, and working on this today/tomorrow.  I'll try to connect with Dave on irc (or vice versa).
Spent time on this yesterday with Dave, doing more this afternoon.  I'm currently trying to convince treehydra to run on this box, and right now treehydra is winning this argument.
I've now got a working build/indexing toolchain setup, and it's doing a full index of mozilla-central.  I had to use a very old treehydra, so I might try and update that next week.  The build is not as fast as I'd like (not sure what the hardware is), but I'll tweak what I can.

I'll connect with Dave next week to talk about the web-facing stuff.
Finally finished building/indexing on this box:
 
real    429m34.836s
user    601m25.441s
sys     222m50.737s

That's the longest I've ever seen it take (~2.5x longer than on my desktop).  What is the hardware on this box?
[root@dm-dxr01 ~]# dmidecode | grep DL
        Product Name: ProLiant DL385 G2

4GB RAM, 2x Dual-Core AMD Opteron(tm) Processor 2216 HE .

What do you actually need?  Hardware was never defined and we went with what we had in inventory.
It's going to be materially more resource intensive per-index than MXR; I would say 12G and 4 modern cores, as a somewhat conservative guess.
Rebuilding on https://inventory.mozilla.org/systems/show/1587/ .

    HP - BL460c G6 (1x L5520 12GB)

Hoping that once this box is up it'll be a quick turnaround to get it doing work.
Box is up, needs firewall rules.

dmoore, phx-fw help please?
Assignee: justdave → dmoore
dp-dxr01.phx.mozilla.com / 63.245.216.215
Assignee: dmoore → justdave
Box is in puppet.  I lost track of what has to happen to get the right keys in here so back to justdave.
David H - you all set here?
I've never heard more about the new box.  Is it accessible to me?  If so, I can set it up again.  Let me know.
Yep, sorry - lost communication somewhere.  I believe you can use the same login credentials you used in before to:

dp-dxr01.phx.mozilla.com / 63.245.216.215
dh - you're able to get in?
I'm winding down a whole bunch of other Mozilla projects so I can focus on dxr full-time in july/aug.  As such I've been struggling to find large blocks of time, but I've got some booked tomorrow.
Component: Server Operations → Server Operations: Projects
justdave, I need some stuff on this box I can't do.  When you're on irc, can you grab me?
Can't tell if you guys got together on irc.  Can you list out what you're stuck on?
OK, I'm now off other work, and focused on DXR fulltime for the next 2 months.  Dave, can you ping me on irc so we can finish this?  I think I just need you to make me a web dir and I can do the rest.  Thanks.
justdave responded on irc today, and I am now able to get this setup.  I hope to have it running soon.
moving this out of Projects so I'll actually notice the bugmail in the future (since we're currently working on this now)
Component: Server Operations: Projects → Server Operations
justdave has helped me throughout the day and dxr.mozilla.org is setup (db/html isn't there yet, just getting first index built now).
This is now working.  The dxr instance is running at http://dxr.mozilla.org, and I've also thrown the dehydra/jshydra debugging tool up at http://dxr.mozilla.org/viewsource.  There are a few content issues I've noticed and am working on, but these are just bugs in code vs. server setup/config.

Anything I hit from this point forward, I'll do as a separate bug.  I plan to update this box in sync with my work through August so people can test things.  When I get to the end of August, I'll loop back with IT folks to figure out the right way to leave the config for running on its own.
ok, we'll consider this resolved then, and deal with anything else that comes up as new bugs.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.