550582 - MDC (developer.mozilla.org) is very slow

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: General, defect)

Description

[Ben Bucksch (:BenB)]

MDC (https://developer.mozilla.org) is very slow. Very often, it takes 20s to respond. I expect 200ms, maximum 1s. This slows down work.

Sometimes, I even get "service unavailable" for some pages, reload fixes it.

The "service unavailable" shows is that it's not the pure web server which is the problem. Please upgrade/add server(s) *and* (much more importantly) check out the software layers, how you can cut down unnecessary processing dramatically. Probably there's a huge amount of processing going on that could be removed, somehow - the response times can't be explained in another way.

Comment 1

[Dave Miller [:justdave]]

I restarted the dekiwiki daemon on both webheads.  When this has become an issue in the past that usually fixes it.

I suspect you're absolutely right that there's software issues, but there's not much we can do about that ourselves.  Maybe Sheppy can pester the vendor again. :)

Comment 2

[Ben Bucksch (:BenB)]

Please keep open until the underlying issue is fixed.

Also, response time here is around 2s. As said, *maximum* should be 1s, esp. for a site that's integral in our work (and basically just displays webpages).

Comment 3

[Ben Bucksch (:BenB)]

(Thanks for restarting, BTW. Definitely much better - for now.)

Comment 4

[Dave Miller [:justdave]]

(In reply to comment #2)
> Please keep open until the underlying issue is fixed.

I think you're arguing for the sake of arguing here. I was just poking around at it before you reopened to make sure my restart actually got some speed back, and it seemed plenty responsive to me. We've been through this a dozen times already, there's not much else we can do on our end. If anything further needs to be done it's going to have to be done within the app.

Comment 5

[Ben Bucksch (:BenB)]

> We've been through this a dozen times already

I know :-), I've seen it before, too. That's why I reopened it, because I don't want this to keep coming back.

>  it's going to have to be done within the app.

Yes, that's what the initial description said :).
Thanks for re-assigning, I didn't know the other Product.

Comment 6

[matthew zeier [:mrz]]

(In reply to comment #2)
> Also, response time here is around 2s. As said, *maximum* should be 1s, esp.
> for a site that's integral in our work (and basically just displays webpages).

To be far, basically displaying -dynamically- generated webpages in a C# app.  This is in no way a simple app or something that "just displays webpages" (the dream that would be if it were!).

Comment 7

[Ben Bucksch (:BenB)]

> displaying -dynamically- generated webpages

Not necessarily. The content is basically static. The dynamic generation is just how we chose to do the content management.
You could just as well have them all as static webpages, and generate and update them every time somebody modifies the wiki page.

Comment 8

[Ben Bucksch (:BenB)]

I.e. each time somebody clicks "save", a complete, static page is generated, written to /cached/, and the web server serves from there when somebody requests the wiki "page" (not "edit" mode, history or whatever). I think you could even implement that with little or no modification of the Wiki application.

Comment 9

[Ben Bucksch (:BenB)]

It's very slow again

Comment 10

[Ben Bucksch (:BenB)]

Filed bug 559337 about the static page idea (given that it was ignored here), and closing this.

Comment 11

[Ben Bucksch (:BenB)]

Expected response time: 200ms.
Actual response time: 20000ms or no response at all.

Hello? Is somebody home? MDC is again extremely slow, with failure to respond for >50% of the requests, since about a week at least. I expect such problems to be fixed within the hour, not days or weeks, and not to come back every other week. The situation is unbearable.

Comment 12

[Eric Shepherd [:sheppy]]

This is a known problem and is being worked on. Please see http://www.bitstampede.com/2010/04/23/stating-the-obvious/ for details.

Comment 13

[Ben Bucksch (:BenB)]

That's good to know, but the problem is known since almost 2 months (when I filed this bug).

And I made a suggestion, which is fairly easy to implement, does not depend on proprietary software, and solves the problem for most cases (viewing page): bug 559337. So far, in all bugs I filed, you've only responded to tell me "can't do".

Comment 14

[Eric Shepherd [:sheppy]]

I understand your frustration (trust me, I guarantee I'm more frustrated than you are). But things have taken a recent turn for the worse in terms of performance and we're working to understand why.

It's not a matter of "can't do". It's a matter of "it doesn't make sense to do that right now."

Comment 15

[Ben Bucksch (:BenB)]

> I guarantee I'm more frustrated than you are

OK, I will not add any frustration, then. Good luck!

I still do think that my caching suggestion is the fastest solution to your current problems, as that can be implemented in a matter of hours, without any dependence on MindSource vendor.

Comment 16

[Mike Shaver (:shaver emeritus)]

We are not dependent on the vendor unless we choose to be; it is an open source application, which we insisted on when we deployed it.

I don't think you could implement that solution in a matter of hours, but if you would like a contract to do so, I would be happy to pay 5 times your usual hourly rate for a fixed-price delivery of such a system.  You know how to contact me!

Comment 17

[Dave Miller [:justdave]]

(In reply to comment #16)
> We are not dependent on the vendor unless we choose to be; it is an open source
> application, which we insisted on when we deployed it.

This is not true.  It's closed source, we have no access to the source code, it's a huge black box (which is why IT can't fix it ourselves).  It's all precompiled C# code running in Mono.

Comment 18

[user01]

What happens if two Deki templates reference each other? Can this eat up CPU time?

Comment 19

[Mike Shaver (:shaver emeritus)]

(In reply to comment #17)
> This is not true.  It's closed source, we have no access to the source code,
> it's a huge black box (which is why IT can't fix it ourselves).

The Mindtouch Core is GPL-v2, and the Mindtouch Pro is Shared Source.  Both of them have source available.  I will ensure that we get our hands on the source that matches what we have deployed, so that IT can fix it ourselves!  I look forward to it!

Comment 20

[Dave Miller [:justdave]]

Probably doesn't help that we don't have anyone on staff that knows C#

Comment 21

[Mike Shaver (:shaver emeritus)]

There's a pretty big difference between "it is a black box that we can't change, or we could fix it ourselves" and "the source doesn't help us, because we don't know how to use it".  Which is it?  I just mailed the developers demanding the source code...

Comment 22

[Dave Miller [:justdave]]

Yeah, big difference indeed.  We can always learn. ;)  The fact that the source was available was news to me.  Quite welcome news, mind you. :)  From what I understand blizzard and morgamic are already working on poking at it since we made that discovery.

Comment 23

[Ben Bucksch (:BenB)]

http://mdn.beonex.com/
Short-term workaround, in case you need to read the docs, but can't reach MDC. Limitations:
- DNS should get active within 24h, but will randomly fail ATM
- "dir.1" URLs
- Only English
- No edit, history etc.

Comment 24

[Ben Bucksch (:BenB)]

Response time is currently about 3s for me, which is workable. But of course still not as fast as it could be. Please do implement bug 559337.
Closing this bug, will reopen in case it gets bad again.