220373 - web-sniffer won't compile on RedHat Linux

Status: RESOLVED FIXED

(Webtools Graveyard :: Web Sniffer, defect)

Description

[Dave Miller [:justdave]]

It either won't compile on Linux or just won't compile...  It's been there
forever on the old Solaris box, so I assume it compiled at some point, at least
on Solaris. :)

In file included from hash.c:28:
mutex.h:25:20: thread.h: No such file or directory
mutex.h:26:19: synch.h: No such file or directory
In file included from mutex.h:29,
                 from hash.c:28:
main.h:28:19: synch.h: No such file or directory
make: *** [hash.o] Error 1

Comment 1

[Dave Miller [:justdave]]

It's quite possible we're missing a needed -devel package, anyone know which one?

Comment 2

[Frank Wein [:mcsmurf]]

From debian-Package search i got this (ok this is not redhat, but it give some
hints, which might be useful for this; because Debian has packages for nearly
everything). For synch.h it means package libace5.2-dev. This package also
includes the thread.h, so this seems to be the right one. Go look for libace-dev
for Redhat :)!

Comment 3

[Frank Wein [:mcsmurf]]

Ok under http://deuce.doc.wustl.edu/Download.html you can download a source
version of ACE. http://cvs.doc.wustl.edu/ace-cvs.cgi/ACE_wrappers/ace/ there
seem to be the two missing files. So ACE+TAO should be the right file to
download (probably there isn't a red hat rpm, is it?!)

Comment 4

[Frank Wein [:mcsmurf]]

Ok, i think this is wrong. See up/lowercase. You might try googleing for these
files?!

Comment 5

[Martin Sch]

I just looked into the code and found no real reason why this mutex stuff should
be necessary. It is used in net.c and robot.c. In net.c it is necessary because
the used variables are static. But why are they static?

Comment 6

[Dave Miller [:justdave]]

Seeing as how UNIX != Linux (despite what SCO wants to think ;) it's also quite
possible that some of the APIs being used are defined in different header files
on Linux than they are on UNIX/Solaris.

Comment 7

[tenthumbs]

The Solaris and Linux thread api's have different names as well and may
operate differently. Lots of work to fix it.

Is there a good reason why this thing is threaded? It's purpose is 
essentially serialized anyway.

Comment 8

[tenthumbs]

OK, I figured I'd waste a few hours. I chopped out anything that I didn't like
and I'v made it compile and work. It's a hack, an atrocity, an abomination but
it works for me. It is highly likely that I've broken other things, or reduced
performance so it will hammer the server, or introduced some  awful security
hole.

You probably don't want to run this anyway. It doesn't use CSS so it produces
ugly output.

Comment 9

[tenthumbs]

Attachment: patch

Wouldn't know a thred if one bit it. Won't work on Solaris. Won't work on
Windows. Probably won't work anywhere.

Comment 10

[Frank Wein [:mcsmurf]]

btw: if someone needs a quite good replacement, try http://web-sniffer.net/

Comment 11

[tenthumbs]

Another possibility would be to replace this with a script and use
source-highlight <http://www.gnu.org/software/src-highlite/> as a back 
end.

Comment 12

[Gervase Markham [:gerv]]

Easiest to redirect to web-sniffer.net IMO. It works well. justdave: can you
arrange that?

Gerv

Comment 13

[Dave Miller [:justdave]]

Done.

Comment 14

[Erik van der Poel]

I realize that mozilla.org's web sniffer location now redirects to another HTTP
viewer, and that's fine, but I have ported Web Sniffer to Linux, and I would like
to check in the changes. The "diff -u" output is about 2500 lines and the new
files are about 750 lines (excluding autoconf output). Should I attach the diffs
and new files to this bug for review, or should I just check it all in, given
that I am the original author of Web Sniffer?

http://webtools.mozilla.org/web-sniffer/

Comment 15

[Erik van der Poel]

By the way, I have registered a new domain to host my own Web Sniffer:

http://websniffer.org/

Comment 16

[Erik van der Poel]

(In reply to comment #5)
> I just looked into the code and found no real reason why this mutex stuff should
> be necessary. It is used in net.c and robot.c. In net.c it is necessary because
> the used variables are static. But why are they static?

Well, most of the tools in the Web Sniffer source directory are single-threaded,
but the "robot" tool (aka web crawler, or spider) gets much better performance
when it is threaded. All the tools happen to use the same routines, and the
robot collects some statistics and measures the time it takes for a DNS lookup,
etc. In order to count various things, the multi-threaded robot must use static
variables. (Per-thread storage is another possibility, but why bother?)

I suppose I could place various hooks in the right places to call back to the
app, which can then synchronize threads if any. I'll think about it.

Comment 17

[Simon Montagu :smontagu]

(In reply to comment #14)
> I realize that mozilla.org's web sniffer location now redirects to another
> HTTP viewer, and that's fine

(In reply to comment #15)
> By the way, I have registered a new domain to host my own Web Sniffer:
> 
> http://websniffer.org/

Can the redirect be changed to point to http://websniffer.org/ pending
resolution of this bug? The original mozilla web sniffer is much more useful
than the current redirectee (at least for anything that I ever use it for ;-) )

Comment 18

[Erik van der Poel]

After this bug, I still have to fix bug 57556 before it can be deployed
anywhere.

Comment 19

[Erik van der Poel]

Changed name from Web Sniffer to SniffURI. See sniffuri.org.

Ported to Linux. See Bonsai for file version details and log message.

Comment 20

[Katsuhiko Momoi]

I agree with Dave. I missed this tool -- we should get this deployed again at
mozilla.org. Thanks, Erik.

Comment 21

[Erik van der Poel]

(In reply to comment #8)
> You probably don't want to run this anyway. It doesn't use CSS so it produces
> ugly output.

I've checked in changes to make it use CSS. See sniffuri.org.