231904 - Evaluate use of Windows Services for UNIX (SFU) in place of Cygwin

Status: RESOLVED WONTFIX

(Firefox Build System :: General, enhancement)

Description

[Ali Ebrahim]

Microsoft has recently released Windows Services for UNIX 3.5 (SFU) for free. I
think it would be great if we could get Mozilla to compile using SFU with MSVC
or with the GCC 3.3 which is included in SFU. SFU is based on Interix.

Microsoft claims that SFU's performance is greatly increased, and they claim
that file I/O is now within about 10% of native I/O performance, which might be
a big performance win over Cygwin.

Microsoft Windows Services for UNIX 3.5 can be obtained from:
http://www.microsoft.com/windows/sfu/

Comment 1

[Benjamin Smedberg]

Just a note, we're going to support cygwin indefinitely, we may also decide to
support WSFU as an alternative. At 222MB download, it definitely isn't the
tinkerer's tool, but it might help tinderboxen and such to cycle a lot faster.

Comment 2

[Heikki Toivonen (remove -bugzilla when emailing directly)]

Does that package provide Windows tools on Unix, or Unix tools on Windows (or
both)? It is a little unclear to me...

Comment 3

[Ali Ebrahim]

Unix tools on Windows (basically its like Cygwin). Yes, the name is very unclear.

Comment 4

[[e-mail]]

Services for Unix appears to provide the following utilities:
[	addr	alias	ar	at	atq	atrm	awk	banner	basename	batch	bc	bg	bp	c89	cal
calendar	captoinfo	cat	cc	cd	chgpath	chgrp	chmod	chown	chsh	cksum	clear	cmp
col	column	comm	command	cp	cpio	crontab	csh	csplit	ctags	cut	date	dc	dd	df
diff	diff3	dig	dircmp	dirname	dnsquery	du	echo	ed	egrep	env	ex	expand
expr	FALSE	fc	fg	fgrep	file	fileinfo	find	finger	flip	fmt	fold	ftp	gencat
getconf	getopt	getopts	grep	head	hexdump	host	hostname	iconv	id	infocmp
infotocap	install	ipcrm	ipcs	ispell	jobs	join	kill	ksh	last	less	lessecho
lesskey	lex	line	ln	locale	localedef	logger	login	loginenv	logname	lp	ls	m4
mail	mailx	make	man	mkdir	mkfifo	more	mpack	mt	munpack	mv	mvwtmpx
newgrp	nice	nl	nm	nohup	nslookup	nsupdate	ntpath2posix	nvi	od	passwd	paste
patch	pathchk	pax	pdomain	pg	ping	posixpath2nt	pr	printf	ps	pstat	psxoffset
pwd	ranlib	rcp	rdist	rdistd	read	regpwd	renice	reset	rlogin	rm	rmdir	rpcgen
rpcinfo	rsh	rstartd	runwin32	script	sdiff	sed	service	sh	showmount	size
sleep	sort	spell	split	ssimda	strerror	strings	strip	strsignal	stty	su	sum
syslogd	tail	talk	tar	tcsh	tee	telnet	test	tftp	tic	time	tip	toe	touch
tput	tr	TRUE	truss	tset	tsort	tty	tzselect	umask	unalias	uname	unexpand
unifdef	uniq	unixpath2win	unvis	uudecode	uuencode	vi	view	vis	wait	wc	who
whoami	winpath2unix	wvisible	xargs	yacc	yearistype
Are there native versions of gawk and gmake we can use?

Comment 5

[Simon Paquet [:sipaq]]

(In reply to comment #4)
> Are there native versions of gawk and gmake we can use?

The SFU stuff from Microsoft/Interix is based on OpenBSD 2.9 IIRC. Which
features of gmake and gawk do we use, that the BSD-equivalent (SFU contains awk
and make) does not have?

Comment 6

[Benjamin Smedberg]

We require GNU make, other versions of make do not have the featureset we need.

Comment 7

[Kevin Brosnan]

GNU make for SFU is available from
http://www.interopsystems.com/tools/warehouse.htm

Comment 8

[[e-mail]]

Attachment: Get my tree building

Obviously this does everything wrong for various reasons. Basically the changes
fall into categories:
* Specifying the case and extension of executables; I suppose this could be
worked around using symbolic links. Note that the Interix bash shell's path
search will also search case insensitively for NT executables, but in all other
cases the case and extension must be specified exactly.
* Making Interix detect as a Windows OS.
* Using the special Interix-wrapper (as I only support MSVC builds).
* Applying the wrapper to LINK.EXE
* Fixing several of the unix utilities to be passed unix paths. Many MSYS,
mingw and cygwin utilities will happily accept a mixture of Windows and unix
paths.
* Interix does not accept cp -a. Maybe this should use nsinstall?

Comment 9

[[e-mail]]

Attachment: interix-wrapper c.f. cygwin-wrapper

Sigh, my build just failed because Interix zip doesn't support -X :-(

Comment 10

[[e-mail]]

Attachment: chrome packaging

Note that interix zip doesn't accept -X, I didn't try using GNU zip instead.

Comment 11

[[e-mail]]

I succesfully built without PSM. Would some build guy mind looking at my patches
to see if I should try to beat PSM into shape or cut my losses?

Comment 12

[[e-mail]]

autoconf-2.13-bin detects executables by running ./conftest however Interix
requires an exact filename match (Interix bash is only clever about this for the
path). Anyone wanting to build with SFU will need to install autoconf-2.13-bin
and modify acgeneral.m4 and configure client.mk to run autoconf locally.

Comment 13

[[e-mail]]

Attachment: Builds but needs autoconf changes

Comment 14

[[e-mail]]

Attachment: Changes to mozilla/security

OK, with these changes I can now enable crypto.

Comment 15

[[e-mail]]

Attachment: All changes to mozilla/security

Forgot to edit Makefile.in instead of Makefile :-[

Comment 16

[Alessandro Minelli]

I suggesto "wontfix": Microsoft has decided against enhancing its Services for
Unix (SFU) product and will not release any new versions of it going forward,
according to company officials.

http://www.eweek.com/article2/0,1895,1855274,00.asp

Comment 17

[Jonathan Stanley]

Was going through my votes list and kind of forgot this one... 

(In reply to comment #16)
> I suggesto "wontfix": Microsoft has decided against enhancing its Services for
> Unix (SFU) product and will not release any new versions of it going forward,
> according to company officials.
> 
> http://www.eweek.com/article2/0,1895,1855274,00.asp

Quoting out of context. ;) Said article states that SFU3.5 itself will be supported till 2014. Furthermore, SFU is effectively bundled into Windows, first starting with Windows Server 2003 R2, though it is called "UNIX Interoperability Components" (http://www.microsoft.com/windowsserver2003/r2/unixinterop/default.mspx)

The native UNIX-like subsystem will also make it into Windows Longhorn Server on the serverside, though more importantly (IMO), it'll also make it into Windows Vista as an optional component and will be called "Subsystem for UNIX-based Applications (SUA) (http://www.microsoft.com/windowsvista/getready/editions/enterprise.mspx). SUA will be available in Vista Enterprise and Vista Ultimate, where the latter will be the edition of choice by pretty much every Windows "power user".

Being able to compile under windows natively would be "A Good Thing". :)

Comment 18

[Jonathan Stanley]

An attempt to keep this on the radar for Mozilla to support building binaries in an Interix environment on either Windows Server 2003 with SUA 5.2 or Windows Vista/Windows Longhorn Server with SUA 6.0.

I've posted a thread over at Interix's forum, linkage: http://www.interopsystems.com/tools/tm.aspx?m=10011

It's best to consider the environment as being *BSD like and "out of the box" (eg: paths are case-sensitive, using install and not cp, etc) so it should be possible to build Mozilla source as a POSIX application, though will require X-Server for display.

For the compilation of Win32 (and potentially Win64) binaries, there is a special script called wcc which then allows the use of UNIX make, gmake, jam or any other tool (eg: MSVC) for the building/installing of Win32 binaries.

Comment 19

[Jonathan Stanley]

And hot on the heels of wcc is wgcc: http://sourceforge.net/projects/interix-wgcc

"wgcc is a cross-compiler tool primarily written for Microsoft's Interix. Its primary purpose is to produce native Windows binaries (internally using the Microsoft Tool chain), and to mimic the behaviour of the GNU compiler collection. This means that wgcc understands many of GCC's command line arguments, and in most cases delivers the same results as expected, sometimes manipulating the underlying tool's input and output."

... and the rest: http://sourceforge.net/forum/forum.php?forum_id=613346

Sounds like this could be supported as a build path without too much work?

Comment 20

[Benjamin Smedberg]

Since our build system supports building against MSVC directly in the unix build environment, there is no need for interix/wgcc. All we need is a shell-scripting/gmake support with the standard linux tools (find/grep/sed/iconv).

This is a non-priority from my POV; MSYS is a stable system for this work, as is cygwin for the most part. I'll accept patches if they're not too intrusive (and they shouldn't need to be).

Comment 21

[[e-mail]]

I tried to write up the process here:
http://developer.mozilla.org/en/docs/Building_on_Windows_with_SFU_%28Interix%29

Comment 22

[(not currently active) Ted Mielczarek]

MozillaBuild is the only supported build environment.