Before Warp Server for e-business, OS/2 had a process address space limit of 512 MB. The reason for this was so that any block of memory could be shared between 16-bit and 32-bit OS/2 applications. However, IBM lifted this limit in the Warp Server for e-business kernels. It is possible with these kernels to tell OS/2 when allocating memory that it can be above the 512 MB limit. I suggest that we do this by default in NSPR so that an NSPR application is not subject to the limit. This is more than a theoritical limit, I actually ran into it.
Did it go in a Merlin fixpak? If not, we can't do it. Most OS/2 users are running merlin with fixpak 14.
Mike, I'm not sure if it did - my guess is no. However, this may be a change that is still compatible with older versions. It can easily be tested if you have access to a machine running Warp 4.0. Both of my home PCs are running Warp server for e-business, as well as the work system. Unfortunately, I couldn't find the documentation on the >512 MB support in the Warp Server for e-business documentation. There are only two documents that are relevant - the base "Control Program Programming Guide and Reference" and the "OS/2 Programming Guide and Reference Addendum". It's not listed in either place. I know this is something that got added in Warp Server 3 Advanced SMP (ie. before E-business and before Warp 4.0), and it was documented in the addendum for its Toolkit. I no longer have it installed. I would have expected it to be included in the WSEB doc, but it isn't. Regardless, I found it in the headers, and the flag for DosAllocMem to allocate memory above 512 MB is OBJ_ANY, which is defined as 0x00000400U. This tells OS/2 that it can allocate memory "anywhere" according to the comment in bsememf.h . The meaning is "anywhere including above the 512 MB limit". My guess (and hope) is that on a Warp 4 system, this flag will be ignored, and DosAllocMem will simply return memory below the 512 MB limit. But I'm not sure of that, I haven't used Warp 4.0 in years. It needs to be tested. If the flag causes an error, a specific error will probably be returned, and in the worst case we could try again without OBJ_ANY.
There is another problem in the way of implementing this, which is potentially a lot more serious. It is related to the browser's very relaxed memory management code. A lot of the code in the browser currently mixes PR_Malloc() with a C-library free() call, or malloc() and PR_Free(). This is alright today because PR_Malloc currently maps to the C library malloc() call. If however we starting mapping it to DosAllocMem - with or without the OBJ_ANY flag - both of these cases will cause either a crash or a memory leak (depending how relaxed the free() function in the C runtime is). We know about the problem because we have another NSPR allocator called the zone allocator, which doesn't use the C malloc() function directly. This zone allocator is only implemented on NT and Unix currently. It is turned off by default and is enabled with an environment variable. We know that it is incompatible with several Netscape server applications that haven't bothered to straighten their allocation code, and the client suffers from this problem too. So if we go with this other allocator, all the code in the browser needs to be reviewed. Basically, there should never be any calls to malloc() or free(), only PR_Malloc() or PR_Free(). This is something that should be done regardless to clean the browser code, as it stands in the way of implementing a more efficient allocator on any platform. A possible short-term transparent fix which doesn't involve fixing the incorrect browser code would be to fix the C runtime library malloc() call to use OBJ_ANY . I don't know of any way to do this with the C runtime from either VACPP or EMX. Unlike on Unix, we can't provide new malloc() and free() functions and override the existing functions in the C runtime library DLL by virtue of DLL linking order in the executable. However, on OS/2, there is no single shared C runtime library. Every application ships with its own C runtime DLL - including Mozilla. So it should be possible to do this. But it would require rebuilding the C runtime DLL from source, with modified malloc and free calls. The C library source is not included with IBM's compiler, so I don't think that's an option either. Perhaps the best choice would be to couple this implementation of >512 MB memory allocation on OS/2 with the implementation of the zone allocator, since the later is already incompatible with the browser currently. Applications that need large amounts of memory are also likely to be using this zone allocator, so this may be a good compromise.
I wanted to give a quick shot at trying to run some NSS apps with a modified NSPR allocator that would allow allocating >512 MB memory (ie. calls DosAllocMem with OBJ_ANY). However, I ran into the problem that I need an implementation of realloc() ... Mike, is it at all possible to obtain the source code to the C runtime library ? Short of that, a binary patch of the DLL is probably the easiest fix. However, I'd much prefer the source because I'm investigating another problem that I suspect involves locking in the allocator of the C runtime library (164512).
I'll try to get the source, but there is no way I can support a Mozilla that is using a custom C runtime. This is simply going to be a limitation of Mozilla on Os/2. Note that this could be a nice opportunity to fix the issue that Mozilla has of using a zillion different calls throughout that go to malloc rather than PR_Malloc.
I recommend against this. Many OS/2 APIs will cause crashes, traps or weird behavior when they get high memory buffers. We have a lot of experience on this issue and we ended up with many workarounds in our applications where we required high memory.
This can't be fixed. Memory allocated above 512MB cannot be passed to all OS/2 APIs, so we cannot change the default Mozilla allocators to allocate memory in this area. The only way allocating above 512MB is useful is in cases where you have complete control of what is happening with that memory which is simply not the case in Mozilla. the 512MB limit is going to be a permanent restriction of Mozilla on OS/2. The GCC folks agree with me as well - GCC will not allocate memory above 512MB for this same reason.