Machines with large page sizes should be in our testing configuration

VERIFIED FIXED in flash10.1

Status

P3
normal
VERIFIED FIXED
9 years ago
8 years ago

People

(Reporter: dansmith, Assigned: trbaker)

Tracking

unspecified
flash10.1
x86
All
Dependency tree / graph
Bug Flags:
in-testsuite -
flashplayer-qrb +

Details

(Reporter)

Description

9 years ago
Windows and Linux have supported machines with large page sizes (> 4K) for several years.  We should ensure that Tamarin can execute properly on these systems.

Information on operating systems and hardware that support large page sizes can be found at  http://en.wikipedia.org/wiki/Page_(computer_memory)
(Assignee)

Comment 1

9 years ago
AFAICT, Windows versions since Vista and Server 2003 support "large pages" out-out-the-box for user applications.  

http://www.microsoft.com/whdc/system/sysinternals/memmgt.mspx
A user-mode application can allocate pages as large as 4 MB on x86-based systems by using the VirtualAlloc function with the MEM_LARGE_PAGES flag.

Is this a request for shell builds with different VMPI_allocateCodeMemory values?

Comment 2

9 years ago
The idea here is to test a configuration where the VM page size as reported by sysconf() or similar is not 4KB, ie, the page size has been pinned to something other than 4KB /outside/ the application.

We could probably simulate that in part by adding a command line switch -Dpagesize=n and then overriding page size computation internally, but I don't think it would be a fully satisfactory test, the problems show up when the protection granule is not equal to the MMgc block size, and I worry that simulating the page size may not in fact change the protection granule from 4KB to the simulated page size.

We know for a fact that MIPS uses 8KB pages in some configurations and Chris Dearman has filed patches to work around bugs; perhaps he can suggest a suitable test platform.
(Assignee)

Comment 3

9 years ago
coincidentally, we just got a working mips avmshell today!  :-)
(Reporter)

Comment 4

9 years ago
QE: I'd like this tested before shipping 10.1 if possible.
Assignee: nobody → trbaker
Flags: flashplayer-qrb+
Priority: -- → P3
Target Milestone: --- → flash10.1
(Assignee)

Comment 5

9 years ago
Chris, how can I determine the page size on mips (broadcom 97405)?  thanks.

Comment 6

9 years ago
Looks like the Broadcom boards we are using are 4KB pages:

Running the following code on the Broadcom 97405 boxes:
http://en.wikipedia.org/wiki/Page_%28computer_memory%29

#include <stdio.h>
#include <unistd.h> // sysconf(3)
 
int main(void) {
	printf("The page size for this system is %ld bytes.\n",
	       sysconf(_SC_PAGESIZE)); // _SC_PAGE_SIZE is OK too.
 
	return 0;
}


# ./pagesize_mips
The page size for this system is 4096 bytes.
(Assignee)

Comment 7

9 years ago
And for linux, again, your application must be configured to use large pages.  the primary page size is still 4k, but you can pre-allocate a number of pages of a desired size > 4k, mount that as a hugetlbfs filesystem, and then tell your app to use that (and you app must know how to do so).

See the mysql doc here:
http://dev.mysql.com/doc/refman/5.1-maria/en/large-page-support.html

Samilar info can be found for Oracle.

Here is more info on the kernel support for large pages:
http://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt

It appears to me that on windows and linux avmshell would need code to use large pages.  The default page size on these eprating systmes is still 4kb and the app needs to know how to use large pages when they're configued at runtime in said operating system.

As Brent said, the Broadcom 97405 images we have also have 4kb page size.  I don't know about the Sigma ref software package.

I need to investigate rebuilding either the Broadcom or Sigma image with > 4kb pages as default.  Open to other suggestions.
(Assignee)

Comment 8

9 years ago
I could create large pages in vmware Ubuntu 9.10, appears kernel was built with  support for CONFIG_HUGETLBFS and CONFIG_HUGETLB_PAGE.

$ grep -i huge /proc/meminfo
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       4096 kB


$ sudo sysctl -w vm.nr_hugepages=10
vm.nr_hugepages = 10

$ grep -i huge /proc/meminfo
HugePages_Total:      10
HugePages_Free:       10
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       4096 kB


But, now what?  AFAICT, vm won't use those without configuration/code in vm.

Comment 9

9 years ago
Running /usr/bin/pagesize on our solaris machine says that it is 8192 pagesize:

-bash-3.00$ /usr/bin/pagesize
8192
(Assignee)

Updated

9 years ago
Depends on: 540929
(Assignee)

Comment 10

9 years ago
540929 should be reverted?

Comment 11

9 years ago
Digging around I found this bug 540929, which disables page protection on solaris because it uses 8k pages. If I revert the change from that bug to enable page protection on solaris the shell runs the acceptance suite correctly in release|debug hybrid|JIT. I am going to reopen bug 540929 to have the change reverted back out.
(Assignee)

Updated

9 years ago
Depends on: 551173

Comment 12

9 years ago
* Solaris builds have been re-enabled in the build system and are passing all acceptance runs
* Solaris machine reports a pagesize of 8192k
* page protection is enabled and verified via avmshell -Dversion which does report that the /AVMFEATURE_PROTECT_JITMEM/ feature is enabled

Comment 13

9 years ago
I seem to remember that AVMFEATURE_PROTECT_JITMEM was enabled on Sparc in the past as well, but that there were hacks to work around problem.  (Imperfect memory on that.)  I removed at least one such hack with the new VMPI layer for code memory; that hack pinned the VM page size to 4KB IIRC.
(Assignee)

Updated

8 years ago
Depends on: 551051
(Assignee)

Comment 14

8 years ago
How to use large pages from java on windows.
http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp?topic=/com.ibm.java.doc.user.win32.60/user/alloc_large_page.html

Only noting as further evidence that your application must know how to use pre-reserved large pages and that it is not an operating system wide setting on Windows.  Same conclusions drawn from MySQL, Oracle and kernel documentation for Linux.

Comment 15

8 years ago
http://msdn.microsoft.com/en-us/library/aa366720(VS.85).aspx
http://support.microsoft.com/kb/975992#more

On windows large page support requires application changes just like linux.    So our options for testing beyond solaris 8kb pages are to port to freebsd or build in specialized linux/windows code for using large pages.
(Assignee)

Comment 16

8 years ago
Right, and from the wikipedia entry:

"On FreeBSD and Solaris, applications take advantage of huge pages automatically, without the need for modification."
(Reporter)

Comment 17

8 years ago
Ed, Lars, referring to comment #13, are there any remaining hacks that prevent Sparc from using an 8KB pagesize?

Comment 18

8 years ago
(In reply to comment #17)
> Ed, Lars, referring to comment #13, are there any remaining hacks that prevent
> Sparc from using an 8KB pagesize?

I just grepped the code for 'sparc' and 'solaris' (case-insensitively) and I found nothing to suggest that such hacks still exist under any likely ifdefs for the platform.

Comment 19

8 years ago
I can't think of anything.  CodeAlloc and VMPI are up to date in argo.
(Assignee)

Comment 20

8 years ago
marking fixed.  Solaris is back online (8kb page size) with AVMFEATURE_PROTECT_JITMEM enabled.
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
(Assignee)

Updated

8 years ago
Status: RESOLVED → VERIFIED
Flags: in-testsuite-
You need to log in before you can comment on or make changes to this bug.