"Insufficient virtual memory" error message in Linux

[Lorents]

I am running a fortran program (it's a quantum chemistry program, a
modification of "Columbus") on a machine with a Pentium D (that is, a dual
core pentium 4), 4GB of ram and linux redhat. The program was compiled with
Intel Fortran compiler 9.1

My problem is that if I tell the program to use more than 2 GB of ram the
program eventually crashes with a message "insufficient virtual memory". I
am still trying to understand if this issue arises because of an intrinsic
limit of the CPU, of linux or of the compiler (or combinations of all
three).
I'm not an expert and I've heard a lot of contrasting opinions about this.
This is what I've gathered so far, and I'd very much appreciate if anybody
more skilled than me could tell me

1) CPU limitations
Intel's IA32 architecture traditionally uses (from the 386DX on) 32-bits to
address the memory, hence it can handle at most 4GB of ram
(physical+virtual).

1a) Intel introduced from the Pentium Pro onwards a wider 36-bit address bus
("Physical Address Extension" , PAE ), thus raising the amount of
addressable memory to a good 64GB.
Anyway, a single application still uses 32 bit addresses and so can see at
most 4GB. The advantage of PAE is that, provided the OS supports it, several
applications can be run simultaneously, each one taking at most 4GB.
However, there seems to be some "tricks" that allow a single application to
access more than 4GB. In windows, this should be called "Address Windowing
Extensions" (AWE). I don't know if something of this kind exists for linux.

1b) From the latest versions of the Pentium 4 (from the second half of 2004)
Intel introduced its "Extended Memory 64-bit Technology" EM64T, an
implementation of AMD's AMD64 extention to the IA32 architecture. This
theorically makes available 2^64 bytes of memory, even if present
implementations support "just" 2^40 bytes = 1TB.
To take advantage of this both the OS and the application at hand should be
compiled so as to take advantage of the 64 bit extention.

So, from all this, I understand that **potentially** my CPU is capable to
handle more than 4GB of ram, either via EM64T or perhaps via PAE.

2) Linux limitations
Here the situations get a bit more fuzzy (to me).

2a) The "standard" IA32-compiled version of linux's kernel supports only up
to 4GB or memory (physical+virtual). Moreover, the actual amount of memory
available for a single application is always less than that; from what I
understood, in the usual configuration this amount is of 2GB (and this would
explain why my application crashes at this point) but it can be increased to
3GB. The remaining 1GB is reserved to the kernel (that much?!)
Can anybody confirm this important point? If this is the case it'd be good
news for me -- at least I could give my application one extra GB of memory.

2b) It is possible to compile linux with the PAE, but amont of memory
available to a single applications should still top 3GB: see
http://www.spack.org/wiki/LinuxRamLimits


2c) It should be possible with my CPU to install a 64 bit version of linux,
which should remove all these memory limitations.
Would the application require recompiling as well? After all I just want to
make available to it 4GB, no more than that.

Any comment or suggestion would be very welcome!

Lorenzo

[Tim Prince]

Post by Lorents
I am running a fortran program (it's a quantum chemistry program, a
modification of "Columbus") on a machine with a Pentium D (that is, a dual
core pentium 4), 4GB of ram and linux redhat. The program was compiled with
Intel Fortran compiler 9.1
My problem is that if I tell the program to use more than 2 GB of ram the
program eventually crashes with a message "insufficient virtual memory". I
am still trying to understand if this issue arises because of an intrinsic
limit of the CPU, of linux or of the compiler (or combinations of all
three).
I'm not an expert and I've heard a lot of contrasting opinions about this.
This is what I've gathered so far, and I'd very much appreciate if anybody
more skilled than me could tell me
1) CPU limitations
Intel's IA32 architecture traditionally uses (from the 386DX on) 32-bits to
address the memory, hence it can handle at most 4GB of ram
(physical+virtual).
1a) Intel introduced from the Pentium Pro onwards a wider 36-bit address bus
("Physical Address Extension" , PAE ), thus raising the amount of
addressable memory to a good 64GB.
Anyway, a single application still uses 32 bit addresses and so can see at
most 4GB. The advantage of PAE is that, provided the OS supports it, several
applications can be run simultaneously, each one taking at most 4GB.
However, there seems to be some "tricks" that allow a single application to
access more than 4GB. In windows, this should be called "Address Windowing
Extensions" (AWE). I don't know if something of this kind exists for linux.
1b) From the latest versions of the Pentium 4 (from the second half of 2004)
Intel introduced its "Extended Memory 64-bit Technology" EM64T, an
implementation of AMD's AMD64 extention to the IA32 architecture. This
theorically makes available 2^64 bytes of memory, even if present
implementations support "just" 2^40 bytes = 1TB.
To take advantage of this both the OS and the application at hand should be
compiled so as to take advantage of the 64 bit extention.
So, from all this, I understand that **potentially** my CPU is capable to
handle more than 4GB of ram, either via EM64T or perhaps via PAE.
2) Linux limitations
Here the situations get a bit more fuzzy (to me).
2a) The "standard" IA32-compiled version of linux's kernel supports only up
to 4GB or memory (physical+virtual). Moreover, the actual amount of memory
available for a single application is always less than that; from what I
understood, in the usual configuration this amount is of 2GB (and this would
explain why my application crashes at this point) but it can be increased to
3GB. The remaining 1GB is reserved to the kernel (that much?!)
Can anybody confirm this important point? If this is the case it'd be good
news for me -- at least I could give my application one extra GB of memory.
2b) It is possible to compile linux with the PAE, but amont of memory
available to a single applications should still top 3GB: see
http://www.spack.org/wiki/LinuxRamLimits
2c) It should be possible with my CPU to install a 64 bit version of linux,
which should remove all these memory limitations.
Would the application require recompiling as well? After all I just want to
make available to it 4GB, no more than that.

I don't know of any adequate support for PAE in Fortran compilers.
32-bit linux is quite limited in usable address space available to a
Fortran program. Normally, there is little point in installing more than
2GB RAM for 32-bit operation, unless MPI or OpenMP is in use.
Running a 32-bit application under the 64-bit linux should give some
expansion in address space available to a program, possibly making use
of up to 4GB RAM. Re-compiling with a 64-bit compiler will make
available 2GB space for each static data object (array or common block),
and relatively unlimited space for allocatable data.

[glen herrmannsfeldt]

Post by Tim Prince

Post by Lorents
I am running a fortran program (it's a quantum chemistry program, a
modification of "Columbus") on a machine with a Pentium D (that is, a
dual core pentium 4), 4GB of ram and linux redhat. The program was
compiled with Intel Fortran compiler 9.1

(snip)

Post by Tim Prince

Post by Lorents
1) CPU limitations
Intel's IA32 architecture traditionally uses (from the 386DX on)
32-bits to address the memory, hence it can handle at most 4GB of ram
(physical+virtual).

Well, theoretically large model is still available, so virtual
addresses can be 45 bits. (Three bits for ring and local/global.)

(Last I knew the Watcom compiler would generate large model 32 bit
code. It might be that OS/2 knows what to do with it, but other
OS don't, as far as I know.)

Independent of virtual and physical address, the MMU is only
32 bits wide. This makes it hard for the OS to allow more than 4GB.

(snip)

Post by Tim Prince

Post by Lorents
1b) From the latest versions of the Pentium 4 (from the second half of
2004) Intel introduced its "Extended Memory 64-bit Technology" EM64T,
an implementation of AMD's AMD64 extention to the IA32 architecture.
This theorically makes available 2^64 bytes of memory, even if present
implementations support "just" 2^40 bytes = 1TB.
To take advantage of this both the OS and the application at hand
should be compiled so as to take advantage of the 64 bit extention.
So, from all this, I understand that **potentially** my CPU is capable
to handle more than 4GB of ram, either via EM64T or perhaps via PAE.

As I understand it, PAE is a lot of work for the user program.

Post by Tim Prince

Post by Lorents
2) Linux limitations
Here the situations get a bit more fuzzy (to me).
2a) The "standard" IA32-compiled version of linux's kernel supports
only up to 4GB or memory (physical+virtual). Moreover, the actual
amount of memory available for a single application is always less
than that; from what I understood, in the usual configuration this
amount is of 2GB (and this would explain why my application crashes at
this point) but it can be increased to 3GB. The remaining 1GB is
reserved to the kernel (that much?!)
Can anybody confirm this important point? If this is the case it'd be
good news for me -- at least I could give my application one extra GB
of memory.

I know that is true for windows. Normal W2K allows 2G, but some
versions have a boot time option to allow 3G. The OS needs some OS
owned memory to be available to user programs. It is easiest (for the
OS writer) to divide memory at 2GB. Especially it means that programs
(including the OS) that use 32 bit signed integers get the right result.

(snip)

Post by Tim Prince

Post by Lorents
2c) It should be possible with my CPU to install a 64 bit version of
linux, which should remove all these memory limitations.
Would the application require recompiling as well? After all I just
want to make available to it 4GB, no more than that.

This might be so, but I still wouldn't expect more than 2GB for a single
memory object (array).

Post by Tim Prince
I don't know of any adequate support for PAE in Fortran compilers.

I believe PAE takes explicit support from user code. Compilers aren't
likely to do it for you.

Post by Tim Prince
32-bit linux is quite limited in usable address space available to a
Fortran program. Normally, there is little point in installing more than
2GB RAM for 32-bit operation, unless MPI or OpenMP is in use.
Running a 32-bit application under the 64-bit linux should give some
expansion in address space available to a program, possibly making use
of up to 4GB RAM. Re-compiling with a 64-bit compiler will make
available 2GB space for each static data object (array or common block),
and relatively unlimited space for allocatable data.

The 2GB limit may still exist for a single dynamically allocated
object, even on a 64 bit system.

-- glen

[Jim]

Post by glen herrmannsfeldt

Post by Tim Prince

Post by Lorents
I am running a fortran program (it's a quantum chemistry program, a
modification of "Columbus") on a machine with a Pentium D (that is, a
dual core pentium 4), 4GB of ram and linux redhat. The program was
compiled with Intel Fortran compiler 9.1

(snip)

Post by Tim Prince

Post by Lorents
1) CPU limitations
Intel's IA32 architecture traditionally uses (from the 386DX on) 32-bits
to address the memory, hence it can handle at most 4GB of ram
(physical+virtual).

Well, theoretically large model is still available, so virtual
addresses can be 45 bits. (Three bits for ring and local/global.)
(Last I knew the Watcom compiler would generate large model 32 bit
code. It might be that OS/2 knows what to do with it, but other
OS don't, as far as I know.)
Independent of virtual and physical address, the MMU is only
32 bits wide. This makes it hard for the OS to allow more than 4GB.
(snip)

Post by Tim Prince

Post by Lorents
1b) From the latest versions of the Pentium 4 (from the second half of
2004) Intel introduced its "Extended Memory 64-bit Technology" EM64T, an
implementation of AMD's AMD64 extention to the IA32 architecture. This
theorically makes available 2^64 bytes of memory, even if present
implementations support "just" 2^40 bytes = 1TB.
To take advantage of this both the OS and the application at hand should
be compiled so as to take advantage of the 64 bit extention.
So, from all this, I understand that **potentially** my CPU is capable
to handle more than 4GB of ram, either via EM64T or perhaps via PAE.

As I understand it, PAE is a lot of work for the user program.

Post by Tim Prince

Post by Lorents
2) Linux limitations
Here the situations get a bit more fuzzy (to me).
2a) The "standard" IA32-compiled version of linux's kernel supports only
up to 4GB or memory (physical+virtual). Moreover, the actual amount of
memory available for a single application is always less than that; from
what I understood, in the usual configuration this amount is of 2GB (and
this would explain why my application crashes at this point) but it can
be increased to 3GB. The remaining 1GB is reserved to the kernel (that
much?!)
Can anybody confirm this important point? If this is the case it'd be
good news for me -- at least I could give my application one extra GB of
memory.

I know that is true for windows. Normal W2K allows 2G, but some versions
have a boot time option to allow 3G. The OS needs some OS owned memory to
be available to user programs. It is easiest (for the OS writer) to
divide memory at 2GB. Especially it means that programs (including the
OS) that use 32 bit signed integers get the right result.
(snip)

Post by Tim Prince

Post by Lorents
2c) It should be possible with my CPU to install a 64 bit version of
linux, which should remove all these memory limitations.
Would the application require recompiling as well? After all I just want
to make available to it 4GB, no more than that.

This might be so, but I still wouldn't expect more than 2GB for a single
memory object (array).

Post by Tim Prince
I don't know of any adequate support for PAE in Fortran compilers.

I believe PAE takes explicit support from user code. Compilers aren't
likely to do it for you.

Post by Tim Prince
32-bit linux is quite limited in usable address space available to a
Fortran program. Normally, there is little point in installing more than
2GB RAM for 32-bit operation, unless MPI or OpenMP is in use.
Running a 32-bit application under the 64-bit linux should give some
expansion in address space available to a program, possibly making use of
up to 4GB RAM. Re-compiling with a 64-bit compiler will make available
2GB space for each static data object (array or common block), and
relatively unlimited space for allocatable data.

The 2GB limit may still exist for a single dynamically allocated
object, even on a 64 bit system.
-- glen

In addition, most virtual memory systems divide the 4 GB virtual address
space into a 2 GB region for the operating system and a 2 GB region for user
programs. This allows the operating system to be mapped such that a user
program can access whatever services it needs.

Hence, there is a maximum amount of virtual address space available to any
user program which depends on the design of the operating system.

Jim

[Lorents]

Post by Jim
In addition, most virtual memory systems divide the 4 GB virtual address
space into a 2 GB region for the operating system and a 2 GB region for user
programs. This allows the operating system to be mapped such that a user
program can access whatever services it needs.
Hence, there is a maximum amount of virtual address space available to any
user program which depends on the design of the operating system.

Thanks everybody. It seems that things are not straighforward unless one
recompiles the code to 64 bits. I guess I'll have to live with the
2GB limitation for the time being :-/

Lorenzo