This is somehow similar to the one above, except that this time an attacker can exploit the memory quota. This quota defines how much memory a user process can allocate at a time. If there is no capping on this value, a user process can try to allocate all the memory available in the system (including the virtual memory), thus preventing any other process that needs to allocate memory from doing so — leading to either crashes in the system processes or preventing these from working.

The code to exploit this is quite simple, as you might have guessed it: an infinite loop that simply calls malloc() this time– so the memory allocated at each iteration will grow by a certain amount and pretty pretty shortly it will cause the system processes to crash or at least prevent them from working properly — such that the only thing you can really do is a hard reset. (Again, you won’t be able to log in either via the local console or remotely — or even kick off a reboot process — as the system will no longer have memory to allocate to new processes; or even worse, some of the daemon processes — e.g. ssh — will die because of lack of memory.)

Below is the code:

#include <stdlib.h>
 
/* We are going to allocate 1Meg at a time */
#define ALLOCATION_UNIT (1024 * 1024)
 
int main( int argc, char ** argv )
{
   while( 1 )
      malloc( ALLOCATION_UNIT );
}

Remember that in order to prevent such an exploit on your machine you should set a memory quota for your users. Consult your man pages on how to do so.

Download the above source code here.

NOTE: This is an article from my old website, way before I restructured it. Since I see visitors going through search engines and requesting the old page, I decided to resurrect this, as it obviously presents a certain level of interest to users out there, so at least future searches will not get a simple 404 but get instead this page.