Causing malloc() to return NULL on CentOS

Question

I'm about to teach an introductory computer science course in C and I'd like to demonstrate to students why they should check whether malloc() returned a NULL. My plan was to use ulimit to restrict the amount of available memory such that I could exercise different code paths with different limits. Our prescribed environment is CentOS 6.5.

My first attempts to make this happened failed and the shell showed "Killed". This led to me discovering the Linux OOM killer. I have since tried to figure out the magic set of incantations that will cause the results I'm looking for. Apparently I need to mess with:

• /etc/sysctl.conf
• ulimit -m
• ulimit -v
• vm.overcommit_memory (which apparently should be set to 2, according to an Oracle article)

This far either I get "Killed" or a segmentation fault, neither of which is the expected outcome. The fact that I'm getting "Killed" with vm_overcommit_memory=2 means that I definitely don't understand what's going on.

If anyone can find a way to artificially and reliably create a constrained execution environment on CentOS so that students learn how to handle OOM (and other?) kinds of errors, many course instructors will thank you.

Answer 1

It is possible to [effectively] turn off overcommitting from kernel >= 2.5.30.

Following Linux Kernel Memory :

// save your work here and note your current overcommit_ratio value

# echo 2 > overcommit_memory
# echo 1 > overcommit_ratio

this sets the VM_OVERCOMMIT_MEMORY to 2 indicating not to overcommit past the overcommit_ratio, which is set to 1 (ie no overcommitting)

Null malloc demo

#include <stdio.h>
#include <stdlib.h>

int main(int argc,char *argv[])
{
  void *page = 0; int index;
  void *pages[256];
  index = 0;
  while(1)
  {
    page = malloc(1073741824); //1GB
    if(!page)break;
    pages[index] = page;
    ++index;
    if(index >= 256)break;
  }
  if(index >= 256)
  {
    printf("allocated 256 pages\n");
  }
  else
  {
    printf("memory failed at %d\n",index);
  }
  while(index > 0)
  {
    --index;
    free(pages[index]);
  }
  return 0;
}

Output

$ cat /proc/sys/vm/overcommit_memory 
0
$ cat /proc/sys/vm/overcommit_ratio 
50
$ ./code/stackoverflow/test-memory 
allocated 256 pages
$ su
# echo 2 > /proc/sys/vm/overcommit_memory 
# echo 1 > /proc/sys/vm/overcommit_ratio 
# exit
exit
$ cat /proc/sys/vm/overcommit_memory 
2
$ cat /proc/sys/vm/overcommit_ratio 
1
$ ./code/stackoverflow/test-memory 
memory failed at 0

remember to restore your overcommit_memory to 0 and overcommit_ratio as noted