Linux memory monitoring (allocations Vs usage)

How to use some of Linux’s standard tools and how different types of memory usage shows up.

Examples of using malloc and writing to memory with three use-cases for a simple process

No memory allocation at all no_malloc.c
A call to malloc() but memory is not written to malloc only.c
A call to malloc() and then memory is written to the allocated space malloc and write.c

In each case we run the example with a 64MB allocation so that we can see the usage from standard linux tools.

We do something like this

gary@linux:~/git/unixfun$ ./malloc_and_write 65536
Allocating 65536 KB
Allocating 67108864 bytes
The address of your memory is 0x7fa2829ff010
Hit <return> to exit

top
free
pmap
gdb

top

top malloc only

When we run the malloc process which only allocated memory, we see the VIRT value hit around 68MB (size of the process plus the 64MB that we asked it to allocate) – but the RES value stays small (956KB)

top malloc and usage

When we run the malloc_and_write which means we additionally write data (‘N’) to the memory which we allocated. With 64MB as the allocation we will see ~67-68MB for both VIRT and RES.

top (no memory allocation)

With a very simple process that does not allocation at all both VIRT and RES are tiny

free

Using the free tool we can see memory usage is only counted when the process actually writes to memory

start with initial state
Run process which only allocates memory (does not write any data).
Quit that program and check to see we are back to the initial state
Run process which makes an allocation of 64MB and then writes to it

Whne we allocate and use 64MB we see the free value decrement by ~70MB and the used value increases by ~70MB. When we just run the malloc() and no actual usage – the free/used change by about 2MB which is probably the size of the process and libraries.

pmap

With pmap -x we can look into the process and see the various memory regions that make up the address space.

pmap allocate and use

We see the Kbytes and RSS with calues around 64MB – as well as how much is dirty

pmap allocate only

If we only malloc – then we see the anon segment of ~64MB but no RSS and no Dirty

pmap no allocation

With no alloc we do not see the second anon segment.

gdb

Using the output of pmap we can use gdb to dump the memory area that has been allocated. We can also use the address returned by malloc if we know it.

gdb for memory allocated and written

Our example code writes the character "N" to every memory location that we write to. So we expect to see that in the process address space.

open gdb and attach to the process we want to inspect e.g. as root gdb --pid $(pidof malloc_and_write)
use the command dump memory <filename> <start-memory-address> <end-memory-address>
e.g. (gdb) dump memory /tmp/pmap2 0x7fa2829ff000 0x7fa2829ff0ff
quit gdb
usd od to print out the contents of the dumpfile e.g. od -x /tmp/gdbdump or od -c /tmp/gdbdmp

gdb for memory allocated only

If we do not write to memory – what is returned for a read to that area? We can use gdb to find out.

attach to process with gdb

Attach to the process and dump memory

dump output with od

The output of od shows us that the memory contains NULLs which makes sense

gdb for unallocated

If memory is unallocted then we need to guess a region. We get an error back from the mmu as expected since it’s not allocated – if we tried to read this memory in the process we would expect to get a segv or similar

Code

Example no malloc

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[]) {
    int bytes;
    int *ptr;
    getchar();
}

Example malloc only

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[]) {
    int bytes;
    int *ptr;
    if (argc == 2) {
        printf("Allocating %s KB\n", argv[1]);
        bytes=(atoi(argv[1])*1024);
        printf("Allocating %d bytes\n",bytes);
    } else {
        printf("Please supply KB to allocate\n");
        exit(1);
    }
    if (bytes>0) {
    ptr=malloc(bytes);
    printf("The address of your memory is %p\n",ptr);
    }else{
        printf("No memory allocated\n");
    }
    getchar();
}

Example malloc and write

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[]) {
    int bytes;
    char *ptr,*ref;
    if (argc == 2) {
        printf("Allocating %s KB\n", argv[1]);
        bytes=(atoi(argv[1])*1024);
        printf("Allocating %d bytes\n",bytes);
    } else {
        printf("Please supply KB to allocate\n");
        exit(1);
    }
    if (bytes>0) {
        ptr=malloc(bytes);
        printf("The address of your memory is %p\n",ptr);
        for (ref=ptr;ref < (ptr+bytes);ref++) {
            /*char is single quote, string is double quote*/
            *ref='N';
    }
    }else{
        printf("No memory allocated\n");
    }
    printf("Hit <return> to exit\n");
    getchar();
    printf("\n");
}

n0derunner