Episode 008 – free: Understanding Linux Memory Usage

The free command is a handy snapshot into your systems memory and how much of it is being used. In conjunction with other tools like top you can get begin to understand where your system resources are being utilized and weed out potential bottlenecks and bugs. But before jumping into the deep end in system analysis, you need to have a decent grasp on how the Linux kernel utilizes memory, or you initial observations may send you tearing through the interwebs looking for a solution to a problem that does not exist.

Invoked by itself free will show a standard output like this:

By default the values are reported in kilobytes. Depending on your version of free the following display options may be available:

  • -b – view in bytes (–bytes)
  • -k – view in kilobytes (default) (–kilo)
  • -m – view in megabytes (–mega)
  • -g – view in gigabytes (–giga)
  • –tera – view in terabytes
  • -h – view in human readable mode where values are “automatically scaled to the shortest three digit unit” (–human)

Depending on the age of your system, the last two options may not be available.

Displaying the above using the -h or human readable mode produces this:

A first glance at the first line might produce alarm! The system is reporting 1.5G of RAM with 1.4G being used leaving 74 megs of RAM available.  The current spec of this system running the free command is an Arch install using Fluxbox as the Window Manager, running Chromium (to write this post) with 5 tabs open, the Gimp with three images, Xchat, and Thunderbird. Not a lot going on there so why is the system using 1.4 GB of RAM? I thought Linux was easy on memory, especially when using a light weight window manager?

If I kill a few of these processes off, specifcally: Thunderbird, a few terminal windows, an errant sflphone and sflphone daemon process running from an iax connection a few days ago I bring my usage down a bit:

I now bring my usage down to 854 Megabytes of RAM with 647 Megabytes free according to the first line. That is still a lot of memory for a few applications. Where is the vaunted low footprint Linux is supposed to offer?

Just looking at the first line of free is not a clear picture of memory usage that one expects because it does not take into account how the memory is being utilized, particularly the cache buffers. That is where the second line comes in. The second line details the amount of memory being utilized by programs and not by the buffer cache. Hence the “-/+” which shows the amount of memory being utilized minus the buffer cache and the amount of free memory with the amount utilized by the buffer cache returned.

So what exactly is buffer cache? Buffer cache is an area of free physical memory utilized by the kernel to store data that has been recently read from a disk and/or utilized by a program, or that is ready to be written to the disk. The logic behind this utilization of memory is that reading from disk is slow compared to reading from memory. So if a bit of data is read from the disk there is a pretty good chance that data might be used again in the near future. So instead of just losing the data to the bit bucket, the data is stored in memory. This retrieval is far faster from memory than having to access it from disk again.

The same principle holds for writing to a disk. If a program writes data to disk it may be more efficient to cache this write in the buffer cache and allow the program to continue on instead of waiting for the data to be physically written to the disk. The kernel will handle writing the information to disk and the program can continue with what it was doing.

The buffer cache is dynamically allocated by the kernel and if you observe a running system over a period of time you will see the cache grow and shrink. You may even observe it consume over 90% of your memory as in the first screen shot above and become alarmed at such low resources available. But without taking the second line into consideration, you are missing the full picture.

As applications consume more memory, the kernel will begin dumping out any data from the buffer cache on a least used basis returning the cache to be used by actual applications that need it. Thus for all intents and purposes you can consider the memory used by the buffer cache to be “free” as you would expect to see when looking at how much memory is free in your system. To the kernel, though, it is memory in use, but a block of memory that will be immediately available to a program should it be required.

That being said, it is important to note that as applications consume more RAM, the performance of your system will begin to degrade after a certain point. If over 90% of your system memory is actually being used by applications this reduces the amount of memory available to the buffer cache so the performance boost the buffer cache offered in the past may not be available as physical RAM becomes consumed. Again, you can observe this by paying particular attention to the “-/+ buffers/cache” line.

Having a better understanding of the cache buffers will shed light on the rest of the output of free. The last three columns of the first line indicate the amount of memory being “shared,” utilized by the “buffers,” and utilized by the “cache.” The shared column is obsolete and can be ignored. But if you add up the amount of buffers and cached memory utilization and subtract it from the used column, you should get a value close to what is reported as used memory in the “-/+ buffers/cache” line. Or add it to the free value to get the free value of the second line.

The Swap line (3rd line of output) shows the amount of swap space being used by the system. This should be the value of your swap partition(s).

There are a few more options to the free command to outline. The first is the -t or –total options which displays a fourth line totaling the sum of the first line and the swap line of free for total, used, and free. This value will be displayed in bytes by default or in the unit value you specified:

free -mt

This would display the amounts in megabytes.

There is the -o or –old which displays the information of free in the “old” mode which essentially removes the buffers/cache line.

The -s or –seconds switch will display the output of free and then continue to display the output of free every n seconds specified:

free -h -s5

This will display the output of free in human readable format every 5 seconds until killed by ctrl-c.

The -s can be used with the -c or –count options which will display the output of free for n iterations with n seconds between each iteration:

free -m -s5 -c10

The output of free will be displayed every 5 seconds for 10 iterations. After the 10th output of free control will be returned to the shell and free will no longer be called. The -c option must be used in conjunction with the -s options.

Finally there is the –si option which will show the values in the power of 1000 as opposed to 1024.

The free command is a window into the performance of your system, but understanding how memory is utilized by the Linux kernel is important to determine the health of your system. By focusing on the amount of used and free memory in the first line of free, and not taking into account the amount of memory being used by the cache buffers, you may be setting yourself up for unnecessary concern.

Bibliography:


If the video is not clear enough view it off the YouTube website and select size 2 or full screen.  Or download the video in Ogg Theora format:

Thank you very much!

This entry was posted in Uncategorized. Bookmark the permalink.

4 Responses to Episode 008 – free: Understanding Linux Memory Usage

  1. Prince says:

    Just wanted to say that loving the series a lot. Didnt had a chance to say thanks before but do appreciate the effort.

  2. Rory McPherson says:

    Thanks for the taking the time to put all of this information together – it must take a lot of your time. Thank you!

  3. Larry North says:

    Hi dann,

    I would like to express my appreciation of your efforts.
    I was a high school IT / programming teacher in Sydney for 25 years and so I understand just how much effort goes into each and every page as well as th video resources.
    I found out about your site from the latest Steve ‘Door-to-door’ podcast and he also thinks it is great.
    I have just got two raspberry pi computers and so am going to make a concerted effort to learn shell scripting. Your series is an excellent primer for this. I will be posting a link to your site on the various user groups I am a member of.

    Thanks again

    Larry North

    • dannSWashko says:

      Thank you for the compliments, I am glad this site is a service to you. If you have any questions or feedback please feel free to post or email.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>