Real Vs Virtual Memory
Real memory in your computer is the physical RAM present. This memory is very fast. In some cases, you may not have enough physical memory. When this occurs, the computer needs to supplement the physical memory. This allows it to use more memory than what is physically installed.
This extra memory is called virtual memory. Virtual memory uses secondary storage to supplement physical memory, meaning the computer has access to more memory. The downside of this is that it is slower than physical memory.
Before I deep dive into how real and virtual memory works, I will first look at some terms and the basics of how memory works.
Volatile Memory Vs Non-Volatile
When working with computers, you will hear the terms volatile memory and non-volatile memory. Volatile is memory that requires power in order to keep its data. If power is lost then the data is lost.
Non-volatile memory is storage that does not require power to keep its data. Volatile is faster than non-volatile, so when the computer is using data, you always want to use volatile as much as you can to make your computer run as fast as possible.
Let’s now have a look at how the CPU gets this information.
Memory Controller
In order to get data to and from the CPU, a Memory Controller is used. In the old days the Memory Controller was a separate chip on the motherboard; nowadays, the Memory Controller is part of the CPU. However, its purpose remains the same. That is, it consists of circuits that manage data movements, which essentially controls the flow of data to and from the CPU.
The Memory Controller has a 64-bit data bus to the memory modules. If your computer has a dual channel, it will have two 64-bit data buses. If it has a triple channel, it will have three. The bus effectively moves data to and from the Memory Controller and the memory modules.
In order to determine what data is to be accessed, the CPU has an address bus. Each byte the computer can access is assigned a number. The address bus is set to this number, so the Memory Controller knows what data it needs to retrieve. In the case of a 32-bit CPU, the address bus is 32 bits wide. This effectively means, a 32-bit CPU can only address a maximum of four gigabytes of memory. This becomes problematic when you start getting close to this limit. Later in the video we will see why.
In the case of 64-bit CPUs, the specifications require the bus to be at least 48 bits wide. This means the CPU can effectively address 256 terabytes. That is quite a lot, but we will see later on in this video why we want this number to be quite high.
To gain a better understanding of how memory and virtual memory works, let’s take a look at how computers in the past would access memory to help us understand the key concepts.
The Old Days
For this example, let’s consider that we have an old 32-bit CPU that does not support modern memory management features. The computer has two gigabytes of memory installed. In order to access a specific byte in memory, the CPU utilizes an address, which is essentially a numerical value that selects the desired byte. In this example, the number of addresses for two gigabytes of memory would be over two billion. Since this is a 32-bit CPU, the largest number of addresses that can be accessed is just over four billion.
Since this example only has two gigabytes of memory, essentially upper memory is unused. However, the CPU can still make use of this address space. Additional devices like video cards have memory on them. In order to access the memory on the video card, the CPU uses a memory address just like it would be accessing main memory. Essentially, the video card’s memory is mapped to the address bus. The CPU accesses the memory on the video card just as it would access the memory modules.
If you are using a 32-bit system and you install four gigabytes of memory, you won’t be able to access all the memory. This is because the CPU needs to access other devices’ memory on the computer, for example, the video card. The CPU is only able to map one memory address to one location. Thus, all the memory addresses are not available, since the CPU requires access to device memory on the computer to operate. On some types of BIOS, you can change the amount of address space that is mapped to devices like the video card. This will increase the amount of memory your computer will be able to access; however, it will reduce the performance of the video card.
Before I start looking at how modern CPUs handle memory management, there is one last concept I will look at.
Memory Fragmentation
Modern CPUs don’t have the problem I am going to look at, but I will talk about it briefly to help you understand how modern CPUs work.
Let’s consider that there are two applications running on the computer. The first application requests some memory. The first application will then request some additional memory. The second application will also request some memory. After this, the first application requests some more memory.
We have allocated most of the memory for the computer. If the first application no longer requires some of its memory, it releases it. You can now see there is a gap in the memory, and this is referred to as memory fragmentation.
The problem now occurs when the second application requests more memory. The problem is that, while there is enough free memory to meet the request, there is not enough contiguous space available. Contiguous simply means that the memory needs to be allocated as a single section of memory, that is, it resides together in the one place. There is enough memory to fulfill the request when both blocks of memory are added together; however, neither block is big enough on its own.
Memory fragmentation prevents efficient allocation of memory. This ineffective memory allocation may prevent contiguous memory being allocated even when there is enough free space available.
So, let’s have a look at how modern CPUs deal with this problem, which will help with our understanding of how memory and virtual memory are managed.
Memory Translations
Modern computers use memory translation to allow contiguous memory blocks to be allocated, even when the memory is fragmented. Essentially what occurs is, the software has a physical address which is recorded in a page table.
The page table is used to map this address to the memory. Thus, the page table is used to store mappings. So, let’s have a look at what has occurred.
The application has been given a virtual address. This is translated using the page table to a physical address. The application sees a contiguous block of memory. However, in reality this block of memory could be spread out randomly across the memory module. The whole process is seamless and transparent to the application.
Now that we have an understanding of how memory translation works, let’s have a look at how we can use it with virtual memory.
Virtual Memory
Virtual memory allows the computer to access more memory than it physically has. To understand how this works, let’s consider that we have an application running on the computer. The application has a block of memory available to it; however, it is currently using more than is available.
Currently, the page table has four of the pages mapped to physical memory; however, it does not have enough memory to store the last page.
This page is currently stored on secondary storage, which in this case is the hard disk. When the application requests this page, as it is currently not in memory, the CPU requests the page from the operating system.
The operating system now swaps the page on the hard disk with one of the pages in memory and updates the page table. The application is unaware this is even occurring. This process is often called swapping, because the pages are physically being swapped between the hard disk and memory. Since the page table is also being updated, you may also hear it being called paging. Either term is correct and they are often used interchangeably.
If your computer is doing too much paging, this will slow down the applications on the computer and the user may notice the computer becoming sluggish or experience delays in responding to requests.
If you are experiencing problems like these, the solution will often be to add more memory to the computer. More memory means the computer will page a lot less. Some paging, for a computer, is normal and you should not be concerned if you see a little paging happening under normal operating circumstances. Let’s have a look.
Starting Applications In Swap Space
Different operating systems use different algorithms for memory management, but effectively they are trying to achieve the same thing. That is, attempting to keep commonly used pages in memory and less commonly used pages put into virtual memory. Even if you have a lot of memory, the operating system is still going to use some virtual memory.
Some operating systems, when the system is not under load, will attempt to pre-emptively determine which pages are likely to be swapped out later. The idea behind this is, if the system is suddenly put under a lot of load, the operating system has already done some pre-work. On some operating systems, the operating system will keep the page in memory and write the page to local storage. If the memory is needed later, the operating system essentially just frees the page and uses it as the data has already been written to virtual memory ahead of time.
There are also other times when virtual memory is used. It is common for UNIX based operating systems to start batch software in virtual memory rather than actual memory. Batch software is usually scheduled to run at certain times, often during the night. Since it is scheduled, it is generally not time critical and fast responses are not required. Thus, the software is loaded directly into virtual memory.
Essentially loading it directly into virtual memory means the pages are only swapped into real memory if they are asked for. Typically, software is generally loaded straight into memory and then the computer attempts to work out what to move to virtual memory. In order to do this, it needs to gather some statistics and try to predict what is unlikely to be used later. Batch software, in contrast, treats the problem as I don’t know what is needed, so will only swap in what is asked for. Thus, pages that are never used are never put into memory. In contrast, loading it all in memory, the operating system would need to work out or guess which pages are most likely never to be asked for.
Loading an application directly into virtual memory also helps when a large program is loaded, then exits very quickly. For example, a backup program may be loaded which does a quick check to see if tape media is in the drive but, when no tape is present, it exits. Doing the check uses very little memory compared to actually running the backup. Doing it this way makes sure we are not loading a full program into memory, using a small part of what was loaded and then exiting. It may not seem like much of a problem, but let’s consider if you swapped out other software in order to have it fit in memory only to use a small amount of what was loaded and then exit.
Loading software directly into virtual memory makes the software a bit sluggish when it starts up. This is because, effectively anything the program needs, it will need to ask for, then it will need to be swapped into memory before it can be accessed. In the case of batch programs, they often run as a background process and as long as they complete after a certain amount of time, then no one is concerned if it was sluggish when it first started up. For example, it doesn’t matter if a backup process that takes a few hours to complete was a bit slow when it started up. However, saving memory on large servers running lots of processes is more important than a sluggish start up.
Storage Location
Virtual memory is stored by default in Windows in the C: drive in the file called pagefile.sys. On a Linux installation, it is generally stored in a partition designed for virtual memory called a swap partition. It can be stored in a file if you prefer.
Using a swap partition is generally more efficient, since the operating system can access the data directly. A partition will always be contiguous since it is one block of data; however, a file may be segmented, particularly if it expands in size. A swap partition can also bypass the cache and write directly to the partition and avoid system overhead. Having said all that, the more modern Linux operating systems are very efficient, and you may not notice the difference between the two. You will generally get better performance from a swap partition, but the difference is not as noticeable as it used to be.
When swapping is done between virtual memory and real memory, this is called swapping or paging and these terms are used interchangeably. Now that we have an understanding of how virtual memory works, let’s throw a bit of a curve ball into what we have learned.
Virtual Memory Definition
Let’s consider a definition of virtual memory. If this was storage related, we could have a definition of allocating a section of storage to act as part of memory. This is what we have seen so far. Essentially when the computer needs more memory, it uses storage to supplement memory, swapping between them as required.
However, if I consider it in a software sense, I could say virtual memory gives an application the impression it has a contiguous working address space. Which is correct? Well, they are both correct in the right context. Thus, when talking about virtual memory, keep in mind the context in which it is being talked about. Let’s have a closer look and also bring the main concepts of this video together.
Putting It All Together
Let’s consider that we have a computer that has eight gigabytes of real memory. The computer also has 12 gigabytes of virtual memory. On your computer the virtual memory may be a fixed size, or it may be dynamic, expanding as needed until it runs out of space to expand into.
The computer will see this as 20 gigabytes of usable memory. In the case of physical memory, this will be connected using a 64-bit bus. If your computer has multiple channels with multiple memory modules, there may be multiple data buses connecting to the memory modules. Each data bus is 64 bits.
The 12 gigabytes of virtual memory on the hard disk is connected to the computer and seen as part of the usable memory. Since the computer can only access data in the memory modules and not the hard disk directly, when a request comes through for data stored on the hard disk, the operating system swaps the data on the hard disk with the memory module. As we saw earlier, this may also be referred to as paging.
Thus, in the case of virtual memory storage, this is storage that is used to increase usable memory.
In order for software on the computer to run efficiently, a translation layer is added so the software on the computer does not need to worry or know where the memory is stored. The translation layer is in the Memory Controller. This used to be a separate chip, but nowadays the chip is located inside the CPU.
The Memory Controller provides the translation layer. This is connected by an address bus. In x64 CPUs, the address bus needs to support a minimum of 48 bits, which means it can access 256 terabytes. This is the minimum and could be increased all the way up to 64 bits, but even in the largest computers, 256 terabytes should be enough. Well currently, but don’t quote me on saying that in 20 years, because who knows what will happen. If it is not, it can always be increased and remain compatible with older 64-bit software.
Now, let’s consider an application running on the computer. Let’s consider the application has access to a memory range. Some operating systems will give the software a dedicated address range, while others will allocate memory to the application as required. For this example, let’s consider that the software has a memory range.
The application requests memory to be allocated to it on two different occasions. After this, the application requests some more memory which is then allocated.
To make things interesting, let’s consider that the second amount of memory allocated is no longer required, so the application releases the memory, which makes it free for other applications to use. Now the interesting part. The application requires a very big block of memory, much bigger than any allocated to it before that. You can see there is a gap from the memory released earlier; however, it is too small to fit the new block in. But, since we have such a big memory range, we can allocate it to another location. Thus, the application does not need to worry about memory management, it just needs to ask for memory and if fragmentation occurs, it is not a problem; the memory range is so large, it just allocates free space.
So, in the case of software, virtual memory is the memory addresses the software can access. You can see that modern CPUs make memory management easy. The application simply requests what it needs and sees a contiguous block of memory.
Since we have come so far in this video, there is one more topic that I think is worth covering.
Second Level Address Translation (SLAT)
If you start using virtual machines, you may come across Second Level Address Translation or SLAT. Most newer CPUs should support SLAT, some older CPUs may not. To understand how it works, consider that we have two first generation virtual machines running. These two virtual machines are running on a host computer.
The two virtual machines in this case are running different operating systems. They run on top of our physical host computer, allowing the virtual machines to share the physical hardware of the host computer.
Let’s now consider that both operating systems work with their virtual CPU and thus have their own page tables for translating memory addresses.
Since the virtual machines are essentially running in a virtual environment, they share the hardware of the host computer, thus, the host computer runs its own page table for translating memory addresses. This means, first generation virtual machines require two translations when they access memory. One for the virtual machine and one for the host.
Now let’s consider we have a CPU with SLAT support. SLAT support means that the virtual machines can access the same page table used by the host, thus only one translation is required. Therefore, if you are planning to run virtual machines, it may be worth making sure that the CPU supports SLAT.
End Screen
That concludes this video on real and virtual memory. I hope you have found this video useful. Until the next video from us, I would like to thank you for watching.
References
“The Official CompTIA A+ Core Study Guide (Exam 220-1101)” pages 65 to 66
“Mike Myers All in One A+ Certification Exam Guide 220-1101 & 220-1102” pages 77 to 84
“What’s Virtual Memory? – Computerphile” https://www.youtube.com/watch?v=5lFnKYCZT5o
“48-bit computing” https://en.wikipedia.org/wiki/48-bit_computing
Credits
Trainer: Austin Mason http://ITFreeTraining.com
Voice Talent: HP Lewis http://hplewis.com
Quality Assurance: Brett Batson http://www.pbb-proofreading.uk