What is Virtual Memory?

Virtual Memory is a space where large programs can store themselves in form of pages while their execution and only the required pages or portions of processes are loaded into the main memory. This technique is useful as large virtual memory is provided for user programs when a very small physical memory is there.

In real scenarios, most processes never need all their pages at once, for following reasons :

Error handling code is not needed unless that specific error occurs, some of which are quite rare.
Arrays are often over-sized for worst-case scenarios, and only a small fraction of the arrays are actually used in practice.
Certain features of certain programs are rarely used.

Benefits of having Virtual Memory

Large programs can be written, as virtual space available is huge compared to physical memory.
Less I/O required, leads to faster and easy swapping of processes.
More physical memory available, as programs are stored on virtual memory, so they occupy very less space on actual physical memory.

What is Demand Paging?

The basic idea behind demand paging is that when a process is swapped in, its pages are not swapped in all at once. Rather they are swapped in only when the process needs them(On demand). This is termed as lazy swapper, although a pager is a more accurate term.

Demand Paging

Initially only those pages are loaded which will be required the process immediately.

The pages that are not moved into the memory, are marked as invalid in the page table. For an invalid entry the rest of the table is empty. In case of pages that are loaded in the memory, they are marked as valid along with the information about where to find the swapped out page.

When the process requires any of the page that is not loaded into the memory, a page fault trap is triggered and following steps are followed,

The memory address which is requested by the process is first checked, to verify the request made by the process.
If its found to be invalid, the process is terminated.
In case the request by the process is valid, a free frame is located, possibly from a free-frame list, where the required page will be moved.
A new operation is scheduled to move the necessary page from disk to the specified memory location. ( This will usually block the process on an I/O wait, allowing some other process to use the CPU in the meantime. )
When the I/O operation is complete, the process's page table is updated with the new frame number, and the invalid bit is changed to valid.
The instruction that caused the page fault must now be restarted from the beginning.

There are cases when no pages are loaded into the memory initially, pages are only loaded when demanded by the process by generating page faults. This is called Pure Demand Paging.

The only major issue with Demand Paging is, after a new page is loaded, the process starts execution from the beginning. Its is not a big issue for small programs, but for larger programs it affects performance drastically.

Page Replacement

As studied in Demand Paging, only certain pages of a process are loaded initially into the memory. This allows us to get more number of processes into the memory at the same time. but what happens when a process requests for more pages and no free memory is available to bring them in. Following steps can be taken to deal with this problem :

Put the process in the wait queue, until any other process finishes its execution thereby freeing frames.
Or, remove some other process completely from the memory to free frames.
Or, find some pages that are not being used right now, move them to the disk to get free frames. This technique is called Page replacement and is most commonly used. We have some great algorithms to carry on page replacement efficiently.

Basic Page Replacement

Find the location of the page requested by ongoing process on the disk.
Find a free frame. If there is a free frame, use it. If there is no free frame, use a page-replacement algorithm to select any existing frame to be replaced, such frame is known as victim frame.
Write the victim frame to disk. Change all related page tables to indicate that this page is no longer in memory.
Move the required page and store it in the frame. Adjust all related page and frame tables to indicate the change.
Restart the process that was waiting for this page.

FIFO Page Replacement

A very simple way of Page replacement is FIFO (First in First Out)
As new pages are requested and are swapped in, they are added to tail of a queue and the page which is at the head becomes the victim.
Its not an effective way of page replacement but can be used for small systems.

LRU Page Replacement

Below is a video, which will explain LRU Page replacement algorithm in details with an example.

Thrashing

A process that is spending more time paging than executing is said to be thrashing. In other words it means, that the process doesn't have enough frames to hold all the pages for its execution, so it is swapping pages in and out very frequently to keep executing. Sometimes, the pages which will be required in the near future have to be swapped out.

Initially when the CPU utilization is low, the process scheduling mechanism, to increase the level of multiprogramming loads multiple processes into the memory at the same time, allocating a limited amount of frames to each process. As the memory fills up, process starts to spend a lot of time for the required pages to be swapped in, again leading to low CPU utilization because most of the proccesses are waiting for pages. Hence the scheduler loads more processes to increase CPU utilization, as this continues at a point of time the complete system comes to a stop.

Thrashing