Memory hierarchy

The computer memory holds the data and instructions needed to process raw data and produce
output. The computer memory is divided into large number of small parts known as cells. Each
cell has a unique address which varies from 0 to memory size minus one.

Computer memory is of two types: Volatile (RAM) and Non-volatile (ROM). The secondary
memory (hard disk) is referred as storage not memory.

But, if we categorize memory on behalf of space or location, it is of four types

o Register memory
o Cache memory
o Primary memory
o Secondary memory

Memory hierarchy
In the Computer System Design, Memory Hierarchy is an enhancement to organize the
memory such that it can minimize the access time. The Memory Hierarchy was developed
based on a program behaviour known as locality of references. The figure below clearly
demonstrates the different levels of memory hierarchy:

• Memory hierarchy is the hierarchy of memory and storage devices found in a computer

system.
• It ranges from the slowest but high capacity auxiliary memory to the fastest but low
capacity cache memory

Need-
There is a trade-off among the three key characteristics of memory namely-

• Cost

• Capacity

• Access time
Memory hierarchy is employed to balance this trade-off.
Memory Hierarchy Diagram-

This Memory Hierarchy Design is divided into 2 main types:

1. External Memory or Secondary Memory –

Comprising of Magnetic Disk, Optical Disk, Magnetic Tape i.e. peripheral storage
devices which are accessible by the processor via I/O Module.
2. Internal Memory or Primary Memory –
Comprising of Main Memory, Cache Memory & CPU registers. This is directly
accessible by the processor.

Level-0:
• At level-0, registers are present which are contained inside the CPU.
• Since they are present inside the CPU, they have least access time.
• They are most expensive and therefore smallest in size (in KB).

• Registers are implemented using Flip-Flops.

Level-1:
 • At level-1, Cache Memory is present.

• It stores the segments of program that are frequently accessed by the processor.
• It is expensive and therefore smaller in size (in MB).
• Cache memory is implemented using static RAM.

Level-2:
• At level-2, main memory is present.
• It can communicate directly with the CPU and with auxiliary memory devices through
an I/O processor.
• It is less expensive than cache memory and therefore larger in size (in few GB).
• Main memory is implemented using dynamic RAM.

Level-3:
• At level-3, secondary storage devices like Magnetic Disk are present.

• They are used as back up storage.

• They are cheaper than main memory and therefore much larger in size (in few TB).

Level-4:
• At level-4, tertiary storage devices like magnetic tape are present.
• They are used to store removable files.
• They are cheapest and largest in size (1-20 TB).

Observations-
The following observations can be made when going down in the memory hierarchy-

• Cost / bit decreases

• Frequency of access decreases
• Capacity increases
 • Access time increases

Goals of Memory Hierarchy-

The goals of memory hierarchy are-

• To obtain the highest possible average access speed

• To minimize the total cost of the entire memory system
• No single memory sub-system can be considered “best” in all categories

Characteristics of Storage

The different characteristics of storage devices are as follows:

Volatility

Volatile memory needs power to work and loses its data when power is switched off. However,
it is quite fast so it is used as primary memory.

Non - volatile memory retains its data even when power is lost. So, it is used for secondary
memory.

Mutability

Mutable storage is both read and write storage and data can be overwritten as required. Primary
storage typically contains mutable storage and it is also available in secondary storage
nowadays.

Accessibility

Storage access can be random or sequential. In random access, all the data in the storage can
be accessed randomly and roughly in the same amount of time. In sequential storage, the data
needs to accessed in sequential order i.e. one after the other.

It is the time interval between the read/write request and the availability of the data. As we
move from top to bottom in the Hierarchy, the access time increases.

Addressability
Each storage location in memory has a particular memory address. The data in a particular
location can be accessed using its address.

Capacity

The capacity of any storage device is the amount of data it can hold. This is usually represented
in the form of bits or bytes.

It is the global volume of information the memory can store. As we move from top to bottom
in the Hierarchy, the capacity increases.

Cost per bit:

As we move from bottom to top in the Hierarchy, the cost per bit increases i.e. Internal
Memory is costlier than External Memory.

Performance

Performance can be described in terms of latency or throughput.

• Latency is the time required to access the storage. It is specified in the form of read
latency and write latency.
• Throughput is the data reading rate for the memory. It can be represented in the form
of megabytes per second.

Some Important Definitions

Register memory

Register memory is the smallest and fastest memory in a computer. It is not a part of the
main memory and is located in the CPU in the form of registers, which are the smallest data
holding elements. A register temporarily holds frequently used data, instructions, and memory
address that are to be used by CPU.

Registers are the smallest data holding elements that are built into the processor itself. These
are the memory locations that are directly accessible by the processor. It may hold an
instruction, a storage address or any kind of data such as a bit sequence or individual characters.
For example, an instruction may specify that the contents of two defined registers be multiplied
together and then placed in a specific register.
Auxiliary Memory
Auxiliary memory is known as the lowest-cost, highest-capacity and slowest-access storage in
a computer system. Auxiliary memory provides storage for programs and data that are kept for
long-term storage or when not in immediate use. The most common examples of auxiliary
memories are magnetic tapes and magnetic disks.
A magnetic disk is a digital computer memory that uses a magnetization process to write,
rewrite and access data. For example, hard drives, zip disks, and floppy disks.
Magnetic tape is a storage medium that allows for data archiving, collection, and backup for
different kinds of data.

Main Memory
The main memory in a computer system is often referred to as Random Access Memory
(RAM). This memory unit communicates directly with the CPU and with auxiliary memory
devices through an I/O processor.
The programs that are not currently required in the main memory are transferred into auxiliary
memory to provide space for currently used programs and data.

Cache Memory

The data or contents of the main memory that are used frequently by CPU are stored in the
cache memory so that the processor can easily access that data in a shorter time. Whenever the
CPU requires accessing memory, it first checks the required data into the cache memory. If the
data is found in the cache memory, it is read from the fast memory. Otherwise, the CPU moves
onto the main memory for the required data.

Primary Storage

This is also known as the main memory and is the memory directly accessible by the CPU. All
the instructions are executed in the main memory by CPU and the data required by these
instructions is also stored in main memory.

Main memory primarily consists of the RAM which is volatile in nature. It is also quite small
compared to secondary memory and expensive as well.

Secondary Storage

Secondary or external storage is not directly accessible by the CPU. The data from secondary
storage needs to be brought into the primary storage before the CPU can use it.
Secondary storage is non volatile i.e. the data stored is not lost when power is switched off.
Mostly hard drives are used as secondary storage.

Tertiary Storage

This is a third level of storage that mostly contains data that needs to be archived. This is
because it is quite slow. Data is normally retrieved from tertiary storage to primary storage
when it needs to be viewed.

Tertiary storage is mostly formed of tapes and disks.