Computer Architecture Essesntials by ARM

Module 1, Intro
[music] I'm Richard Grisenthwaite, Chief Architect at Arm. What does a Chief
Architect do? The Chief Architect is responsible for the evolution of the Arm
architecture. One of the things about an architecture is, you want your software to
continue running on future products. So it's a very long term commitment, and the
role of the chief architect is essentially to curate the architecture over time,
adding new features as needed by the partner. Why are microprocessors so important
and ubiquitous? I think the fundamental reason is that they are a general purpose
device. They describe a basic instruction set that does the stuff that you can
construct pretty much any program out of. Indeed, they're Turing complete, which
means, by definition, you can create any computable problem, solving a computable
problem with the processor. They're not customized to any particular usage, but they
do two things really rather well, which are data processing and decision making. In
other words, having, for example, added two numbers and compared them, you're then
taking a branch and making a decision based off that. Now in reality, an awful lot
of general purpose problems are ones that involve: work out some value or some
criteria, compare against that, and then make a decision on it. And essentially that
allows you to solve a huge number of different problems and that's why the
microprocessor has become so ubiquitous. What is the difference between architecture
and microarchitecture? Architecture is what it has to do, and microarchitecture is
how it does it. So for example, we will define in the architecture a set of
instructions that do "add" or "load" or "branch", but it says nothing about whether
you've got a ten-stage pipeline or a three-stage pipeline. It says nothing about
branch prediction. All of those sort of features which actually determine the
performance of the device; all of those are part of the microarchitecture. The key
point is that software that is written to be compliant on the architecture will
actually run on lots of different implementations, lots of different
microarchitectures that essentially implement that architecture in different ways,
choosing a trade-off between the power, the performance and the area of the design.
What does it take to build a commercially successful processor today? If you
actually start from scratch with an established architecture, but we want to create
a new microarchitecture, we reckon, for a high end processor, you're talking about
three-to-four hundred person-years' worth of work in order to create a properly
competitive multi-issue out-of-order machine compared with the state-of-the-art that
you can get from Arm. In addition to that — and that's just to come up with the RTL
— in addition to that, you've then got to do the implementation. If you're going to
build that on a three-nanometer process, the leading edge to get the best possible
performance, you're talking about tens of millions of dollars for the mask sets.
There's a whole bunch of software you've got to go and build on top of that and do
all of that work. When we've looked at companies that are interested in actually
starting up, taking an Arm architecture license — say I want to go and build my own
business — we reckon that you need to be prepared to spend getting on for half a
billion dollars before you're actually going to be successful because it takes time.
Your first product is not necessarily going to be fully competitive because it would
be slightly surprising if the first thing that you built was as good as what people
have been building for many years. It takes time to build up that expertise and
skills. And so you're going to see a couple of iterations before even the best teams
end up really being competitive. And so as I say, I think if you went from nothing
and wanted to essentially create something, and that's using the Arm architecture
with all of its existing software and so on, you're talking the best part of half a
billion. What is the Arm business model? The Arm business model fundamentally is the
licensing of semiconductor IP to as wide a range of companies as possible and in as
many ways as possible, in order to maximize the uptake of the IP. When we talk about
IP, we're talking about essentially designs for processors and we license that both
as an architecture license, where you effectively gain the right to build your own
processor to the specification of the Arm architecture, or an implementation license
where we are licensing, actually, an implementation that is compliant with the Arm
architecture in the form of some richer transfer-level RTL. What makes Arm different
from its competitors? So if you go back to where we were when Arms started out in
the early 90s, there were many, many different architectures available and they were
all kind of doing the same thing but slightly differently. They would have different
instructions, different instruction sets, and so software needed to be ported. A
huge part of Arm's success actually comes from the fact that we created a business
model of licensing the IP to make it very easy for people to build processors, to
incorporate the designs into their SoCs, into their systems. And that then made it
very straightforward for people to be able to use the Arm architecture. Now what
this then meant was, people said: I will port more software to this because it's
more widely available and you get this positive feed-forward effect whereby more
availability of hardware encourages more software, encourages more hardware, and so
on. And essentially that meant that a lot of people said: there's no point in me
having a different architecture. I'm not getting a valuable difference from doing
that. All I've got is, kind of, a needless change to the software that I need to
make. So actually the whole model, we went on at Arm, which was: let's license our
IP to a great number of different players to come up with different solutions to
meet the form factor of a camera or a PDA, or whatever it was back in the day. Those
things made it much more straightforward for people to incorporate our technology.
[music]

Module 1, Video 1
Computer architecture is the study of tools and techniques that help us to design
computers. More precisely, it helps us understand how to meet the needs of
particular markets and applications, using the technology and components that are
available. For example, we might need to produce the chip at the heart of a
smartphone using 10 billion transistors and a power of less than two Watts. How do
we achieve the performance a customer wants? The challenge is a fascinating one, and
one that requires a broad understanding. For example, what target market are we
designing for? What are the characteristics of the applications to be run? How will
programming languages and compilers interact with the microprocessor? How best to
craft the narrow interface between the hardware and software? How to organize the
components of our microprocessor? And how to design a circuit, given the
characteristics of individual transistors and wires? Like many design problems,
computer architecture requires many trade-offs to be made and evaluated. Each design
decision will impact trade-offs between size, performance, power, security,
complexity, and cost. Trade-offs must be re-evaluated regularly, due to advances in
fabrication technology, applications, and computer architecture. Computer
architecture must be grounded in quantitative techniques and experimentation, but
the endless number of possible designs means that the field depends on a high degree
of human ingenuity and art. Perhaps surprisingly, the earliest computers and today's
most advanced machines have much in common. They both execute a stored program
constructed from machine instructions. These instructions perform simple operations
such as adding two numbers. Nevertheless, greater numbers of faster transistors, and
the application of a relatively small number of computer architecture concepts, have
enabled us to construct machines that can perform billions of instructions per
second, and shrink these machines to fit in hand-held battery-powered devices. It is
this rapid progress that has supported breakthroughs in machine learning, drug
discovery, climate modeling, and supports our modern world where computation and
storage are almost free. The task of designing a microprocessor is split into
different levels of abstraction: "Architecture;" "Microarchitecture;" and
"Implementation." "Architecture" focuses on the contract between programmers and
hardware. It allows compatible families of microprocessor products to be built. The
ARMv8-A architecture is an example of this. Architecture includes the "Instruction
Set Architecture," or ISA, which defines what instructions exist. It also defines
precisely the behavior of memory and other features needed to build a complete
processor. "Microarchitecture" focuses on the organization and structure of the
major components of a microprocessor. It has to match the rules set by the
architecture. The microarchitecture still has flexibility though; and so the
implementation specifies the circuit detail precisely. This culminates in the exact
circuit design for manufacture. Each of these levels is vital, and each comes with
its own challenges and opportunities.

Module 1, Video 2
At the beginning of the 20th century, "computers" were people employed to perform
calculations. These computers used mechanical calculators to help them perform
arithmetic. They followed instructions to decide what calculation to perform next.
These instructions defined the algorithm or program they were executing. They would
consult paper records to find the inputs to their calculations, and would store
their intermediate results on paper so they could refer back to them later. A modern
electronic computer is organized in a similar way. We will look in more detail at
these components of a microprocessor in the next video, but for now let's look at
how it operates. Microprocessors also follow instructions one by one, and then
perform relevant calculations. This idea of fetching instructions from memory and
executing them is called the "Fetch-Execute Cycle." In the "Fetch" stage, the
computer reads the next instruction from the program. This instruction is encoded in
binary as ones and zeroes, so it must be decoded to understand what it means. This
is done in the "Decode" stage. Once it is clear what to do, we move to the "Execute"
phase. This can involve different tasks such as reading memory, performing a
calculation, and storing a result. Once done, the computer is then ready to begin
the cycle again, by fetching the next instruction. Instructions are normally fetched
sequentially in order, but some special instructions called "branches" can change
which instruction will be executed next. For branches, a calculation determines the
next instruction. This can mean evaluating a condition, or reading a register to
determine the next instruction's location. Branches allow computers to make
decisions, and to re-use instruction sequences for common tasks. A modern computer
program, like a web browser, contains millions of instructions, and computers
execute billions of instructions per second, but they all conceptually follow this
"Fetch-Execute Cycle."

Module 1, Video 3
Modern microprocessors are circuits built using anywhere from 1,000 to 100 billion
tiny transistors. The key to designing circuits with such huge numbers of parts is
to build them from re-usable blocks. Let's take a look at some of these. Each
transistor is an electrically-controlled switch. When there is too little voltage at
the gate, the switch is off, so the electrical signal cannot propagate from the
drain to the source. However, when there is sufficient voltage at the gate, the
switch is on, so the signal does propagate. When designing processors, we use
digital electronics. The only voltage or current values we consider represent zero
and one, enabling us to build robust circuits from imprecise components, even in the
event of electrical noise or manufacturing imperfections. This means our circuits
represent binary numbers, since they only have two states: zero and one. We use
transistors to build increasingly complex circuits. We can design circuits that can
remember binary values, or select between multiple inputs, or even do arithmetic,
like addition. We can then use those circuits as building blocks in even larger
designs. When designing a digital system, we must keep everything synchronized to
control when its behavior occurs. To do this, we use a "Clock Signal," which is a
wire in the circuit whose signal cycles between zero and one. We measure the rate in
Hertz. For example, if you hear a processor has a clock speed of two Gigahertz, it
means 2 billion ones and zeroes per second. The maximum speed of the clock signal is
determined by the longest, and therefore slowest, path in the circuit between 2
clocked flip-flops. This is referred to as the "Critical Path." The signal must have
time to propagate all the way along the critical path before the clock cycle
completes. A microprocessor needs to do many types of arithmetic. For this, we build
an "Arithmetic Logic Unit" or "ALU." This circuit receives 2 numbers as input, as
well as an indication of what operation to perform, for example, addition or
subtraction. In addition to logic, a microprocessor needs memory. Memory is
organized as arrays of memory cells that are able to store many "Words" of data. A
specific word, commonly 32 bits in length, can be accessed by specifying its
"Address." Each address is a number that indicates the location in the memory that
should be read or written. Memory cells range from hundreds of bits to millions of
bits in size, but larger ones are slower to access, as signals and their long
internal wires take longer to propagate. For that reason, almost all microprocessors
include at least two types for storing data: a big slow "Data Memory," and a small
fast memory called a "Register File." In reality, the "Data Memory" may be
implemented using many different sizes of memory, as we'll see in Module 4. As well
as storing data in memory, we also use some memory to store the instructions. We
need a way to keep track of which instruction we will fetch next, so we have a
"Program Counter." This stores the address in Instruction Memory of the next
instruction to be accessed. Since instructions are encoded in binary, we also have
"Instruction Decode Logic" that converts that binary to the various signals needed
to control the microprocessor.

Module 1, Video 4
We've already seen how a microprocessor is controlled by instructions. But what are
they really? An instruction is a simple command that the microprocessor hardware can
perform directly. We write them as text like this to make them easier to read. But
for the microprocessor we encode them in binary. We use a program called an
assembler to translate between the human text and the binary. In this video we'll be
looking specifically at an Arm instruction set called A64 but other instruction sets
follow similar principles. Arithmetic and logic instructions are the simplest type
of instruction. The first word tells us what operation will be performed such as
addition or multiplication. The values after this tell the processor where to put
the result and where to get the inputs. Values starting with X are addresses in the
register file. Arithmetic instructions read one or two registers and then put the
result into a third register. Branch instructions are used to make decisions and to
repeat instructions. Normally the microprocessor executes instructions in sequential
order but branches change that, and explicitly tell the microprocessor the address
of the instruction to run next. This is done by giving the address of the next
instruction in the instruction memory. Some branches are unconditional, meaning they
always occur and always affect the next instruction address. Other branches are
conditional, meaning the processor will perform a calculation to decide whether to
follow the branch or to continue executing instructions sequentially following the
branch. These are preceded by comparison instruction to calculate the condition.
Loads and stores are the instructions for accessing the data memory. Loads copy
values from memory to the register file. Stores do the opposite. In both cases, the
instruction needs to know the address in the data memory and the location in the
register file to copy between. For data memory, loads and stores read an address
from a base register. They can also optionally add to this base address by reading
another register, or by simply specifying a number in the instruction itself. Using
sequences of instructions we can build programs. Here is an example of a small
program that implements Euclid's greatest common divisor algorithm. Let's take a
look at it working, one instruction at a time. To start with, the input stored in X1
and X2 are compared. If they're equal, because we have found the greatest common
divisor a conditional branch instruction moves to instruction 7. If they're not
equal, another conditional branch instruction can be used to determine whether X1 is
smaller. Finally, we use an arithmetic instruction to subtract either X1 or X2
depending on which was larger and then unconditionally branch back to the start.
Here we can see an example of the program running. One instruction executes at a
time. As we step through the program, the values in the registers X1 and X2 are
shown. And we see these are modified each time we reach instruction 3 or 5. The
instructions repeat in a loop, as is common for many different types of program.
Each time round the loop instruction 0 and 1 are checking whether or not to continue
the loop. When they detect X1 and X2 have become equal, the program finishes and the
processor moves on to the next task.

Module 1, Lab
In this exercise, we're going to be using ASim. ASim is a behavioral simulator for a
subset of the Arm AArch64 instruction set. Behavioral simulator means that it's not
simulating real circuit level details of a microprocessor it's just simulating the
behavior of each instruction as it runs. Nevertheless, behavioral simulators are
vital tools for computer architects. We use them to check the designs we've built,
match with the behavior we intended and we can also use them to explore new ideas,
for example adding new instructions to the instruction set. We're just going to be
using it though to get familiar with the basics of Arm AArch64. So we can create a
new file which will allow us to type in some Arm AArch64 instructions and when we
press assemble that will cause ASim to begin the simulation of the instructions. If
we scroll down we can see that below we've put a little cheat sheet of Arm AArch64
instructions that you can use as a reference. Do note that these are just examples
though. So for example this first one is mov X1 X2 and the descriptive text says
that this copies the value of register X2 to register X1. But equally that would
imply that if we use mov X3, X4 instead that would copy the value of X4 to the
register X3. So we don't have to just use these exact instructions, we can tweak
them as we need to, to achieve our goals. So going back to the ASim, let's say that
we wanted to add the number two to the number three. In our cheat sheet we could see
that there is an 'add' instruction but it requires that the inputs for the addition
are stored in registers. So first of all, we're going to need to load up two
registers with the numbers that we want to add in this case two and three. So
looking at our cheat sheet again we can see that there's an instruction mov that
allows us to do that. So if I write mov X2, #3 this would cause the register X2 to
be loaded with the value 3 and similarly I could write mov X1,#2 and this would
cause the register X2 X1 to be loaded with the value two. Last but not least, we
could then do the addition we wanted to do, such as add X3 X1 X2. This will cause X1
to be added to X2 and the results stored in X3. If I press assemble now, we will see
that the UI changes slightly. Here we can now see the simulated state of the
processor. ASim is only going to, because it's a behavioral simulator it's only
going to show the state before and after each instruction. And so right now we are
before executing the first instruction. That's the yellow highlighting there and we
can also see it in the simulated memory. If I press step we can now see that that
instruction has completed, and it's before executing the mov X1 X2. And notably we
can see that X2 has been loaded with the number 3 which is exactly what we would
expect. If I press step again, we can see that mov X1 X2 has now occurred. MOV X1
#2, sorry has now occurred which has loaded the value 2 into the register X one. And
last but not least, if we step again we see that X3 has become equal to five which
is exactly what we would expect if registry X1 was added to registry X2. So this
allows us to get a feel for what it's like for a machine to execute these
instructions. We can reset back to the beginning if we want to watch it go through
again. If we want to do an edit for example adding a new instruction we can just do
that. Press assemble and that will add the new instruction to our program, and we
can see its effects by simulating. Now in the exercise you will be invited to take
an existing program and add 1 new instruction to it at the indicated position. Do
note that when you assemble the exercise program, you'll be taken to a gray bit of
code, which is our testing code. But as you step, you'll see that the simulator
flicks between the testing code and the code that we wrote the exercise one code.
You can also click between them using this. Feel free to try to work out what our
testing code does but you can just ignore it if you want to. The point of the
exercise is just to add the instruction in the indicated position. When you're
comfortable you've got the right instruction you can press run to get all the way to
the end. And if you really think that's correct if you scroll down to the bottom of
the page you'll see the submit button which you can use to send in your answer. Best
of luck.

Module 2, Intro
[music] Hello, my name is Martin Weidmann. I'm an engineer and product manager with
Arm's Architecture and Technology Group. I look after the A-profile architecture and
I maintain Arm's Interrupt Controller specifications. Computer architecture is
sometimes called a science of trade-offs. Why is everything a trade-off when it
comes to designing a processor? Let's take an example of where we've had to make
trade-offs when developing process architecture. So the Arm architecture has a
feature called Pointer Authentication. This is often abbreviated to PAC for Pointer
Authentication Code. What this feature is trying to do is protect against a form of
attack called ROP and JOP. These are Return Orientated and Jump Orientated
Programming, and it's where an attacker tries to subvert things like the call stack
to run legitimate code, but in ways that weren't expected by the programmer or the
compiler. PAC or Pointer Authentication tries to defend against this kind of attack
by using part of an address to provide an encrypted signature. So we can check the
signature and the address match and if they don't, we can spot an attack in
progress. So why is this a trade-off? Well, because to add security, we want that
signature to be as big as possible. The bigger the signature, the more bits we use
for that, the stronger cryptographically that signature is. The trade-off is: the
more bits we use for the signature, the fewer bits we have available for other
things, such as the address. So you can have a big signature with a small address,
but if you want the address to get bigger, then you get a smaller signature, and
that's then cryptographically weaker. So the trade-off we have to make when
designing a technology like that is: What's the right amount of bits for the
signature? What's the strength of cryptography we need from that signature in order
to get the design goal, which is to defeat these attacks and give us more robust
computing? What sort of guiding principles do you use when designing a processor? So
when you're designing a processor, the key thing you have to bear in mind is: What's
it going to be used for? What you don't want to end up with is a very expensive
paperweight. So we need to understand the requirements that the processor has to
meet. We have to understand the design trade-offs we're making and how they work
into meeting those requirements. We also have to consider not just the processor
itself, but how we're going to show that that processor is correct. We'll put as
much time, if not more, into testing and validating the design as we do into
designing it. How do you design a new microprocessor? If you wanted to create a new
processor from scratch, the first thing you're going to have to do is understand the
market that that processor is going to address and to then build a team to design
that processor. There isn't such a thing as one processor for every possible market.
The requirements for something like an embedded microcontroller are going to be very
different to what you want from the processor in your mobile phone, your laptop, or
your server. So you need to understand those requirements as the first step into
building a new processor. What determines the best design for a microprocessor? So
when you're designing a processor, you need to work out what the best design for a
given market or application is going to be. There's no magic formula for this. It's
going to depend a lot on what you're trying to achieve with that processor. You need
to understand things like the power requirements, the performance requirements. Is
it going to work in a highly noisy electrical environment? There's a big difference
between the reliability requirements you need from something like a watch versus a
satellite. So you would take those requirements and you'd work out what the best set
of trade-offs is going to be, and that's an art more than it is a science. How do
the underlying technologies contribute to this best design? A lot of technologies go
into our processor. There's the design of the microarchitecture, the implementation
of the processor. There's the silicon process you're going to use, how you're going
to integrate that processor into an SoC or ASIC. Is it a single die, or is it going
to be using chiplets or multiple sockets? All of those different technologies are
going to be factors into how you design the process or what trade-offs you make, and
what performance and power you get out of the design once you're done. In reality
there may be many different 'best' designs, so how do you pick one? So when you're
designing a processor, what you want is the best design. But often there isn't "a"
best design, there's just different trade-offs. You have to decide what the best set
of trade-offs is for the particular use case you're going for. And that's also going
to depend on: Is this a device that will be off the shelf, used for lots of
different applications—a general purpose processor? Or is this being designed for
one specific use case? Again, there isn't really a magic bullet or single answer for
this type of question. You need to understand how the processor is going to be used
and then use your experience to judge the trade-offs, and what will give you the
best mix of power, performance, area, cost, reliability for your target use case.

Module 2, Video 1
[music] In this module, we're going to explore how to improve the simple
microprocessor design from Module 1 in order to allow it to execute programs more
efficiently. First, let's find out how long a program takes to execute. The time
taken to perform the average instruction is equal to the number of clock cycles
taken to perform an instruction multiplied by the duration of one clock cycle. The
time taken to run our program is found by multiplying the average time to perform an
instruction by the number of instructions in our program. How could we make this
faster? One thing we could try is to reduce the number of instructions in a program.
We might be able to optimize the code removing unnecessary and repeated work and
selecting instructions to minimize code size and maximize performance. We could give
our microprocessor the ability to perform more operations in order to help
programmers or compilers further reduce the number of instructions in their program.
For example, allowing the loading of two data values at the same time might allow
fewer instructions to be used in the program. The downside to this approach, is that
adding more instructions will require extra circuitry in the processor and therefore
we likely increase the clock period. If the extra instructions are rarely used this
could even mean an overall decrease in performance. We see this theme often in
computer architecture trade-offs that we have to carefully balance. Another approach
is to use faster transistors perhaps constructed from a more recent fabrication
technology. This would reduce the clock period but may increase costs. The rest of
this module focuses on an optimization to reduce the clock period called pipelining.
This is the most important optimization we use when designing processors. It uses a
similar concept to an assembly line in a factory where work can start on the next
item before the previous one finishes. Let's take a closer look. Imagine that each
instruction has to go through four circuits in a processor. If we attempt to do all
of these in one clock cycle this means our clock period is the latency of all four
circuits added together. If we were to pipeline this, we would add a pipeline
register in the middle. This divides the circuit into two sections called stages.
Notice that although each instruction takes a similar amount of time to travel down
the whole pipeline, the pipeline design can execute nearly twice as many
instructions per second. The throughput has doubled. This is because we can set the
clock period much shorter. It's now the maximum latency of the two stages. We can
pipeline into many stages and this allows for much faster execution of programs.
Unfortunately, though, pipelining a real microprocessor design is not quite as
simple because the processor has various feedback signals and loops in the circuit.
In the next video, we'll take a look at the challenges of pipelining in practice.
[music]

Module 2, Video 2
[music] In this video, we're going to look at applying the pipeline optimization to
a realistic microprocessor design. In the first module, we met the components of a
microprocessor, so let's look at how these are really connected. This diagram shows
all the connections needed for a real, unpipelined microprocessor. Each clock cycle,
the processor starts by fetching an instruction from the instruction memory. Once
the instruction reaches the decode logic, it is decoded to produce the control
signals necessary to execute it. The exact control signals vary depending on the
type of instruction. For example, arithmetic instructions access the register file
and interact with the ALU. Ultimately, no matter how the instruction was executed,
the last step of each clock cycle is to update the program counter. This is done by
the branch unit. For non-branch instructions, this just means incrementing the
program counter. However, for branch instructions, the branch unit has to do some
calculations. When we apply our pipeline optimization to this design, we face some
challenges. The design has several loops because instructions have dependencies. How
can we break these cycles? The key observation is that not every instruction is the
same. In real programs, branch instructions usually make up less than 20 percent of
the program. For non-branches, the branch unit doesn't actually need to wait for the
ALU before calculating the result. Let's look at how we can use this fact to
pipeline the processor. Once the first instruction shown in yellow reaches the
pipeline register, we're ready to begin fetching the next instruction, shown in
blue. The yellow instruction can be in the execute stage whilst the blue instruction
is being fetched. Once the yellow instruction is finished, the blue instruction is
ready to enter the execute stage and a new green instruction enters the fetch stage.
What about the branches though? Let's imagine this next yellow instruction is a
branch. The fetch stage works normally until the branch unit, but the branch unit
cannot proceed. Consequently, the pipeline stalls. The fetch stage spends a cycle
waiting whilst the execute stage executes the branch. Finally, once the ALU is done,
the branch unit can proceed and the next instruction, in this case blue, can be
fetched. Overall, this means that the processor wasted one cycle stalling due to the
branch. Since only 20 percent of instructions are branches, this means that each
instruction would require on average 1.2 cycles. The same idea of stalling the
pipeline can be used to create even longer pipeline designs. This diagram shows a
typical five-stage processor pipeline. In the next video, we'll look at how we can
manage or prevent some of the stalls in a design like this. [music]

Module 2, Video 3
[music] Instructions within a program may be dependent on each other. That is, one
instruction may produce a value that a subsequent instruction consumes. Data values
may be communicated through registers or memory. The simple program shown has a
number of so-called true data dependencies. This means we must take care to execute
these instructions in order, and make sure results are correctly communicated.
Additionally, the outcomes of branch instructions may affect the path taken through
the program, and consequently, this affects whether an instruction is actually
executed. This sort of dependency is known as a control dependency. In the previous
video, we met a realistic processor pipeline with five stages. Circumstances that
prevent an instruction making progress in our pipeline are known as pipeline
hazards. Let's take a look at how dependencies cause hazards. This program has a
true data dependency. The first instruction writes to register one, which is then
read by the second instruction. If we send this down our pipeline, we see that the
second instruction must stall, waiting for register one to be written, before it can
read and proceed. This is a data hazard. Unfortunately, dependent instructions are
common and stalling in this way would significantly increase the average cycles per
instruction. Let's take a closer look at the hazard though. The ADD instruction is
in the execute stage, meaning its result is being computed. The SUB instruction
needs that result to proceed. Rather than waiting for the ADD to reach the writeback
stage, we could add an extra path into our pipeline to carry the output of one stage
to a later instruction, making the result available straight away. We call this a
forwarding path. In this case, the ALU result is forwarded to the SUB instruction to
be used as X1. This small piece of extra circuitry allows this data hazard to be
eliminated completely. Unfortunately, even if we add forwarding paths everywhere,
it's not possible to eliminate all data hazards. For example, this program has a
data hazard due to the load instruction. There are other types of hazard too. This
program contains a control hazard. We cannot be sure which instruction to fetch
until after the branch instruction executes. Consequently, this program has two
stall cycles. We will look in more detail at how control hazards can be mitigated in
the next module. Another class of hazards, called "structural hazards" occur when
two instructions require the same resources simultaneously. For example, if
instructions and data were stored in the same memory, and this could only be
accessed once per cycle, we would have to very frequently stall our pipeline to let
these stages access memory one-by-one. [music]

Module 2, Video 4
[music] In the previous videos, we explored how pipelining could improve performance
by reducing our clock period and by overlapping the execution of different
instructions. We also saw that it was sometimes necessary to stall our pipeline to
ensure that instructions were executed correctly. Ideally, our average cycles per
instruction, or CPI, will remain at 1.0. If we must stall, however, this will
increase. For example, if 20 percent of our instructions were loads and each of
these caused one stall cycle, our CPI would be 1.2. If a further 20 percent of
instructions were branches, and each of these caused two stall cycles, our CPI would
be 1.6. The longer we make our pipeline, the more stall cycles there will be, and
eventually the cost of stalls may outweigh the benefit of the faster clock period.
For example, let's imagine we added a stage to our five-stage pipeline from the
previous video. Now the number of stalls after a branch instruction increases to
three, hurting our CPI. On the other hand, our clock period would improve. So
whether or not this helps speed program execution would depend on the exact details.
It may eventually become more difficult to reduce our clock period by adding further
pipelining stages. This is because it becomes harder to perfectly balance the logic
between stages and because of the constant delays associated with clocking and our
pipeline registers. To mitigate these issues, we will need to invest in more
transistors and our design will require more area and power. The deeper our pipeline
gets, the greater the investment we need to make in terms of area and power for the
same incremental improvement. Commercial processes today have anywhere from two to
twenty pipeline stages. The faster, more expensive and power-hungry processors tend
to have longer pipelines than the smaller, cheaper processes in embedded devices. As
with many techniques in computer architecture, eventually it becomes more profitable
to invest our time and resources in an alternative way of improving performance. In
later modules, we'll explore how we can reduce the CPI, even in heavily pipelined
processes. [music]

Module 2, Lab
[music] In this exercise, we're going to be using a model of a processor pipeline to
explore the effect of the pipelining optimization. Computer architects use models
like this to make high level decisions early on about what parameters they will use
for a processor and using a model such as this saves the burden of actually building
the processor to find out its performance. The model is not simulating accurately
the performance of the processor but rather it's giving us an idea for what
performance we might expect. So what can we do with this model? Well, we can
configure the number of pipeline stages, which we can see affects the diagram. And
we can also turn on or off the forwarding optimization. As we change these numbers
notice that the design parameters change down here. So for example, the clock
frequency is improved by increasing the number of pipeline stages but the design
area will get bigger. And so this may be a consideration depending on the problem.
We can also choose which of two programs we're going to put down our pipeline. When
we press the step forward button the pipeline advances to the next clock cycle and
we can see the instructions have started to flow down our pipeline and interesting
events such as forwarding will be noted in the simulation. Sometimes the simulation
will detect that there will be a stall for example, in this case, we can see that
there is a data hazard because the instruction in the red memory stage writes to
register X13 which is read by the instruction in the yellow decode stage and
therefore a stall cycle is necessary in order to allow the result to be generated.
If we press the play button, the simulation will proceed automatically and we can
see various stall events happening as the simulation proceeds. But notice that the
the program we're simulating is nearly 1,000,000 cycles long so watching it play out
at this speed is going to take quite a while. So we can use the fast forward slider
to simulate much, much faster. Notice that the statistics down the bottom have
updated depending on the results of the simulation, and at this point we can see
that the program is finished and the simulation of the program, the simulated
program took 3.98 milliseconds to execute. We can also see that just below, the
results of past simulations are stored in little tables so we can easily refer back
to them when we're doing later experiments. So as an experiment, let's imagine what
would happen if we disabled the forwarding optimization but change nothing else and
we'll just run this program through. What we can see immediately is the design side
is slightly better which is what we would expect. It's got 1% better in fact in this
case because of the lack of the forwarding wires. But now that the program is
finished, we can see that the program execution time is a lot worse. 6.34
milliseconds is about 50% worse. So again, looking in our table, we can compare the
execution times in the area and we can see that in most cases the forwarding
optimization would be a big optimization here because at the cost of an increase in
the area of about 1%, we've had a improvement in execution time of about 50% which
is likely to be a good trade off, but not always. It would depend on the exact
scenario. Perhaps that 1% area is more important than the performance of this
program. In the exercise, you'll be invited to design a or suggest a design using
the number of pipeline stages and whether forwarding is enabled that will meet
certain criteria. You can play about and do as many simulations as you wish to
figure out what the best program might be. Once you've got it set up select the
processor that you're happy with at the top and then scroll down to the submit
button and press that. Good luck. [music]

Module 3, Intro
[music] Hi, I'm Nigel Stevens. I'm Lead Instruction Set Architect for the Arm A-
Profile architecture. I've been at Arm for about 14 years and I have responsibility
for the Arm V8-A instruction set including recent developments such as the Scalable
Vector Extension and Scalable Matrix Extension. What do we mean by the instruction
set architecture? The instructions in architecture, primarily, most people think of
I guess as the OP codes, the encodings of instructions that are executed by an Arm-
based processor. But it also includes other aspects as well such as the exception
model, system programming features, memory management and suchlike. The architecture
for Arm is rather distinct from what other companies may call an architecture. For
Arm, architecture is a legal contract, if you will, between hardware and software.
If software uses only those instructional codes and features of the ISA that are
described by the Arm architecture to perform its work, and the hardware that it's
running on implements all of those op codes and features exactly as defined by the
architecture, then any architecturally compliant software will run on any
architecturally compliant hardware that implements that Arm architecture. And that
doesn't mean just processors from Arm itself, but also processors that are designed
by our partners and which we have validated are conformant with our architecture.
How do you decide which instructions to include in an ISA? When we are looking at
requests from partners or from internal research to add a new instruction, we go
through quite a long process of trying to justify that instruction, or, quite
commonly, a set of instructions rather than a single instruction. We have to show
that it gives us some real benefit in performance, the performance of your code
running on that CPU. Or maybe not performance. Maybe it's security you're trying to
achieve. But it has to give you some really concrete benefit that is worth the cost
of adding all of the software, the validation software, the implementation costs for
all of the different implementations, compiler support, so on and so forth. It has
to meet the... It has to answer that cost benefit analysis. What is the difference
between an ISA and a microarchitecture? The difference between an instruction set
architecture, or ISA, and the microarchitecture is that the ISA is an abstract
concept. It defines a set of instruction encodings which software can use, and which
hardware has to recognize and implement. How that is implemented is a choice for the
microarchitecture. So the instruction set architecture is fixed, it's defined by
Arm. The microarchitecture is defined by whatever team of people is designing that
CPU. And there are many different approaches to implementing the Arm architecture,
from very small, efficient cores with in-order pipelines up to very high-
performance, state-of-the-art, out-of-order execution, and everywhere in between. So
the microarchitecture is implementation-specific, the architecture is generic, and
software written for the architecture should run on any microarchitecture. Why does
Arm produce processors with different instruction sets? Arm supports multiple
instruction sets. Some of that is to do with legacy: you can't abandon your legacy
software, your legacy ecosystem. So as the architecture has advanced and we've
introduced major new instruction sets, we still have to continue to support old
software. It takes years, maybe 10 years to move the software ecosystem to a major
new ISA. So for example, AArch64, which is the 64-bit architecture that we
introduced with Arm V8, also supported the AArch, what we called AArch32, the old
32-bit architecture that was implemented in the Arm V7 architecture, and prior to
that including the Arm and the Thumb instruction sets. And we needed to do that
because, while some software might start to migrate to the 64-bit architecture,
there's still a lot of software on the planet which is going to continue using the
32-bit architecture, and that has to survive. So that's part of the reason: it's
about legacy. You can't just obsolete the whole world when you introduce a new
architecture, a new instruction set architecture in particular. There are other
reasons as well, which is there are certain instruction sets that are different for
reasons of the ecosystem that they're working with. So if you were to compare, for
example, the A-profile architecture that's designed for application processors that
run rich operating systems with virtual memory supporting SMP, symmetric multi
processing operating systems running large applications, whatever it may be: web
browsers on your phone or something, or a web server in a server farm somewhere. You
have your R-profile architecture, which is designed for high-performance, real-time
embedded systems. The constraints there are somewhat different. The instruction set
is actually fairly similar to the A-profile, but some of the underpinnings of the
architecture, the system side of the architecture, are simplified in order to give
more consistent and predictable real-time response to things like interrupts or
memory translation and suchlike for real-time systems. And then at the other extreme
you have the M-profile architecture which is designed to be capable of being built
in a very simple, ultra-low power implementation with low code size and again,
similar to the R profile, very predictable real-time performance. So the answer is
there are different instruction sets for reasons of the market that they're trying
to address, and then there are different instruction sets because, well, we have
history. [music]

Module 3, Video 1
[music] In the previous module, we explored how pipelining can be used to improve
performance. We also saw how it is sometimes necessary to stall our pipeline to
ensure our program is executed correctly. In a simple pipeline, it will be necessary
to stall the pipeline whenever we encounter a branch instruction. This is because we
must wait until our branch is executed before we can be sure which instruction to
fetch next. As a recap, branches are instructions that change which instruction in
the program will be executed next. There are two types of branches: conditional
branches and unconditional branches. Unconditional branches always change which
instruction executes next, whereas conditional ones may or may not, depending on the
computations in the program. In real programs, between approximately one fifth and
one quarter of all instructions are branches, and the majority of these are
conditional. Executing a branch involves calculating the new address to load into
our program counter. This is the branch's "target address." However, conditional
branches have an extra task: we must first determine whether the branch is taken. If
the branch is not taken, we can effectively ignore the branch and fetch the next
instruction as normal. Recall the processor performance equation from an earlier
video. Since we have to wait for branches to complete before fetching the next
instruction, we generate stall cycles. These increase the average number of "cycles
per instruction," which reduces our microprocessor's performance. The longer our
pipeline gets, the longer it is before each branch is resolved, and the more costly
branches become. Can you think of a way to avoid some of this stalling? One idea is
to evaluate branches earlier in the pipeline, for example in the Decode stage
instead of in the Execute stage. This can indeed help to reduce the number of
stalls, but we may still need to stall if the branch depends on other instructions
that haven't been executed yet. Another idea is to continue fetching instructions in
program order, effectively assuming that each branch is not taken. The number of
stalls in the pipeline for a not-taken branch is zero in this design. On the other
hand, if the branch is in fact taken, the subsequent instructions that we fetched
will be incorrect. So, we must remove all instructions that have been fetched on
this incorrect path from our pipeline. This is called "flushing" the pipeline.
Unfortunately, in real programs, branches are taken much more than not taken. Could
we then simply assume instead that all branches will be taken? Sadly not, no,
because then we would also need to know the specific "target address" immediately,
in order to know which instruction to fetch next. It may at first seem impossible to
know this before the instruction is decoded. However, computer architects have found
a way to do exactly this. The next video will look at "dynamic branch prediction:"
the idea of predicting the behavior of the branch instruction before it has even
been fetched. [music]

Module 3, Video 2
[music] In this video, we'll explore how to predict the behavior of a branch
instruction. This can sometimes eliminate the cost of branches altogether.
Fundamentally, a branch involves changing the value of the program counter, which is
the address of the next instruction in the program. If we could predict what this
change will be, quickly and accurately, we would have no need to stall. Precisely,
we need to predict that we are fetching a branch, predict it as taken or not taken,
and predict what its target address is. How could we ever make such predictions?
What would we base them on? Well, since instructions are often executed multiple
times, we can accurately make these predictions based on just the instruction
address. If we've previously seen that a particular address contains a branch, and
we see that address again, we can predict whether that branch will be taken, and its
target address, based on its behavior last time. Amazingly, for real programs,
simply predicting repeating behavior is typically around 90 percent accurate. This
means we could eliminate stalls caused by branch instructions 90 percent of the
time. Let's apply these insights to try to build a branch predictor. We will add two
extra blocks to the Fetch stage of the pipeline we met in Module 2. The first will
remember information about recently executed branches. This will include the program
counter values of branches and their target addresses. This memory is called the
"Branch Target Buffer" or BTB. The second block will make predictions about whether
an address containing a branch is taken or not. We simply call this the "branch
predictor." In the next video, we will look at these in more detail. Combining these
two gives us all the information we need to predict the next value of the program
counter based solely on the current value of the program counter. We don't even need
to decode the instruction to make this prediction. Here we can see how a running
branch predictor behaves for a sample program. Each program counter is checked in
the BTB to see if it's predicted to be a branch and to identify its predicted
target. We also simultaneously check the branch predictor to see if the branch is
predicted to be taken. Based on these predictions, the branch unit computes the
predicted next program counter. Many cycles after the prediction, feedback will be
given by the rest of the pipeline as to whether or not the prediction was correct.
Whenever the prediction is wrong, we have to flush the pipeline and update the BTB
and branch predictor. The pipeline will then resume fetching from the correct
program counter as computed by the pipeline. The majority of instructions are not
branches, so most of the time the branch unit just adds 1 to the program counter.
The BTB contains the instruction address and target address of some recently
executed branches. The exact number of entries in the BTB varies considerably. BTBs
in large modern processors contain many thousands of branches. In operation, the BTB
checks the supply program counter against its memory to see whether it has a match,
and if so, it returns the target address. Otherwise, it predicts that the
instruction is not a branch. After each branch executes, the BTB is updated with the
true target address. BTB cannot be arbitrarily large, so it may have to forget an
existing branch to remember a new one. A simple BTB design like this is typically
around 90 percent accurate at predicting target addresses in real programs. [music]

Module 3, Video 3
[music] In the previous video, we met the two major components of dynamic branch
prediction, the BTB and the branch predictor. In this video, we'll take a deeper
look at the branch predictor. It predicts whether or not a branch will be taken,
based on the program counter. A simple branch predictor would try to remember what
the branch did last time and predict that the same behavior will repeat. Let's see
how such a predictor might be organized. Remembering a branch prediction for every
possible instruction address would take up far too much memory, so we reuse the same
memory for different branches via a process called "hashing." We hash the address of
the branch to a smaller number. This does unfortunately lead to a problem called
"aliasing," where two different branches can hash to the same value, but this is
rare in practice. Let's see what happens now, when we execute a simple loop. We see
a misprediction when it encounters our branches for the first time, and another when
we exit the loop. The first case will be dependent on the value in our predictor's
memory, and it may be that we are able to predict the branch correctly the first
time we see it. The second case is hard to avoid, although some more sophisticated
branch predictors will learn how many iterations a loop will make. A common
improvement to this simple scheme is to avoid instantly flipping our prediction just
because the branch does something unexpected once. This can be achieved with a
saturating counter, which instead remembers how many times the branch has been taken
recently, versus not taken. The counter increments when the branch is taken and
decrements when not taken. It predicts "taken" if the majority of recent executions
of this branch were taken. When building higher-performance processors, we often
have to discard many instructions every time we mispredict a branch, so accurate
branch prediction is very important. Therefore, branch prediction is still an active
area of research. One of the key ideas used is correlation between branches. In real
programs, a common pattern is for example a pair of branches that always have
opposite behavior. A branch predictor can take advantage of this by remembering a
history of whether or not recent branches were taken or not taken and incorporating
this in the hash. Another idea is to combine multiple different types of predictor
in a "tournament predictor." We use a "meta predictor" to predict which of two
branch predictors will do a better job. As you can see, branch prediction can get
quite complicated. [music]

Module 3, Video 4
[music] In this module, we've met the concept of branch prediction. Even using
relatively small simple circuits, we can accurately predict real branch behavior
more than 95 percent of the time. Could we do better, and do we really need to? In
small, simple processors, these prediction accuracies are fine, because each
misprediction causes only a few stall cycles. Increasing the accuracy is not that
impactful. However, in complex processors with very long pipelines, the difference
between 98 percent and 99 percent prediction accuracy can be significant for
performance as a misprediction could incur dozens of stall cycles. Accurate
prediction really does matter. One of the problems we face in making accurate
predictors is that they need to be small enough and fast enough to fit in a
microprocessor. We can imagine all sorts of ways to do accurate branch prediction,
but if their circuit would be slower than simply executing the branch, they would
not be useful. Modern high-performance processors will often have multiple branch
predictors, for example a small fast one and a slower complex one. The slower one
can override the prediction of the fast one if it thinks it got it wrong, which does
incur some stall cycles, but fewer than a total misprediction. Another problem we
face is that some branches are just very hard to predict. No matter what technique
is used, there are some branches in real programs that are effectively "random". For
example, when compressing or decompressing data, the combination of the underlying
algorithm and the input data may provide no clear patterns to a prediction. No
matter how hard we try, some branches will never be predicted correctly 100 percent
of the time. A final problem is that since the predictors work based on observing
the program, there will always be a period of time when the predictors train on a
program to learn its behavior. [music]

Module 3, Lab
[music] In this exercise, we're going to be using a branch predictor simulator to
explore dynamic branch prediction. This simulator will accurately simulate the
details of a branch predictor but it uses a trace of a real program executing on a
real machine to avoid the need to simulate the rest of the processor. Computer
architects use techniques such as this to quickly explore one area of process design
understanding that perhaps the accuracy may not be completely correct given that
we're not simulating the full process of design. The interface allows us to
configure details of our branch predictor for example the maximum of the saturating
counters used in the branch predictors table or the hash function. And we can see
the impact of changes on the delay of the branch predictor and also the design size
which are two key metrics when designing a branch predictor. Once we've happily
configured the design we want we can press run to simulate a program and the results
of that program's execution will be displayed in the rest of the stats. So for
example, we can see here that the predictor predicted 95.24% of the branches
correctly. Which is fairly good and this resulted in overall execution time of the
program of 5.335 milliseconds. Just below we can see a table that records previous
simulation runs as well so we can do multiple experiments and see which produces the
best results. For example, if we were curious about the effects of using a
saturating counter with a maximum of three rather than one we could change that and
notice that the design size has substantially increased and when we press run, we
noticed that the predictor accuracy, however, has also increased. It's gone up to
96.31% and consequently the execution time the program has fallen slightly and so we
can compare these two designs and see whether or not this represents a good trade-
off for our processor. Perhaps the area cost is justified or perhaps it's too much.
It would depend on the exact processor we're trying to design In the problems you'll
be invited in to come up with designs that are suitable for particular constraints.
For example, particular constraints on the runtime of the program or the design
size. Once you've configured the branch predictor that you think meets the
objectives you can scroll down to the submit button and click that and you'll be
told whether the answer is great or not. Good luck. [music]

Module 4, Video 1
[music] So far, we've looked at the microprocessor's "datapath"— meaning its
execution units, registers, and control circuitry. We have given less attention to
its memory. We usually implement memory using a different type of chip than the
microprocessor, using a technology called DRAM. It is very dense, allowing us to
store lots of data in a small area. However, one issue with DRAM is its speed. Since
the 1980s, processor performance has increased very rapidly at roughly 55 percent
per year, so CPUs of today are many orders of magnitude faster than those of 40
years ago. In contrast, memory performance has grown much more modestly. Whilst
memories are also much faster than they were in previous decades, their performance
has not kept pace with processors. This leads to a processor-memory performance gap,
with it becoming more costly to access memory as time goes on. One of the issues
that makes these memories slow is their size. We can make a memory as fast as our
microprocessor, if it is very small. However, this is the opposite of what
programmers want. Programmers can use extra memory to solve more complex problems,
or to solve existing problems faster. This complicates microprocessor design because
although we want a large memory to hold all our data, large memories are slow and
this slows down our processor. How do we overcome this? What if we could get the
speed benefit of a small memory alongside the size benefit of a large memory? One
solution is to have a large, slow memory and a small, fast memory in our system. For
this to be useful, we need the small memory to hold the data that we use most often,
so that we often get the speed benefits of accessing it, and only rarely have to
access the slow memory. Whilst there are different arrangements of these two
memories, the configuration that is most often used is an "on-chip cache memory."
The small memory sits inside the processor between the pipeline and the large
memory. We keep all data in the large main memory, and put copies of often-used data
in the small memory, which we call a cache. But we are not limited to only one
cache! Our pipeline reads memory in two places: when fetching the instructions; and
when accessing the data. It makes sense to have two caches here, each optimized for
different purposes. The instruction cache is optimized for fast reading of
instructions at Fetch. The data cache is optimized for reading and writing data from
the memory stage. We will often put a larger "level 2" cache between these two
caches and the main memory. The L2 cache is a "medium-sized" memory: faster and
smaller than main memory, but slower and larger than the two L1 caches. Using a
hierarchy of caches reduces the bandwidth requirements to main memory and the energy
cost of moving data around. [music]

Module 4, Video 2
[music] We previously looked at the need for a cache, which can be used to store
often-used data for fast access by the processor. But which data is used often
enough that it is worth including in the cache? Programs are mostly made of loops.
Here's a simple one that sums values in memory. It displays the two characteristics
that we can exploit to decide what data to put into a cache. Let's look briefly at
how it works. Each time round the loop, there are two loads and one store. The first
load is to load the data we're summing. The other load, and the store, are to update
the running sum. Notice two things here. First, we access the running sum over and
over again; each time round the loop. Second, when we access part of the data in one
loop iteration, we've already accessed its predecessor in the previous iteration,
and will access its successor in the next. Caches exploit two types of "locality" in
programs in order to be effective. The first is temporal locality: if a piece of
data is accessed, it is likely that it will be accessed again in the near future.
The running sum has temporal locality. The second type of locality is spatial
locality: If a piece of data is accessed, then its close neighbors are quite likely
to be accessed in the near future. By close neighbors, we mean data whose memory
addresses are not far from each other. The data accesses have spatial locality. It
turns out that most programs exhibit lots of temporal and spatial locality. We can
exploit this to determine what to put in the cache. Exploiting temporal locality is
fairly easy: we simply see which values have been accessed and place them in the
cache. Exploiting spatial locality is also quite simple: when a piece of data is
accessed, place its neighbors in the cache. We'll see in the next module how this is
actually achieved. We use locality to guess what values will be accessed next and
store them in the cache. If we are correct, we get the benefit of fast memory, but
if we are wrong, we must perform a slow access to main memory— just as we would if
the cache were not present. In real programs, we see hit rates of 90 percent or more
in microprocessor caches, resulting in vast performance improvements. However, it's
worth noting that some programs have less locality, and for those programs, caches
offer little performance benefit. [music]

Module 4, Video 3
[music] We've looked at the reasons why we build caches, but how do they actually
work? To the outside world, a cache simply takes an address as input, and either
provides the data that is stored at that location at output, or returns a signal to
say that it doesn't have it. If the data is found, this is called a "cache hit". If
the data is not in the cache, this is called a "cache miss", and it means we must
look for the data in main memory instead. After each miss, we update the contents of
the cache. The fundamental building block of a cache is called the "cache line".
It's a number of data bytes from consecutive addresses in main memory. Cache lines
are typically 32 or 64 bytes long, and a cache typically has an array of hundreds to
many thousands of lines. The line captures spatial locality, because it is larger
than the data read by a single load instruction. When a cache miss occurs, the whole
line containing that data is copied to the cache from main memory, meaning we have
nearby values for future accesses. When a request comes in, we use some bits from
that address to index the line array. Just like in the last module on branch
prediction, this leads to the problem of aliasing again, since selecting only some
of the bits to index into the array is like a hash. This means that many addresses
map to the same line in the cache, but we can only store one of their lines of data.
We need to note down which addresses' line is currently stored in the data array, in
what we call the "tag array". There is one tag for each cache line. When we access
the cache, we access the tag array with the same index to see if the data we need is
present. This design is called a "direct-mapped cache". Direct-mapped caches work
fairly well, but for some programs, we can be unlucky with the program accessing to
aliasing lines frequently. We can do something about this by duplicating both the
arrays, so that each line of data can now be stored in one of two places in the
cache. When we access the cache, we look at both arrays and only get a miss if
neither of the tags match. This is called a "2-way set-associative cache", because
each line has a set of two places or "ways" it could reside. The "associativity" of
this cache is therefore two. Set-associative caches introduce a further
complication: when we want to add data, where do we put it? In a 2-way set-
associative cache, there are two choices. How we decide which cache line to evict is
called the "replacement policy". There are many different types of replacement
policy. A simple one is just to make a pseudo-random choice from all the possible
cache lines. Another option is to keep track of when each cache line was last
accessed and to evict the one last used furthest in the past. This is called a
"least recently used policy", and takes advantage of temporal locality. It does,
however, means storing extra information in the tags to track usage. Many other
ideas are possible too. [music]

Module 4, Video 4
[music] Now that we've seen how caches work, let's see how they affect the
performance of a processor. Recall the processor performance equation, where the
processing time is proportional to the average cycles per instruction. Without a
data cache, if 20 percent of instructions are loads, and main memory takes 20 cycles
to access, our CPI figure must be at least 5. However, if we provide a cache that
holds the required data 80 percent of the time... ...and only takes 2 cycles to
access, our CPI reduces to 2.2, which is a significant improvement! We can isolate
the memory terms in this equation to get the average memory access time —abbreviated
to AMAT— which allows us to compare different cache configurations more easily.
Changing the cache configuration will impact the AMAT. There are many different
cache parameters we can change, such as the size, replacement policy, associativity,
whether we put data in the cache for stores or just for loads, and so on. For
example, reducing the size of the cache will improve the access time for a hit, but
will also increase the miss rate. Let's say that we can halve the access time to 1
with a corresponding halving of the hit rate. This alters the AMAT to 13, which in
this case is worse for performance overall. It's also useful to look at why an
address might miss in the cache. Broadly speaking, we can divide cache misses into
three different categories. Compulsory misses occur when we attempt to access an
address that we have never seen before and so never had the opportunity to cache it.
Capacity misses occur when there is more data being accessed than the cache could
hold, even if we had complete freedom in where to put each cache block. Conflict
misses occur in caches where there are more addresses hashing to the same index than
arrays to hold the data. We can alter our cache configurations to lower these
misses, but as always, there are trade-offs involved. Compulsory misses can be
reduced by increasing the cache block size, to take advantage of spatial locality.
But for a fixed cache size, this reduces the number of different addresses or cache
lines that can be stored. A technique called "pre-fetching" can also be used to
predict the addresses that will soon be accessed, and bring their data into the
cache early. But this increases energy consumption, and may make the cache perform
worse if the predictions are not highly accurate. Capacity misses can be reduced
through increasing the size of the cache. Although, as we saw before, this impacts
the number of cycles taken to determine a hit. Conflict misses can be reduced
through increasing the number of cache blocks in each set, with an increase in
energy consumption as a side effect of this. [music]

Module 4, Lab
[music] In this exercise, we're going to be using a cache memory simulator to
explore the effects of cache memories on processor performance. The simulator
accurately simulates the behavior of the cache memory system, but it's using a model
of the rest of the processor to quickly allow us to simulate the effects of the
cache on the processor without needing to simulate the full processor. We can
configure a number of parameters about our cache, for example, the number of levels
in our cache, whether or not the cache separates instructions and data, or is
unified, keeping them both in the same cache. We can also configure the size, the
line size, and the associativity, and changing these numbers will affect design
parameters, such as the access times in the case of an L1 level one cache hit or a
cache miss, and also the design size. Once we're happy we've found design we'd like
to investigate, we can press "run", at which point the simulator will run through
the program. We can see the hit rates for instructions in the level one cache, and
also data in the level one cache are displayed. And we can also see the average
memory access time, the results from this. And then below everything, we can see a
table of past simulations so that we can quickly refer back to our previous
experiments when we do new ones. So, for example, let's say we were curious about
the effects of increasing the size of the cache. If we change the parameter and then
press "run", we can immediately see that the design size has substantially
increased, which sort of makes sense because we've doubled the contents of the
cache, the size of the contents of the cache, and therefore we'd expect roughly
double the area. And we can also see that the hit rates have improved. So there's
about a 1% improvement to the L1 instruction cache hits and a 1% improvement to the
L1 data cache hits, which has reduced the overall average memory access time. And so
we can compare these two designs to see which of them we think is better. It's a
trade-off of course, though. The larger design has got better performance, but it is
larger, and so depending on the context, we may need to pick the smaller design or
the bigger design depending on our performance goals. In the exercises, you'll be
invited to come up with a series of designs for caches that meet certain performance
goals. For example, you'll have a constraint on the area of the design or the
execution time of the program, and you need to optimize the cache to meet those
goals. Best of luck. [music]

Module 5, Video 1
[music] In this module, we'll look at how to further improve performance by
exploiting "instruction-level parallelism." In Module 2, we explored how pipelining
can improve the performance of our processor. This reduced our clock period, and
allowed execution of instructions to be overlapped, improving throughput. One way to
boost performance further would be to create a much deeper pipeline. At some point,
this would mean even the ALU in our Execute stage will be pipelined. Consider the
simple program shown in the slide. Some instructions are dependent on a result from
the previous instruction. Remember in our 5-stage pipeline that these dependent
instructions could be executed in consecutive clock cycles with the aid of data
forwarding. If execution takes place over two pipeline stages within the pipeline,
we need to stall if adjacent instruction share a dependency. This allows time for
the result to be computed. The programmer may be able to rewrite their program to
get the same result with fewer stalls, by placing an independent instruction between
our pair of dependent instructions. In this case, we can move the third and fifth
instructions earlier to optimize performance. The performance of programs that run
on our "super-pipelined" processor would, to some degree, be determined by the
availability of independent instructions that could be executed in parallel in the
pipeline. This is "instruction-level parallelism"—or ILP. Very deep pipelines are
problematic as they would require: a very high-frequency clock to be distributed
across the chip very precisely; careful balancing of logic between many, very short,
pipeline stages; the pipelining of logic that is difficult to divide further into
stages; and the division of logic at points requiring many pipelining registers to
be inserted. A different approach to exploit ILP is to make our pipeline wider
rather than deeper. In this design, the processor will fetch, decode, and
potentially execute multiple instructions each cycle. Such a design avoids the
problems of a super-pipelined processor, although as we'll see in the next video, it
does introduce some new complications. [music]

Module 5, Video 2
[music] In this video, we are going to explore "superscalar" processors, which can
process multiple instructions in each pipeline stage. In our simple 5-stage
pipeline, there is at most one instruction per pipeline stage. At best, we can
complete one instruction per cycle. We call such a design a "scalar" processor. In a
2-way superscalar version of this processor, we would extend this design so it is
able to fetch, decode, execute and writeback up to two instructions at a time. In
general, superscalar processors may vary the number of instructions that can be
processed together in each stage. Let's step through the design. Our instruction
cache will need to supply two instructions per cycle. Typical superscalar processors
only ever fetch adjacent instructions on a given cycle. This can lower performance
if, for example, the first instruction fetched is a taken branch, because then the
second would not be required. Note that now every cycle lost due to control hazards
will cost us two instructions rather than one, so accurate branch prediction matters
even more in superscalar designs. The Decode stage must now decode and read the
registers for two instructions simultaneously. Fortunately, we are able to extend
the register file design to read many register values at the same time. The Decode
stage also needs to check whether the two instructions are independent. If so, and
if the functional units they both need are available, it can "issue" them for
execution in parallel on the next clock cycle. Otherwise, in this simple design, it
will only issue the first, and keep the second back. A simple design such as this—
where two instructions are fetched, decoded and issued— is called a "2-way" or
"dual-issue" processor. In other designs, the width may vary at different stages of
the pipeline. To support the execution of multiple instructions at the same time,
the Execute stage is expanded and contains two execution pipelines. It's common for
these to have slightly different capabilities to save area. For example, the top
pipeline can execute both ALU and memory instructions, while the second pipeline
only executes ALU instructions. To ensure that dependent instructions can execute on
consecutive clock cycles, we must add data forwarding paths. These data forwarding
paths must allow results stored in either execution pipeline to be forwarded to the
input of either ALU. During writeback, we need to store both results to the register
file. This means the register file must be redesigned to allow two writes per clock
cycle. Overall, these changes typically require 25 percent more logic circuitry in
our processor, compared with a scalar processor. But we'd expect an improvement in
execution time of between 25 and 30 percent for real world programs. [music]

Module 5, Video 3
[music] We've seen that instruction-level parallelism can be used on superscalar
processors, to run them faster than would be possible on a scalar processor. But how
much of a speedup is this in practice? Ultimately, this depends on how much
instruction-level parallelism is possible in a typical program. How might we measure
this? We can do this initially without considering any constraints that will be
imposed by the processor it will run on. Let's consider the instructions executed by
the program. Let's assume that we can predict all the branches in the program
perfectly. Then we can ignore branch instructions, as they don't need to flow down
our pipeline. Now let's imagine we can execute any instruction as soon as the data
it needs is ready. That is, we are only restricted by the presence of true data
dependencies. Note that some dependencies are carried through writes and reads to
memory. Rather than considering program order, we can now just look at the order the
dependencies impose on instructions. This is referred to as "data-flow analysis."
Assuming each instruction takes exactly one cycle to execute, the fastest possible
execution time of the whole program in cycles is given by the longest path in the
data-flow graph. The instruction-level parallelism of this program is the number of
instructions divided by this duration, as this gives the average number of
instructions we would need to be able to execute each cycle to achieve this
duration. In real programs, this can be anywhere from around five, to hundreds or
even thousands. An active area of research and innovation for computer architects is
to imagine processor designs that can expose and exploit as much of this parallelism
as possible. One insight architects have had is that superscaler processors need to
have a fast supply of instructions to be able to analyze dependencies effectively.
This often means that the front end of our processor pipeline is much wider than the
rest of the pipeline, so that it can "run ahead" and see what behavior the program
will have next. Fast and accurate branch prediction is vital, as we often have to
predict multiple branches ahead accurately, to achieve good performance. Another key
insight is that we don't have to wait to execute the instructions in program order.
If all the dependencies of an instruction are satisfied, the instruction can proceed
down the pipeline even if previous instructions are yet to execute. This can reduce
program execution time by taking advantage of more instruction-level parallelism. In
practice though, this creates extra complications, as we will see in the next
module. [music]

Module 5, Lab
[music] In this exercise, we will be using a simulator to explore superscalar
microprocessor design. The simulator has a number of parameters that we can
configure, such as the number of pipeline stages, the width of the Fetch stage, the
width of the issue, and the number of ALUs in the design. We can see a diagram of
the processor that we've created and we can also see a number of parameters about
that design, for example the clock frequency and the overall area of the design.
When we press step, the simulator will advance one clock cycle and so for example
here we can see that the fetch stage has fetched the first 4 instructions. However,
immediately we see one of the problems with designs such as this, which is the three
of the four instructions that have been fetched are in fact useless because the
first instruction was an unconditional taken branch and therefore the remaining
three instructions will not be executed by the program and so these will immediately
be discarded on the next cycle. Pressing the run button allows us to simulate and we
can use the fast-forward option to simulate much quicker in order to get to the end
of the long program execution. In this case we can see that our design achieved an
average cycles per instruction less than one, which is to say that we on average
executed more than one instruction per cycle, which means that our superscaler
design has fundamentally worked, and we can see that, for example, the overall
execution time of the program is 1.6 milliseconds. In a table below, we can also see
a record of our previous simulation runs. So let's say for example, we were curious
about the effect of increasing the issue width by one. We can make that change and
then press run again in order to run our new design, and when it finishes we can
scroll down to take a look and we can see that the program execution time has indeed
improved down to 1.51 milliseconds at a cost of only 1% area. So it looks like this
was a very good improvement to our design and it's almost surely going to be a
beneficial trade-off in practice. In the exercise you will be invited to configure a
number of different superscalar processor designs with various targets in terms of
clock frequency, design area and execution time. Once you're happy that you've
configured the process that you think completes the exercise, you can scroll all the
way down to the bottom, where you'll see the submit button that you can press to
have your answer checked. Good luck. [music]

Module 6, Video 1
Introduction:

So hi, my name is Peter Greenhalgh. I'm Senior Vice President of Technology and an
Arm fellow. I'm responsible for the Central Technology Group at Arm. We're about 250
people. We work on everything from machine learning to CPU, GPU, system IP, and the
solutions that we create as well. And we basically path-find future technology at
the product level that goes into all of our products and the IP that we produce. Arm
is known for the power efficiency of its microprocessors. How have you managed to
keep a focus on power when building processors with very complex and power-hungry
features? We've got some really great design teams. In fact, we churn out I think
more CPUs than pretty much anyone else does on the planet. I think we're producing
something like four or five CPUs per year. So we've got a lot of experience in
designing for power efficiency and performance, and in fact we can leverage the
understanding that we have all the way down to microcontrollers through to the
smaller A-class processors, all the way up to the high performance. There's a lot of
sharing between the teams in terms of strong knowledge and capability, and insight
into how to design for both performance and power efficiency. More specifically, I
mean, ultimately, you have a performance goal that you need to achieve, and then as
part of that you have to figure out how to get the best possible power out of the
design when you're achieving that performance goal. And to do that, there's kind of
some different ways of looking at it. There's the really detailed orientated work
that you need to do around things, like clock gating, data gating, all the things to
try and stop unnecessary power use deep within the microarchitecture when the
instructions are flowing through the pipeline or data's moving through the pipeline.
And then there's essentially the structure of the design that you've created. And
that then dictates fundamentally what the power of the design is going to be. You
can't fix a design that's got bad structure with improved clock gating, data gating,
and just good low-level design. You have to marry the two together. And that high-
level work that you do is around making sure that the pipeline is well balanced,
that you aren't opening up the pipeline, going too wide too soon; you're extracting
data, you're extracting information as late as you possibly can and just when you
need it, and not just pipelining it down through the design for the sake of it; and
then, fundamentally, good microarchitecture around branch prediction, which stops
you putting things down through the pipeline that you're just ultimately going to
flush; good pre-fetching on the data side so that you make sure you get the data in
the design when you need it, and you're not sitting around waiting for it. So you
have to marry that altogether, and we've got a lot of great techniques in order to
achieve that, which fundamentally, I say, you need to achieve the performance
target, and then everything else comes together to achieve that performance target
in the best possible energy efficiency. How did Moore's Law affect computer
architectures of the past, and what will its influence be on future designs? Gordon
Moore's influence on the industry has been massive, and the tenets behind the law
still continue today, albeit in a slightly different form. I mean, I started
designing at .18 Micron, essentially 180 nanometers, and here we are today working
on 3 nanometers. So it's a vast difference now compared to when I started 22 years
ago. And there's no way we could have got to where we are today without the process
scaling from all of the foundries out there and all the companies that provide the
foundry technology. So it's a little bit like magic, all of the work that they do. I
can't say I understand it in detail, but it's incredible technology, and that allows
us... If it hadn't happened, we'd still be stuck in all the designs which were
fairly simple. There's no way that we'd have got to the sort of designs that we have
today of massively out of order, deep, deep amount of instruction and depth, very,
very wide designs. All of that has been made possible by the steady improvement, a
predictable improvement of the foundries. And that's kind of one of the key points
which really was captured by Moore's law or is captured by Moore's law of that kind
of predictable knowledge that you will get an improvement in the process. Is it 10%,
is it 15? Is it 20% on, say, power, for example? It kind of doesn't matter in a way
because you can work with what you eventually get. You can do things like voltage
scaling to be able to make use of the power that's available to you. Is it 5%? Is it
10% on frequency? Again, it kind of doesn't matter in a way. But what matters is
when we start designing a processor today and we finish it in 18 months time, and
then two years after that it arrives in the product in the shops that consumers can
buy. We know that over that period, the process improvements have happened, which
allows us to liberate essentially more performance, more energy efficiency from the
design. And we don't mind too much if it takes another three months or six months to
get to the process. We don't mind too much if the performance or power is not
exactly where it was predicted at the start. But, ultimately, we know we'll get an
improvement, and we know there'll be an improvement in two years, and three years,
and four years, and Moore's Law may have slowed, but it's certainly not stopped.

[music] As we saw in the last module, instruction level parallelism can be used to
improve program execution time in our microprocessor designs. To enable this, the
compiler creates an optimized instruction schedule when the program is converted
into machine code. Unfortunately, the compiler cannot know precisely what will
happen at run-time, so this design is constrained by the order of instructions in
the program. The compiler won't know what the program's input data will be, whether
branches will be mispredicted, or whether memory accesses hit or miss in our data
cache. In contrast, a superscalar processor with "out-of-order" execution can
produce an instruction schedule at run-time, only constrained by true data
dependencies and its hardware limits. This schedule is produced on demand and so can
even change each time the code runs. To do this, we introduce an "issue window" or
"issue queue" after the Decode stage. This holds instructions until they can be
executed, not necessarily in the order they arrived in. Within this window,
instructions can be issued whenever their dependencies are available, and when a
functional unit is available to process it. To be able to detect when an instruction
is ready to be issued, we must know whether the instruction's dependencies are ready
when it enters the issue window. We must then update this status as new results are
produced. To implement this, the names of result registers of executed instructions
are broadcast to the issue window. The instructions waiting there compare the
register names to the registers they require. However, this scheme has a problem: A
register will be written multiple times in the program, and since the instructions
are executed out-of-order, the register name alone is not sufficient to record
dependencies. It also means that instructions would have to wait until all previous
reads of a register had finished before executing. These are called "false
dependencies." These problems can be resolved by "renaming" register names at run-
time so that each "in-flight" instruction writes to a unique destination register.
We use a "physical register file" that is large enough to ensure we don't run out.
We keep a "register mapping table" to store the mapping between architectural,
compiler-assigned registers, and physical registers. Register reads to the same
architectural register are renamed consistently, so that dependencies can be tracked
correctly with physical register names. Physical registers are reused only when they
are no longer used by any instruction currently in-flight or any entry in the
register mapping table. The other big issue with out-of-order execution is memory
dependencies. Load and store instructions can have memory dependencies because they
access the same memory location. To detect this, we need to compare the computed
memory addresses that the instructions access. We thus split memory operations into
two steps: address calculator and memory access. We issue their address calculation
step as soon as the dependencies are available. Then, the memory access step is
placed in a special load-store queue to be sent to our data cache as soon as
possible. We carefully ensure that operations that access the same address are kept
properly ordered, but independent accesses can be reordered if beneficial. No access
can occur until the addresses of all previous accesses are known. Since memory
writes are irreversible, store instructions must also wait until we are certain that
they will execute. [music]

Module 6, Video 2
[music] In the previous video, we outlined the concepts of out-of-order execution,
and register renaming. The issue window will be filled with instructions fetched
along the path that our branch predictor believes the program will take. While we
hope our branch predictor will be correct in most cases, it will sometimes be wrong.
How do we handle such cases? A simple approach is to start by recording the original
program order of the instructions, and then to monitor their progress. We call the
structure that stores the instructions the "reorder buffer." As each instruction
executes and produces a result, we can mark it as done. When the oldest instruction
has completed, we can remove it from the end of the reorder buffer, and the
instruction is said to have "committed." This stream of committed instructions
represents how our program would be executed on a simple in-order pipeline or by an
unpipelined processor. It usefully also provides a point at which we can process
exceptions. For example, if the program divides by zero or attempts to access memory
that does not exist. We also check branch instructions as they complete in order. If
they have been mispredicted, we flush the reorder buffer, our instruction window and
any currently executing instructions and start fetching down the correct path. To
preserve correctness, we must also restore our registers and the register map table
to the values they had when we mispredicted the branch. This can be done with the
aid of a second register map table, updated only when instructions commit in program
order. This can simply be copied to the map table used by our renaming hardware to
"rewind time" for the processor. All the register values we need will be present, as
we don't recycle registers before we know they will not be needed again. In reality,
handling branches in this way is too slow. Processors instead take many copies of
the register map tables and can handle branches as soon as they are resolved, and we
discover they have been mispredicted. They can also selectively neutralize the in-
flight instructions in the datapath that are on the wrong path, rather than flushing
all of these instructions away. [music]

Module 6, Video 3
[music] We can now bring everything together and look at what a typical pipeline for
an out-of-order superscalar processor might look like. The Fetch stage is aided by
an accurate branch predictor as we met in Module 3. It will fetch a group of
instructions on every clock cycle. This group of instructions will be requested from
the instruction cache, and will be from consecutive memory locations. Branches may
reduce the number of useful instructions that can, in practice, be fetched in on
each cycle. The Decode stage decodes multiple instructions in parallel. At this
point, modern high-performance processors may also split complex instructions into
simpler operations or "macro-ops." In some cases, there may also be opportunities to
combine simple instructions into a single operation. The next step on an
instruction's journey is renaming to receive a unique destination register. As we
saw in the last video, this increases opportunities for out-of-order execution.
Remember, there are several times more physical registers in our processor than
those available to the compiler. Instructions are placed in the reorder buffer, and
are also "dispatched" to the Issue stage. They will wait in the window as necessary,
and are ready to be issued once all their operands are available. In the most
complex of today's superscalar processors, there may be hundreds of instructions
buffered in the issue window at the same time. Instructions finally commit in
program order. At this point, any physical registers that are no longer needed can
be added back to the pool of free registers. These are then assigned during the
register renaming step. Once an instruction is issued, it reads its operands from
the physical register file. The Execute stage consists of many functional units
operating in parallel. These may each support different operations and take
different numbers of cycles to execute. A network of forwarding paths is also
provided to ensure we can execute any dependent instruction on the next clock cycle
after the generation of the result. This requires being able to quickly communicate—
or "forward"— a result from the output of any functional unit, to the input of any
other. Some instructions will need access to memory. After computing their
addresses, they are placed in the processor's load-store queues. "Stores" are sent
to memory in program order, but "loads" can often be sent out of order, and ahead of
other older stores or loads that are not yet ready to be issued to memory. The
memory system reduces the average memory access time by providing numerous levels of
cache memory. After generating results, we write them back to the register file.
This overview is representative of the fastest modern microprocessors found today in
laptops, smartphones and servers. Whilst much extra innovation goes into real
designs, they generally follow the ideas discussed in the course. [music]

Module 6, Video 4
[music] One question computer architects always ask themselves is: "how much can we
scale up our design?" Let's take a look at some further potential optimizations to
our out-of-order superscalar processor. We could try to make it wider. For example,
by doubling the number of parallel instructions, we can fetch, decode and execute
more instructions per cycle. Would this double our performance? Sadly, no, things
are not that simple! In practice, some components quickly become very complex, and
performance gains may be hard to extract. For example, today's largest machines
fetch at most ten instructions per cycle from their instruction caches. Fetching
more instructions than this offers minimal performance gain, despite a large
hardware cost. If we increase the number of registers, the size of our issue window,
the size of our load-store queues, or perhaps use a larger and more accurate branch
predictor, our processor's performance will only improve slightly despite a
significant increase in the size of these structures. After a point, the increase in
performance is no longer worth the cost of the extra transistors. It's also possible
that performance might reduce overall as we may need to lower our clock frequency as
the structures get larger. Finally, we could introduce more pipeline stages, but we
know this doesn't necessarily lead to higher performance, as mispredictions may
become more costly. The combination of these issues means that extracting
performance using instruction-level parallelism alone becomes more expensive as more
performance is sought. This graph shows how the energy cost of executing an
instruction grows quickly as we try to build higher performance processors. Let's
look at some example designs. Suppose we have a core, which requires a certain area.
If we double its area, its performance improves, although there is a small rise in
energy per instruction. If we quadruple its area instead, its performance has now
doubled compared to our original core, while energy has increased by 50 percent.
Going further, if we increase our processor's area by a factor of 10, performance is
only 2 point 5 times our original core, but energy per instruction is now 3 times
higher. Its performance does not improve as fast as the cost of running it! Of
course, engineers are clever and determined, and are constantly developing new
techniques to bypass many of these issues. This means the performance of processors—
even ones running a single thread or program— still improves by around 10 to 25
percent each year. Nevertheless, ultimately we often need more performance than can
be provided by instruction-level parallelism alone. A modern solution is to employ
multiple processor cores on the same chip—called a "multicore" processor. This
changes the task for programmers; they may need to redesign their programs to take
advantage of such parallelism, but if they can, it can give vast performance
benefits. As we've learned throughout the course, every decision involves trade-offs
and compromise. We are faced with a fascinating but often highly-constrained design
problem. We've seen how performance bottlenecks, that at first seem impassable, can
be overcome with innovative designs. What might the future hold for microprocessors?
Can you think of ideas? What would you design? [music]

Module 6, Lab
[music] In this exercise, we'll be using a simulator to explore an out-of-order
superscalar processor design. The simulator allows us to configure a number of
aspects of our processor, for example, the number of pipeline stages, the width of
the Fetch stage, the size of the issue window, the number of ALUs, and the size of
our re-order buffer. The changes will be reflected in the pipeline diagram, which we
can see below, and also in the statistics below that, with key design metrics, such
as the clock frequency, clock period, and design size, visible below. When we press
"step," the simulation will advance by 1 clock cycle, and so we can see, for
example, on the first clock cycle the 1st 4 instructions are loaded. Although three
of them are unusable because they follow a taken branch and therefore these will not
be executed and will be discarded on the next clock cycle. We can press "run" to
watch our design in action and in order to quickly get to the end, we can use the
"fast forward" feature to simulate the millions of instructions in this particular
program. After the simulation is complete, we can check below to see a number of
statistics about our pipeline, which are useful for understanding why the
performance is as it is, and in particular we can see the program execution time.
1.18 milliseconds gives us the overall execution time of the program that our design
achieved, and notably our average instructions per cycle gives us the number of
instructions that we were able to complete on each clock cycle, in this case well
over 1, indicating that we are taking advantage of instruction level parallelism in
this simulation. Below all that, we can see our completed simulations and we can
check back on these as we explore new designs. So let's say, for example, we were
curious about the effect of increasing the re-order buffer size. We could change
that and then press "run" to quickly run our next experiment. And then if we scroll
down to our table, we can compare the results and see that actually the program
execution time was substantially increased by increasing the size of the re-order
buffer. Although, admittedly, this did come at a not insignificant increase in the
area of our design, and so whether or not this represents a good trade-off in
practice would very much depend on the problem we're trying to solve. In the
exercises, you'll be given a number of scenarios for processors, which generally
revolve around certain constraints on the design size, or the clock frequency, or
the target program execution time. And once you think you've found a design that
meets the goals required, configure it up using the settings and then scroll down to
the "submit" button at the bottom, and click that to have your answer checked. Best
of luck. [music]