FireSim and Chipyard Tutorial: Welcome!

Sagar Karandikar
You must enter a 1. Fill out the form at
UC Berkeley
valid email on this
sagark@eecs.berkeley.edu
[in-person only]
form! → now for EC2 instance access
h en
T

2. You’ll receive two emails.

Follow insts to login, then wait.
FireSim and Chipyard Tutorial: Intro
Sagar Karandikar
UC Berkeley
sagark@eecs.berkeley.edu
Presenters/Organizers

Sagar Jerry Abraham Vighnesh

Karandikar Zhao Gonzalez Iyer

Sophia Bora Krste

Shao Nikolić Asanović
3
Getting Started/Logistics (recap)
• Fill out the form at [in-
person only] now for EC2
instance access
• You’ll receive two emails.
One from Google Forms and
one that looks like →
• Follow the instructions in
this one to login to your
FireSim manager instance,
then wait

4
A Golden Age in Computer Architecture
• No more traditional scaling…
• An architect’s dream: everyone
wants custom microarchitectures
and HW/SW co-designed systems
• Also, a golden age to have direct
impact as researchers
• Exploding open-source hardware
environment
• An open-ISA that can run software
we care about

https://cacm.acm.org/magazines/2019/2/234352-a-new-golden-
age-for-computer-architecture/fulltext
5
A Dark Age in Computer Architecture tools
• What do we need to do good architecture research?
• Need tools that let us evaluate designs on a variety of metrics:
• Functionality
• Performance
• Power
• Area
• Frequency
• Especially in small teams (grad students, startups), these tools need to be agile
• Historically, without good open IP, had to build abstract arch/uarch simulators
out of necessity
• But now, we have much better IP and software compatibility, so what’s stopping us?

6
A Dark Age in Computer Architecture tools
• Designed to be operated by
hundreds of engineers
• Not, 10s of engineers or 1s-10s of
grad students
• Three hard questions:
• Where do I get a collection of well-
tested hardware IP + complex
software stacks that run on it?
• How do I quickly obtain performance
measurements for a novel HW/SW
system?
• How do I get ASIC QoR feedback and
tape-out a design, with portability
across tools and processes?
7
Three hard questions, answered!
• Where do I get a collection of well-tested hardware IP + complex
software stacks that run on it?

• How do I quickly obtain performance measurements for a novel HW/SW

system?

• How do I get ASIC QoR feedback and tape out a design, with portability
across tools and processes? (and open-source and proprietary flows)

Hammer
8
Three hard questions, answered!

+
Hammer
Measure Functionality, Performance, Power,
Area, Frequency for real HW/SW systems,
quickly and easily, with small teams of engineers
9
What kinds of designs can I work with?

Si
• RISC-V Cores:
• Rocket Chip In-Order core, industry proven
• SonicBOOM Out-of-Order Superscalar core

Other Periph.
• CVA6

Rocket

Rocket

Rocket

Rocket
• Ibex

Core

Core

Core

Core

Other Peripherals
• Accelerators

Server
DRAM

Blade
• Gemmini (Berkeley DNN Accelerator)

Sim.
• sha3 accelerator

NIC
• NVDLA (NVIDIA Deep Learning Accelerator)

L1D

L1D

L1D

L1D
L1I

L1I

L1I

L1I
• Hwacha Vector Accelerator

NIC Sim
• FFT Generator
• Many more L2
• Peripherals/other IP
• L2 Cache, UART, Disk, Ethernet NIC, etc.
• FPGA-Simulation Models DRAM Model
Complete Single-
FPG
Fabr
• Large LLCs, large DDR3 memory systems
SoC RISC-V System
10
What kinds of designs can I work with?

Ethernet- Rack Rack Rack Rack

Networked Aggregation Switch

DRAM
L1I

L1D
Rocket
Core
DRAM
Aggregation Pod

DRAM Model
Server
L1I Rocket
L1D Core
Server

L2
L1I Rocket

Blade Blade

FPGA FPGA FPGA

L1D Core

L1I Rocket

Sim. SimulaIon
L1D Core

Other Peripherals
NIC

NIC Sim Other Periph.

Fabric
FPGA
Endpoint Sim Endpoints

1024 Node Rack Rack Rack

PCIe to Host

(4 Sims) (4 Sims) Server

Blade
Server
Blade
(4 Sims)
Simulation Simulation

DRAM DRAM

Host Instance CPU: ToR Switch Model

FPGA FPGA FPGA FPGA

(4 Sims) (4 Sims) (4 Sims) (4 Sims)

(4096 Core)
Step N: Title (placeholder slide)

Root Switch
Modeled System
Resource Util

13

System on
256 Cloud Aggregation Pod Aggregation Pod
FPGAs
11
Join the FireSim Community!:
Open-source users and industrial users
• More than 200 mailing list members • Companies publicly announced
and 850 unique cloners per-week using FireSim
• Projects with public FireSim support • Esperanto Maxion ET
• Intensivate IntenCore
• Chipyard
• Rocket Chip • SiFive validation paper @ VLSI’20
• BOOM • Galois and Lockheed Martin (DARPA
SSITH/FETT)
• Hwacha Vector Accelerator
• Keystone Secure Enclave
• Gemmini
• NVIDIA Deep Learning Accelerator
(NVDLA):
• NVIDIA blog post: https://devblogs.nvidia.com/nvdla/
• BOOM Spectre replication/mitigation
• Protobuf Accelerator
• Too many to list here!
Esperanto announcement at RISC-V Summit 2018
12
FireSim in DARPA FETT
• DARPA SSITH: Building hardware defenses to
address common software vulnerabilities
• DARPA FETT: How good are the defenses built
in SSITH?
• Multiple designs hosted for attack in FireSim [1]
• “Morpheus II: A RISC-V Security Extension for
Protecting Vulnerable Software and Hardware”
• Developed by UT Austin, U Mich., Agita Labs
• Hosted on FireSim for FETT [2]
• Over 500 attackers tried to break Morpheus II
defenses, working for large bug bounties. None [1] K. Hopfer. Leveraging Amazon EC2 F1 Instances for Development and Red
succeeded [3] Teaming in DARPA’s First-Ever Bug Bounty Program. AWS APN Blog. May 2021.
[2] A. Harris, et. al., “Morpheus II: A RISC-V Security Extension for Protecting
Vulnerable Software and Hardware”. In proceedings of the 2021 IEEE International
Symposium on Hardware Oriented Security and Trust (HOST), December 2021.
[3] T. Austin., et. al., “Morpheus II: A RISC-V Security Extension for Protecting
Vulnerable Software and Hardware”. In HotChips 33, August 2021.
13
Join the FireSim Community!:
Academic Users and Awards
• ISCA ‘18: Maas et. al. HW-GC Accelerator (Berkeley) • Awards: FireSim ISCA ‘18 paper:
• MICRO ‘18: Zhang et. al. “Composable Building Blocks to Open up • IEEE Micro Top Pick
Processor Design” (MIT) • CACM Research Highlights
Nominee from ISCA ’18
• RTAS ‘20: Farshchi et. al. BRU (Kansas)
• Awards: FireSim users:
• EuroSys ‘20: Lee et. al. Keystone (Berkeley) • ISCA ‘18 Maas et. al.:
• IEEE Micro Top Pick
• OSDI ‘21: Ibanez et. al. nanoPU (Stanford)
• MICRO ‘18 Zhang et. al.:
• USENIX Security ‘21: Saileshwar et. al. MIRAGE (Georgia Tech) • IEEE Micro Top Pick
• CCS ‘21: Ding et. al. “Hardware Support to Improve Fuzzing • MICRO ‘21 Gottschall et. al.:
Performance and Precision” (Georgia Tech) • MICRO-54 Best paper runner-up
• MICRO ‘21 Karandikar et. al.:
• MICRO ’21: Karandikar et. al. “A Hardware Accelerator for Protocol • MICRO-54 Distinguished Artifact
Buffers” (Berkeley/Google) winner
• IEEE Micro Top Pick Honorable
• MICRO ‘21: Gottschall et. al. TIP (NTNU) Mention
• Over 20 additional user papers on the FireSim website: • DAC ‘21 Genc et. al.:
• https://fires.im/publications/#userpapers • DAC 2021 Best Paper winner

14
Join the FireSim Community!:
Academic Users and Awards
• ISCA ‘18: Maas et. al. HW-GC Accelerator (Berkeley) • Awards: FireSim ISCA ‘18 paper:
• MICRO ‘18: Zhang et. al. “Composable Building Blocks to Open up • IEEE Micro Top Pick
Processor Design” (MIT) • CACM Research Highlights
Nominee from ISCA ’18
• RTAS ‘20: Farshchi et. al. BRU (Kansas)
•
FireSim has
EuroSys ‘20: Lee et. al. Keystone (Berkeley)
been used* in published
• Awards: FireSim users:
• ISCA ‘18 Maas et. al.:
• OSDI ‘21: Ibanezwork from
et. al. nanoPU authors at over 20• academic
(Stanford) • IEEE Micro Top Pick
MICRO ‘18 Zhang et. al.:
• USENIX Security ‘21: Saileshwarandet. al.industrial
MIRAGE (Georgiainstitutions
Tech) • IEEE Micro Top Pick
• CCS ‘21: Ding et. al. “Hardware Support to Improve Fuzzing • MICRO ‘21 Gottschall et. al.:
Performance and Precision” (Georgia Tech) • MICRO-54 Best paper runner-up
• MICRO ‘21 Karandikar et. al.:
*actually
• MICRO ’21: Karandikar et. al. “A Hardware used,fornot
Accelerator only
Protocol cited • MICRO-54 Distinguished Artifact
Buffers” (Berkeley/Google) winner
• IEEE Micro Top Pick Honorable
• MICRO ‘21: Gottschall et. al. TIP (NTNU) Mention
• Over 20 additional user papers on the FireSim website: • DAC ‘21 Genc et. al.:
• https://fires.im/publications/#userpapers • DAC 2021 Best Paper winner

15
Today’s Logistics

You are
here

16
Running a FireSim FPGA Build
• This will take a while, so we will run this in the background:
tmux new -s fpgabuild # this will give you a persistent
# session you can reattach to
firesim managerinit --platform f1
[When prompted, enter your email address to get a build completion
notification]

# runs the HW build, all the way to AGFI

firesim buildbitstream
[Lastly, detach from tmux with “ctrl-b d”. We will return to this build later.]
[this will build a design called firesim_rocket_singlecore_no_nic_l2_lbp]

17
Today’s Agenda
9:00am: Introduction/Overview, Amazon EC2 Instance Setup, Logistics -
Sagar
9:30am: Chipyard Basics – Jerry
10:00am: Building Custom RISC-V SoCs in Chipyard: Part 1 – Jerry
10:20am: Coffee break
10:40am: Building Custom RISC-V SoCs in Chipyard: Part 2 – Jerry
11:30am: Hammer VLSI flow - Vighnesh
12:00pm – 1:40pm: Lunch

18
Today’s Agenda
1:40pm: FireSim Introduction - Sagar
2:10pm: Building Hardware Designs in FireSim – Abe
2:40pm: Building Software Workloads with FireMarshal - Abe
3:20pm: Coffee break
3:40pm: Running a FireSim Simulation: Booting Linux and Running
Hardware Accelerated ResNet-50 – Abe
4:10pm: Debugging and Profiling FireSim-Simulated Designs – Sagar
4:40pm: FireSim Local (On-Prem) FPGA Demo – Abe
4:55pm: Conclusion – Sagar
5:00pm: End of Tutorial
19
Join us at the First FireSim/Chipyard
Workshop Tomorrow! fires.im/workshop-2023

20
Thanks to AWS, Xilinx, and SLICE/ADEPT Lab
Sponsors

NSF Award #2016662

SLICE Lab Sponsors: CCRI: ENS: Chipyard

21