RAI2025-Study-Guide-Learning-Objectives

Download as pdf or txt
Download as pdf or txt
You are on page 1of 20

2024-2025

RAI STUDY GUIDE


AND LEARNING
OBJECTIVES
2024-2025 RAI STUDY GUIDE
AND LEARNING OBJECTIVES
Topic Outlines and Test Weightings

This RAI Study Guide and Learning Objectives document describes the primary topics covered in the RAI Exam. GARP’s AI
Advisory Committee validated these curriculum topics as essential for a well-rounded understanding of the opportunities
and risks associated with artificial intelligence (AI). The curriculum is weighted across these topics and re-evaluated
annually to ensure the RAI Exam is timely and relevant.

Learning Objectives appear as bullet points at the beginning of each module and chapter outlined in this Study Guide.
These Learning Objectives are intended to help exam candidates identify major themes associated with each module or
chapter of the 2024-2025 RAI curriculum.

This document is an important study resource and should be referred to regularly during exam preparation.

RAI Exam Approach

The RAI Exam is practice oriented. The questions are derived from concepts presented in the five modules of the 2024-
2025 RAI curriculum, and often framed in the context of real-world work scenarios.

The Exam is comprehensive in nature, testing candidates on content across the RAI curriculum. The Exam consists of 80
equally weighted, multiple-choice questions. All Exam questions are standalone; however, a set of several questions may
rely on the same stimulus material (i.e., scenario, data set, etc.).

2024-2025 RAI Curriculum

The 2024-2025 RAI curriculum covers knowledge areas necessary for individuals and organizations to successfully manage
the impact of AI on organizations. The topics covered by these modules were vetted and approved by GARP’s AI Advisory
Committee. It is strongly suggested candidates review the curriculum in depth prior to sitting for the Exam. Modules 1, 3,
4, and 5, and each of the 10 chapters within Module 2, begin with a set of Learning Objectives to guide candidates through
their studies. Review questions conclude each chapter. Access to the full RAI curriculum on GARP’s eLearning platform,
GARP Learning, is provided to all candidates who register for the Exam.

RAI Errata

GARP will regularly update curriculum clarifications or printing errors in the RAI Errata document. Candidates can find the
RAI Errata within the “Lessons” tab of the RAI curriculum on GARP Learning. If you identify a potential error or curriculum
discrepancy, please submit this through the “Report Content Errors” link at the bottom of each page of content, but only
after checking the Errata to see if it may already have been addressed. We welcome all types of submissions including
minor editorial, grammar, and typographical errors. However, only substantial errors in content or questions will be posted
to the official RAI Errata.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 1


CONTENTS
3 Module 1:
AI and Risk: Introduction and Overview

4 Module 2:
Tools and Techniques

5 Chapter 1:
Introduction to Tools and Techniques

6 Chapter 2:
Unsupervised Learning

7 Chapter 3:
Supervised Learning – Econometric Techniques

8 Chapter 4:
Supervised Learning – Machine Learning Techniques

9 Chapter 5:
Semi-Supervised Learning

10 Chapter 6:
Reinforcement Learning

11 Chapter 7:
Supervised Learning – Model Estimation

12 Chapter 8:
Supervised Learning – Model Performance Evaluation

13 Chapter 9:
Natural Language Processing

14 Chapter 10:
Generative AI and LLMs

15 Module 3:
Risks and Risk Factors

16 Module 4:
Responsible and Ethical AI

17 Module 5:
Data and AI Model Governance

18 RAI Program Advisers and Content Developers

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 2


MODULE 1:
AI AND RISK: INTRODUCTION
AND OVERVIEW
EXAM WEIGHT: 8-12%

Learning Objectives
The application of artificial intelligence (AI) introduces a novel set of risks to all organizations that use it. This module offers
a historical perspective on AI, an overview of both machine learning methodologies and generative AI, and an introduction
to the risks associated with using AI/ML.

After completing this module, you should be able to:


• Explain some central principles of Classical AI, including search methods and recursion.
• Describe, at a high level, how reinforcement learning works.
• Describe, at a high level, how neural networks work, and how they differ from Classical AI systems.
• Articulate the potential and limitations of deep learning.
• Identify key breakthroughs leading to advances in AI and ML.
• Compare and contrast reinforcement, supervised, and unsupervised learning, and identify practical applications for
each technique.
• Discuss risks associated with inscrutability in AI and ML.
• Discuss risks associated with over-reliance on AI systems.
• Discuss ways in which AI exposes individuals, organizations, and society to risk.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 3


MODULE 2:
TOOLS AND TECHNIQUES
EXAM WEIGHT: 25-35%

This module, organized in 10 distinct chapters, provides an in-depth look at the following AI/ML tools and techniques:
• Introduction to tools and techniques
• Unsupervised learning
• Supervised learning – Econometric techniques
• Supervised learning – Machine learning techniques
• Semi-supervised learning
• Reinforcement learning
• Supervised learning – Model estimation
• Supervised learning – Model performance evaluation
• Natural language processing
• Generative AI and LLMs

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 4


CHAPTER 1:
INTRODUCTION TO TOOLS
AND TECHNIQUES
Learning Objectives
Machine learning (ML) is an umbrella term used to cover a range of techniques for training models to recognize data
patterns for a variety of applications, including prediction and classification. This chapter offers a high-level introduction to
ML techniques and applications, the potential benefits, and considerations for implementation.

After completing this chapter, you should be able to:


• Differentiate between machine-learning techniques and classical econometrics.
• Differentiate among unsupervised, supervised, semi-supervised, and reinforcement learning models.
• Distinguish between different data types.
• Describe how to encode categorical variables.
• Describe how to clean data and the benefits of cleaning.
• Describe data preparation techniques and their benefits.
• Apply transformations to a set of data.
• Discuss how principal components analysis (PCA) is used to reduce the dimensionality of a data set.
• Explain the differences between the training, validation, and test data sub-samples, and how each is used.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 5


CHAPTER 2:
UNSUPERVISED LEARNING
Learning Objectives
Unsupervised learning is associated with a model’s use of unlabeled data to develop insights or pattern recognition
with no specific guidance or rules. This chapter introduces clustering analysis, or segmentation, a common application
of unsupervised learning that separates data points into groups based on the “closeness” of their features. Focusing on
K-means clustering, a widely used approach, the chapter outlines how the method works, how to select the optimal number
of clusters, how its performance can be evaluated, and the strengths and weaknesses in its application.

After completing this chapter, candidates should be able to:


• Differentiate between clustering techniques.
• Illustrate how the K-means algorithm separates data into clusters, and describe the advantages/disadvantages of
K-means clustering.
• Describe performance measures such as Within the Cluster Sum of Squares (WCSS) and Between the Clusters Sum of
Squares (BCSS).
• Apply different methods to determine the optimal number of clusters in unsupervised learning.
• Describe the construction and uses of a dendrogram.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 6


CHAPTER 3:
SUPERVISED LEARNING –
ECONOMETRIC TECHNIQUES
Learning Objectives
This chapter covers the core models used for supervised learning that arise from econometrics. Broken out in two main
parts, the chapter first examines linear regression models that constitute the foundation upon which more sophisticated
approaches are built. It then provides an overview of the types of models used in the context of classification problems
including logistic regression and linear discriminant analysis.

Supervised learning techniques originating from computer science and more commonly associated with machine learning
will be covered in Chapter 4.

After completing this chapter, you should be able to:


• Identify uses and limitations of the following models:
- Single and multi-variable linear regression
- Single and multi-variable non-linear regression
• Interpret the results of the following regression analyses:
- Single and multi-variable regression
- Single and multi-variable non-linear regression
• Identify problems that may occur with linear regression models and possible remedies for them.
• Describe how logistic regression can be applied to classification problems.
• Describe the use of linear discriminant analysis for classification problems.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 7


CHAPTER 4:
SUPERVISED LEARNING –
MACHINE LEARNING TECHNIQUES
Learning Objectives
This chapter continues the discussion of models for supervised learning with a focus on machine learning techniques
grounded in computer science. It provides an overview of techniques applied in classification and prediction problems,
including decision trees, K-nearest neighbors, and support vector machines. An overview of neural networks — a modeling
method used in machine learning to replicate how the human brain processes data to perform functions like time-series
prediction and natural language processing — is also included. The chapter concludes with the presentation of autoencoders.

After completing this chapter, you should be able to:


• Differentiate between the two types of decision trees and illustrate how each is constructed and interpreted.
• Explain how pruning and ensemble techniques can be used to enhance the performance of decision trees.
• Apply the K-nearest Neighbors method for classification.
• Illustrate how support vector machines are used to classify data.
• Describe how neural networks are constructed and discuss associated challenges.
• Discuss advanced neural network structures.
• Describe how autoencoders are used for dimensionality reduction and differentiate between autoencoders and PCA.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 8


CHAPTER 5:
SEMI-SUPERVISED LEARNING
Learning Objectives
Semi-supervised learning is typically applied when data sets contain a mix of labeled and unlabeled data. This chapter
presents the assumptions that must be satisfied for semi-supervised learning to be effective as well as illustrations of self-
training and co-training, both popular methods of semi-supervised learning.

After completing this chapter, you should be able to:


• Explain how semi-supervised learning differs from unsupervised and supervised learning.
• Discuss the assumptions required for effective semi-supervised learning.
• Compare and contrast self-training and co-training methods of semi-supervised learning.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 9


CHAPTER 6:
REINFORCEMENT LEARNING
Learning Objectives
This chapter introduces reinforcement learning, a machine learning technique that applies a trial-and-error feedback loop to
train models to optimize short-term decisions that maximize a defined long-term reward. The “output” from reinforcement
learning applications is a recommended action based on defined parameters rather than a prediction, classification, or
cluster produced in unsupervised or supervised learning applications.

After completing this chapter, you should be able to:


• Explain key principles and frameworks behind reinforcement learning.
• Compare and contrast exploration, exploitation, and ε-greedy strategies.
• Describe reinforcement learning in the context of the Multi Armed Bandit (MAB) problem.
• Explain Markov decision processes.
• Differentiate between the Monte Carlo and Temporal Difference methods.
• Describe how neural networks can be used in reinforcement learning.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 10


CHAPTER 7:
SUPERVISED LEARNING –
MODEL ESTIMATION
Learning Objectives
This chapter builds on concepts learned in Chapter 3, with a focus on the estimation of linear regression models using
ordinary least squares and maximum likelihood methods, the estimation of model parameters when data with nonlinear
characteristics is used, and the optimization of model parameters using gradient descent method. Initial insight on the
predictive value of models and techniques for improving model output is also covered, including over-and-under-fitting,
bias-variance tradeoff, and methods for adjusting models with highly correlated features.

After completing this chapter, you should be able to:


• Compare and contrast the Ordinary Least Squares (OLS) and Maximum Likelihood methods.
• Explain how gradient descent method is used to optimize parameter estimates.
• Explain how backpropagation is used to determine the weights in neural networks.
• Discuss the differences between underfitting and overfitting and potential remedies for each.
• Describe the tradeoff between bias and variance.
• Explain the use of regularization techniques to simplify models.
• Describe cross-validation and its uses.
• Describe the accuracy-interpretability tradeoff.
• Describe how grid search and bootstrapping can be used to optimize hyperparameter estimation.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 11


CHAPTER 8:
SUPERVISED LEARNING – MODEL
PERFORMANCE EVALUATION
Learning Objectives
Previous chapters mentioned the concept of “model evaluation,” specifically in the context of cross-validation and the fine-
tuning of hyperparameters. This chapter formalizes these concepts by introducing metrics that can be used to evaluate the
individual performance of a model or for comparison across models. A distinction is made between the measures used to
evaluate the performance of a model when the output is a continuous variable and those used for classification models.

After completing this chapter, you should be able to:


• Discuss metrics used to evaluate the performance of a model when the outcome variable is continuous.
• Evaluate the performance of a classification model using a confusion matrix and related metrics.
• Explain the relationship between true and false positive rates and how this trade off can be illustrated using the
receiver operating curve (ROC).

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 12


CHAPTER 9:
NATURAL
LANGUAGE PROCESSING
Learning Objectives
Natural language processing, sometimes also known as content analysis, text mining or computational linguistics, is one of
the most exciting and fast-developing applications of machine learning. NLP applies data with an unstructured, free text
format to understand and analyze human language, both written and spoken. The U.S. Securities Exchange Commission
was an early adopter of NLP in its effort to detect accounting fraud.

This chapter provides a comprehensive overview of NLP models, including the preparation of textual information for use in
NLP models, the construction of NLP models, a comparison of non-machine learning approaches to NLP models, and how
NLP model fit can be evaluated.

After completing this chapter, you should be able to:


• Discuss applications of natural language processing (NLP).
• Describe pre-processing steps for NLP.
• Discuss the bag of words (BoW) and n-grams approaches.
• Explain how the naïve Bayes classifier is used to categorize documents.
• Illustrate how term frequency-inverse document frequency (TF-IDF) can be used to determine the appropriate
weighting to assign to words in a document.
• Describe and contrast different approaches to sentiment analysis.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 13


CHAPTER 10:
GENERATIVE AI AND LLMS
Learning Objectives
The well-known ChatGPT is based on a generative AI technology known as a transformer, a specific type of a large
language model (LLM). The rapid adoption of ChatGPT has created confusion regarding the distinctions between GenAI
and LLMs. This chapter provides an understanding of the relationship between GenAI, LLMs, and the technologies and
algorithms that underlie them.

After completing this chapter, you should be able to:


• Describe and distinguish between different generative artificial intelligence technologies.
• Describe the role of LLMs in GenAI.
• Explain how embeddings are used to represent word vectors.
• Differentiate between the two Word2Vec architectures.
• Differentiate between recurrent neural networks (RNNs) and transformers for capturing the relationships between
words in a sentence.
• Describe the basic structure of LLMs at a conceptual level.
• Discuss prompt engineering and temperature in the context of LLMs.
• Describe applications of GenAI and LLMs.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 14


MODULE 3:
RISKS AND RISK FACTORS
EXAM WEIGHT: 15-25%

Learning Objectives
This module provides a comprehensive overview of the primary risks associated with AI development and deployment. It
discusses the numerous challenges associated with the creation of a “fair” algorithm, highlighting the different sources of
bias that might affect algorithmic fairness. It also addresses the twin problems of explainability and interpretability, and
other noteworthy risks, including risk to human autonomy, risk of AI-driven manipulation, reputational risk, existential risk,
and global risks and challenges.

After completing this module, you should be able to:


• Describe and differentiate between the concepts of individual and group fairness.
• Describe various measures of group fairness.
• Discuss trade-offs associated with different concepts and measures of fairness.
• Describe sources of algorithmic bias and unfairness.
• Describe explainability, interpretability, and transparency.
• Describe techniques for making AI algorithms more explainable.
• Discuss risks posed by AI to human autonomy, safety, and wellbeing.
• Describe sources of AI-related reputational risk and strategies for mitigating those risks.
• Discuss global challenges and risks associated with AI.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 15


MODULE 4:
RESPONSIBLE AND ETHICAL AI
EXAM WEIGHT: 15-25%

Learning Objectives
This module builds on the risks examined in Module 3 and explores how ethical principles and governance can guide the
development and deployment of AI technologies in a way that promotes trust, safety, and fairness. It also presents various
ethical frameworks that can be applied to AI, the governance challenges associated with AI, and current global governance
initiatives around AI.

After completing this module, you should be able to:


• Discuss potential benefits of implementing a practical ethics framework.
• Compare and contrast consequentialism, deontology, and virtue ethics.
• Discuss the principles of nonmaleficence, beneficence, justice, autonomy, and explainability.
• Discuss sources of and strategies to address algorithmic bias and unfairness.
• Describe important ethical principles related to privacy.
• Discuss the current regulatory landscape and governance challenges associated with AI.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 16


MODULE 5:
DATA AND AI
MODEL GOVERNANCE
EXAM WEIGHT: 15-25%

Learning Objectives
This module discusses data and model governance and provides a starting point to establish a firm-specific model
validation framework across the entire AI/ML model life cycle — from model development through performance monitoring
and decommissioning. The principles presented apply to a wide range of industries, but the primary focus is on the financial
sector, and the quantitative risk models (QRMs) heavily relied upon and subject to formal regulatory oversight. The opacity
of AI/ML models is also discussed, along with the need for proper governance of the data used to train these models.

After completing this module, you should be able to:


• Describe elements of a data governance framework.
• Describe elements of a model governance framework.
• Describe steps in the model development and testing process.
• Discuss model validation and its importance.
• Discuss policies and procedures related to model governance.
• Describe factors to be considered when registering AI/ML applications in a model inventory.
• Discuss roles and responsibilities associated with model risk management.
• Describe how the model review framework differs for AI/ML models.
• Describe the steps involved in model implementation and adaptation.
• Discuss potential sources of misinterpretation of model results.

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 17


RAI PROGRAM ADVISERS AND
CONTENT DEVELOPERS
Amine Aboussalah, Ph.D, Assistant Professor, NYU, Finance & Risk Engineering
*Joseph Breeden, Ph.D, CEO, Deep Future Analytics; President Model Risk Management International Association
Chris Brooks, Ph.D, Professor of Finance, University of Bristol
*Agostino Capponi, Ph.D, Associate Professor, IEOR, Columbia University
*Rama Cont, Ph.D, Head of the Oxford Mathematical and Computational Finance Group, University of Oxford
*Charles Currat, Ph.D, FRM, Head of Quantitative Modeling, Wells Fargo Investment Institute
*Lucy da Piedade, FRM, Chief Controls Officer Consumer Banking & Payments, Barclays
Thomas Dahlin, Chief Model Risk Officer, Centennial Bank
*Alexander Denev, Ph.D, FRM, Co-Founder, Turnleaf Analytics
*Chris Donohue, Ph.D, Managing Director, GARP Benchmarking Initiative (GBI), GARP
*Kenneth Doucet, Vice President, Content, Information and Continuing Education, GARP
Michael Dowling, Ph.D, Professor of Finance, Dublin City University
*Raghurami Etukuru, Ph.D, FRM, Founder and Principal AI Scientist, AISCIENCES.AI
Paul Feehan, Ph.D, Distinguished Professor of Mathematics, Rutgers University
*Kay Firth-Butterfield, CEO, Good Tech Advisory LLC
Michael Imerman, Ph.D, Assistant Professor of Teaching in Finance, UC Irvine
Satyajit Karnik, Ph.D, Lecturer in Finance, Georgia Institute of Technology
*Lukas Kölbl, Ph.D, FRM, Head of Data Science DACH, Accenture
*William May, Managing Director, Global Head, Certification and Educational Programs, GARP
*Peter Millican, Ph.D, Professor of Philosophy, Head of Education and Outreach, Institute for Ethics in AI, University of Oxford
*Yogesh Mudgal, Head of Risk, Runtime Compute, JP Morgan Chase
Manuela Pedio, Ph.D, Senior Lecturer in Finance, University of Bristol
*Carina Prunkl, Ph.D, Assistant Professor for Ethics of Technology, Utrecht University
Peter Quell, Ph.D, Head of Portfolio Analytics for Market and Credit Risk, DZ BANK AG
*Anand Rao, Ph.D, Distinguished Service Professor of Applied Data Science and Artificial Intelligence, Carnegie Mellon
University; Global AI Lead, PwC (retired)
*Alberto Rossi, Ph.D, Professor of Finance and Director of the AI, Analytics and Future of Work Initiative,
Georgetown University
Evan Sekeris, Ph.D, Chief Model Risk Officer, Capital One, FRM Advisory Committee Member
*Rajesh Shekhar, FRM, Head of Global Data Science, H2O.ai
*Stephen Slade, Ph.D, FRM, Senior Lecturer in Computer Science, Yale University
*Agus Sudjianto, Ph.D, Senior Vice President, Risk and Technology, H2O.ai
Carissa Veliz, Ph.D, Associate Professor, Institute for Ethics in AI, University of Oxford
Jennifer Voitle, FRM, Model Validation Expert
Samuel Po Shing Wong, Ph.D, Independent Researcher in Machine Learning and Applied Statistics
*Bo Xu, FRM, Principal, AI Lead, BCG
*Qiuyan Xu, Ph.D, FRM, Managing Director, Gravitate AI
Kishore Yalamanchili, Ph.D, Director, Risk Content, GARP

* Advisory Committee member

Risk and AI Study Guide and Learning Objectives 2024-2025 garp.org/rai | 18


HEADQUARTERS
111 Town Square Place
14th Floor

garp.org
Jersey City, New Jersey
07310 USA
+1 (201) 719.7210
ABOUT GARP | The Global Association of Risk Professionals is a non-
partisan, not-for-profit membership organization focused on elevating the
practice of risk management. GARP offers the leading global certification LONDON
for risk managers in the Financial Risk Manager (FRM ), as well as the
®
17 Devonshire Square
Sustainability and Climate Risk (SCR®) Certificate, Risk and AI (RAI™) 4th Floor
Certificate, and ongoing educational opportunities through Continuing London, EC2M 4SQ UK
Professional Development. Through the GARP Benchmarking Initiative +44 (0) 20 7397.9630
(GBI) and GARP Risk Institute, GARP sponsors research in risk management
®

and promotes collaboration among practitioners, academics, and regulators.


HONG KONG
Founded in 1996 and governed by a Board of Trustees, GARP is The Center
headquartered in Jersey City, N.J., with offices in London and Hong Kong. 99 Queen’s Road Central
Office No. 5510
For more information, visit garp.org or follow GARP on LinkedIn, Facebook, 55th Floor
and X. Central, Hong Kong SAR,
China
© 2024 Global Association of Risk Professionals. All rights reserved. (11.24) +852 3168.1532

You might also like