AI VIETNAM

All-in-One Course
(TA Session)

Deploy Deep Learning models

with FastAPI
Extra Class

Dinh-Thang Duong – TA
Nguyen-Thuan Duong – STA

Year 2024
 AI VIETNAM
All-in-One Course
(TA Session) Getting Started
❖ Objectives

Our objectives:
- Discuss about the definition of API.
- Learn the basis of FastAPI.
- Learn how to deploy a Deep Learning model
as an API service with FastAPI.

2
 AI VIETNAM
All-in-One Course
(TA Session)

Outline
➢ Introduction
➢ API
➢ FastAPI
➢ Model Deployment
➢ Question

3
 AI VIETNAM
All-in-One Course
(TA Session)

Introduction

4
 AI VIETNAM
All-in-One Course
(TA Session) Introduction
❖ Getting Started

How do we actually interact with ChatGPT behind the scene?

5
 AI VIETNAM
All-in-One Course
(TA Session) Introduction
❖ Getting Started

The ChatGPT website get the response from a something called API
6
https://semaphoreci.com/blog/function-calling
 AI VIETNAM
All-in-One Course
(TA Session)

API

7
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ Getting Started
• API (Application Programming Interface) is a facilitator that enables apps, databases, softwares and IoT
devices to communicate with each other.

User Application API Server

8
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ Getting Started

Make the order Take the order

Delivery of order Bringing from

kitchen

Customer Waiter Chef

9
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ Getting Started
• The application sending the request is called the client, and the application sending the response is called
the server.

Client Server

10
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ How do APIs work ?
• A client sends a request to API server. The request is made using a specific protocol (such as HTTP) and
includes information about the operation the client wants to perform, e.g., retrieving data or updating a
resource.

Request

Response

Client API server

• The API server receives the request and processes it. The API server sends a response back to the client,
which may include data, an error message, or a status code indicating the result of the operation.

• The client receives the response and processes it.

11
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ Types of APIs

Database
Private API
API

Public API Remote API

Partner API Web API

12
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ Private / Public / Partner API

APIs are exposed only to

APIs are open to any APIs are open to select
existing developers within the
developer business partners
enterprise

Apps are more targeted Apps could be targeted at end App are usually targeted at
towards end consumers consumers or business users employees of the enterprise

13
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ Database API
• A database API allows applications to interact with a database to access and manipulate data.

Request Query database

Response with data Data

Client Database

14
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ Remote API
• Using a remote API, two distant applications communicate across a communications network, primarily the
internet.

Internet

Application Server Database

15
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ Web API
• Web API (or Web Service API) is an application inferface between a web browser and a web server.

Request Request

Response Response

Web Application REST / SOAP Web Server

API

• All web services are APIs but not all APIs are web services.
• Web services require an internet connection to work, but APIs do not.

16
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ API protocol types

17
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ Timeline

SOAP Json-RPC GraphQL

1999 2005 2015

1998 2000 2016

XML-RPC REST gRPC

18
 AI VIETNAM
All-in-One Course
(TA Session) API
❖ Conclusion
REST GraphQL SOAP RPC
Structure Follows six Schema and type Message structure Local procedural
architectural calls
constraints
Format Json, XLM, HTML, Json XML Json, XML,
plain text Flatbuffers, etc
Advantages Flexible terms of Sovles over-fetching Highly secure and Lightweight
data format and and under-fetching extensible payloads make it
structure high performing
Use cases Resources based Mobile APIs Payment gateways Command-focused
apps systems

19
 AI VIETNAM
All-in-One Course
(TA Session)

FastAPI

20
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Introduction

FastAPI is a modern, fast (high-performance), web

framework for building APIs with Python 3.8+ based on
standard Python type hints.

21
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Python Web Framework

22
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Django vs Flask vs FastAPI

23
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Which companies use FastAPI ?

24
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ The key features of FastAPI

High performance On par with NodeJS and GO

Fast to code Increase the speed to develop features by about 200% to 300%

Easy Easy to use and learn. Less time reading docs

Robust Get production-ready code. With automatic interactive documentation

25
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice

Learn how to use FastAPI to deploy an API service

26
https://www.apisec.ai/blog/rest-api-and-its-significance-to-web-service-providers
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Initialize first API

1. Create a FastAPI instance 2. Run uvicorn to deploy the API

27
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Initialize first API

28
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Initialize first API

Swagger UI: https://127.0.0.1:8000/docs

29
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Initialize first API

Swagger UI: An interface to

visualize and interact with
API. Can be accessed
through subfix /docs.

30
https://swagger.io/tools/swagger-ui/
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Initialize first API

Swagger UI: An interface to

visualize and interact with
API. Can be accessed
through subfix /docs.

31
https://swagger.io/tools/swagger-ui/
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations

Request Response
“Hello world”

Client

Make API to return “Hello world” when we access to it

32
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations GET Method

Define a function with decorator “@app.get(‘/’)”:

- The root() function is the handler for requests to the

root URL path ‘/’ using HTTP GET method.

Defining a route

33
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations GET Method

34
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations

An API service may have several operations with

difference purposes. To separate those, we
define them as API Endpoints (routes, paths).

35
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations

https://api_domain.com
POST https://api_domain.com/catdog_classification/predict

POST https://api_domain.com/speech_classification/predict

GET https://api_domain.com/features_database/{features_id}

GET https://api_domain.com/speech_classification/results/{task_id}

API

36
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations

37
https://api.arcadier.com/introduction-to-arcadier-api
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations

How to retrieve and upload data from

a database through API ?

Database

38
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations GET Method

39
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations (Path Parameters)

Path Parameter: A part of URL path that is expected

to be a variable and is used to capture a value
directly from the path.

40
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations POST Method

41
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations PUT Method

42
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations DELETE Method

43
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Path Operations

44
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Pydantic Model

Request body: Data that is sent by Client.

Response body: Data that is sent by API.

Item: {
”name”: string,
”price”: number,
“description”: string,
“tax”: number
}

An example of request body

45
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Pydantic Model

Item: {
”name”: string,
”price”: number, To ensure API to receive the exact
“description”: string, structure, we can use Pydantic
“tax”: number Model.
}

An example of request body

46
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Pydantic Model

Pydantic Model is a way of defining data

structures with type annotations, ensuring
that the data adheres to a specified format
and type (Data Validator). We can call a
pydantic model as a model or schema.

47
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Pydantic Model

E.g: Define student information record:

- ID: string
- Name: string
- Age: number
- Gender: string

48
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Pydantic Model

49
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Pydantic Model

50
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Pydantic Model

A pydantic model can be nested

51
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Pydantic Model

52
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Parameters Annotation

FastAPI allows us to provide additional

information for parameters (path and
query) using typing Annotated.

53
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Parameters Annotation

Use Annotated to give additional information

for read_items() function.

54
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Response Model

We can specify the response type of the route

55
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Status Code

HTTP status codes are three-digit response codes or

messages sent by a server to user at the other end of the
server. These codes or responses enable servers to
communicate with users on the internet.

56
https://www.infidigit.com/blog/http-status-codes/
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Status Code

When a request is completed successfully, we got 200 OK

57
https://www.infidigit.com/blog/http-status-codes/
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Status Code

We can define the status code return in decorator

58
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Upload File

Beside Pydantic model, we can also have request body as a uploaded file 59
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Upload File

Use File or UploadFile module in FastAPI

60
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Middleware

A "middleware" is a function that works

with every request before it is processed
by any specific path operation. And also
with every response before returning it.

61
https://semaphoreci.com/blog/custom-middleware-fastapi
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Middleware

62
https://semaphoreci.com/blog/custom-middleware-fastapi
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Middleware

Add a response time (process time) to response headers.

63
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: Middleware

64
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: CORS Middleware

CORS (Cross-Origin Resource Sharing) refers to the

situations when a frontend running in a browser has
JavaScript code that communicates with a backend, and
the backend is in a different "origin" than the frontend.

65
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: CORS Middleware

66
https://www.haproxy.com/blog/enabling-cors-in-haproxy
 AI VIETNAM
All-in-One Course
(TA Session) FastAPI
❖ Practice: CORS Middleware

67
 AI VIETNAM
All-in-One Course
(TA Session)

Quiz

68
 AI VIETNAM
All-in-One Course
(TA Session)

Model Deployment

69
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Introduction

Deep
Learning
Model

You trained a new model

But how … ?

You want to share it for

other people to use… 70
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Pipeline

Applications

Data Collection Model Training Model Serving

71
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Practice
Problem statement: Deploy a Cat Dog Image Classification model as an API service using FastAPI.

Request

Response
{
"prediction_id": 0,
User "prediction_cls": "cat" API
}

72
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Practice
Problem statement: Deploy a Cat Dog Image Classification model as an API service using FastAPI.

Build a Deep Learning model

1

Request Query model

Response with Model prediction

prediction

Client Build an API service Model

2

73
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build model

74
https://www.analyticsvidhya.com/blog/2021/06/beginner-friendly-project-cat-and-dog-classification-using-cnn/
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build model: Dataset

Cat Dog dataset on HuggingFace. 75

 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build model step 1: Install and import libraries

76
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build model step 2: Load dataset and train val test split

77
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build model step 3: Create DataLoader

78
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build model step 4: Build model

79
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build model step 5: Training and saving model weights

80
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API
Problem statement: Deploy a Cat Dog Image Classification model as an API service using FastAPI.

Build a Deep Learning model

1

Request Query model

Response with Model prediction

prediction

Client Build an API service Model

2

81
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 1: Organize source code

We build this API in local computer with the

following source code structure

82
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 1: Organize source code
- config/: Folder containing configuration for some modules.
- logs/: Folder containing logging information when running API.
- middleware/: Folder containing code for middleware.
- models/: Folder containing Deep Learning weights.
- routes/: Folder containing API Endpoints declaration.
- schemas/: Folder containing Pydantic model declaration.
- utils/: Folder containing codes for general purpose (varying
between projects).
- app.py: Python file containing codes for FastAPI app initialization.
- requirements.txt: File containing packages version information to
run the soruce code.
- server.py: Python file containing codes to host the API service.
83
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 2: List packages requirements

84
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 3: Define configuration

We define configuration for Deep Learning model

and logging module

85
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 3: Define configuration

86
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 4: Build logging function

87
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 4: Build logging function

88
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 5: Define models folder

catdog_model.py

89
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 5: Define models folder

catdog_predictor.py
90
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 6: Define middleware folder

cors.py

We declare each Middleware in separated

files

91
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 6: Define middleware folder

http.py

92
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 7: Define Schemas

We define all Pydantic model in

schemas folder

93
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 8: Define routes

94
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Build API Step 9: Define app.py and server.py

95
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment
❖ Result
1

96
 AI VIETNAM
All-in-One Course
(TA Session)

Model Deployment (MLOps)

97
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment (MLOps)
❖ Getting Started

Applications

Data Collection Model Training Model Serving

98
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment (MLOps)
❖ Data Drift Problem

User input

User input
Applications
Data Collection
99
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment (MLOps)
❖ Data Drift Solution

Applications

More Data Collection Model Training Model Serving

100
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment (MLOps)
❖ Machine Learning Life Cycle

Data drift

Data Collection Model Training Model Serving Monitoring

Performing Poorly Concept drift

101
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment (MLOps)
❖ Machine Learning Operations (MLOps) pipeline

Experiments

Data Source Data Pipeline Training Pipeline

Model Registry

Source Control CI/CD Model Serving

Monitoring

Caching
102
 AI VIETNAM
All-in-One Course
(TA Session) Model Deployment (MLOps)
❖ MLOps simple pipeline

Experiments

Data Source Data Pipeline Training Pipeline

Model Registry

Caching Model Serving

Source Control

103
 AI VIETNAM
All-in-One Course
(TA Session) Summarization
❖ What we have learned so far
Summarization:
- Discuss about the definition of API.
- Learn the basis of FastAPI:
- Path Operations.
- Pydantic Model.
- Response Code.
- Middleware.
- Learn how to deploy a Deep Learning model
as an API service with FastAPI.
- Deploy a Cat Dog Classification model.
- Simple MLOps pipeline.
104
 AI VIETNAM
All-in-One Course
(TA Session) Question

?
105
106