Scribd
Upload
12 views5 pages

Web Scraping in Python

This document provides a guide on how to scrape inspirational quotes from the website goodreads.com using Python. It explains the concept of web scraping, the structure of the target website, and includes a detailed programming example using BeautifulSoup and requests libraries. The article concludes by encouraging readers to explore further resources on Python programming.

Uploaded by

Tharaka Gaddam
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
12 views5 pages

Web Scraping in Python

This document provides a guide on how to scrape inspirational quotes from the website goodreads.com using Python. It explains the concept of web scraping, the structure of the target website, and includes a detailed programming example using BeautifulSoup and requests libraries. The article concludes by encouraging readers to explore further resources on Python programming.

Uploaded by

Tharaka Gaddam
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 5

WEB SCRAPING IN PYTHON:

mmmmmmmmmmmmmmmmmkmmmmmmmmmmmmmmmmmmmmmkmWEB SCRAPE ON
INSPIRATIONAL QUOTES USING PYTHON:

{HOW TO SCRAPE INSPIRING QUOTES FROM THE INTERNET WITH PYTHON}

In this article, I will show you how to scrape inspirational quotes from a website using the Python
programming language. I think everyone likes to hear some inspirational quotes from time to time
and hopefully the quotes that we will scrape within this article will brighten your day.

For anyone reading this article and don’t know what web scraping is, I will define it now. Web
scraping, or simply scraping is the act or process of extracting data from a website. This means you
will learn how to extract information from a website using Python.

Understanding the Concept Before Writing the Program


Before writing any code, we must first understand the concept and method for scraping the website.
We have to find a website that contains inspirational quotes. Once, that site is found, then we need
to understand how that website is structured to find and extract the quote data that we want.

1. Find A Website That Contains Inspirational Quotes


Like I said before, we need to find a website that contains the data that we want to extract. Luckily, I
was able to find a great website called goodreads.com. The link is
https://www.goodreads.com/quotes/tag/inspirational?page=0. This website contains inspirational
quotes, and the author who is being quoted.
2. View The Structure Of The Website
Now, we need to know how this website is structured. This can be done by using the inspection tool
on the Google Chrome Browser.

By using the inspection tool, I can see that the quotes appear to be under the div tag with class =
“quoteText”, and the author of the quote is under the span tag with class = “authorOrTitle” ,so I will
use this information to help me gather the data.
Furthermore, I can see that those two classes are under other div tags. So this information will help
me locate these two classes that contain the data that I want through the program. Also the site
itself has many pages with quotes, so this means I can iterate through each page to gather more
quotes simply by changing the page number at the end of the URL.

Example of iterating through the web page:


https://www.goodreads.com/quotes/tag/inspirational?page=0
https://www.goodreads.com/quotes/tag/inspirational?page=1

https://www.goodreads.com/quotes/tag/inspirational?page=2

If you prefer not to read this article and would like a video representation of it, you can check out the
YouTube Video . It goes through everything in this article with a little more detail, and will help make
it easy for you to start programming even if you don’t have the programming language Python
installed on your computer. Or you can use both as supplementary materials for learning.

Programming
First I will write a description about the program, this way I can simply read the description and know
what this program is about or is supposed to do.

#Description: Scrape Inspirational Quotes Using Python

Next, I want to import the libraries that are needed throughout the program

#Import the dependencies

from bs4 import BeautifulSoup


import pandas as pd

import requests

import urllib.request

import time

Now, create empty lists to store the inspirational quote and the author of the quote.

#Create lists to store the scraped data

authors = []

quotes = []

Time for the “meat” of the program. I will create a function to automatically scrape the quote and
the author of the quote and store that data into the empty lists created previously.

#Create a function to scrape the site

def scrape_website(page_number):

page_num = str(page_number) #Convert the page number to a string

URL = 'https://www.goodreads.com/quotes/tag/inspirational?page='+page_num #append the page


number to complete the URL

webpage = requests.get(URL) #Make a request to the website

soup = BeautifulSoup(webpage.text, "html.parser") #Parse the text from the website

quoteText = soup.find_all('div', attrs={'class':'quoteText'}) #Get the tag and it's class

for i in quoteText:

quote = i.text.strip().split('\n')[0]#Get the text of the current quote, but only the sentence before a
new line

author = i.find('span', attrs={'class':'authorOrTitle'}).text.strip()

#print(quote)

quotes.append(quote)

#print(author)

aLoop through ’n’ number of pages to scrape the quotes from.

#Loop through 'n' pages

n = 10

for num in range(0,n):

scrape_website(num)
Combine the two lists together.uthors.append(author)

#Combine the lists

combined_list = []

for i in range(len(quotes)):

combined_list.append(quotes[i]+'-'+authors[i])

Finally, time to show the inspirational quotes and the author of that quote!

#Show the combined list

combined_list

That’s it, you are done! Hopefully this was useful to you!

If you are interested in reading more on Python one of the fastest growing programming languages
that many companies and computer science departments use, then I recommend you check out the
book Learning Python written by Mark Lutz’s.

Conclusion
Thanks for reading this article I hope its helpful to you all! If you enjoyed this article and found it
helpful please leave a comment to show your appreciation. Keep up the learning, and if you like
machine learning, mathematics, computer science, programming or algorithm analysis, please visit
and subscribe to my YouTube channels (randerson112358 & computer science).

You might also like