SQL For Data Science
SQL For Data Science
com/assessments/1601018393668/b7630d28-d7af-4e58-8391-f18a8465d4a5…
Data Scientist Role Play: Profiling and Analyzing the Yelp Dataset Coursera Worksheet
This is a 2-part assignment. In the first part, you are asked a series of questions that will help
you profile and understand the data just like a data scientist would. For this first part of the
assignment, you will be assessed both on the correctness of your findings, as well as the code you
used to arrive at your answer. You will be graded on how easy your code is to read, so remember to
use proper formatting and comments where necessary.
In the second part of the assignment, you are asked to come up with your own inferences and
analysis of the data for a particular research question you want to answer. You will be required
to prepare the dataset for the analysis you choose to do. As with the first part, you will be
graded, in part, on how easy your code is to read, so use proper formatting and comments to
illustrate and communicate your intent as required.
For both parts of this assignment, use this "worksheet." It provides all the questions you are
being asked, and your job will be to transfer your answers and SQL coding where indicated into
this worksheet so that your peers can review your work. You should be able to use any Text Editor
(Windows Notepad, Apple TextEdit, Notepad ++, Sublime Text, etc.) to copy and paste your answers.
If you are going to use Word or some other page layout application, just be careful to make sure
your answers and code are lined appropriately.
In this case, you may want to save as a PDF to ensure your formatting remains intact for you
reviewer.
1. Profile the data by finding the total number of records for each of the tables below:
SELECT COUNT(*)
FROM table
2. Find the total distinct records by either the foreign key or primary key for each table. If two
foreign keys are listed in the table, please specify which foreign key.
SELECT COUNT(DISTINCT(key))
FROM table
i. Business =10000
ii. Hours =business_id: 1562
iii. Category =business_id: 2643
iv. Attribute =business_id: 1115
v. Review =id:10000, business_id: 8090
vi. Checkin = business_id: 493
vii. Photo =id: 10000, business_id: 6493
viii. Tip = user_id: 537, business_id: 3979
ix. User = id: 10000
x. Friend = user_id: 11
xi. Elite_years =user_id: 2780
Note: Primary Keys are denoted in the ER-Diagram with a yellow key icon.
3. Are there any columns with null values in the Users table? Indicate "yes," or "no."
Answer:No
4. For each table and column listed below, display the smallest (minimum), largest (maximum), and
average (mean) value for the following fields:
SELECT AVG(column)
FROM table
i. Table: Review, Column: Stars
6. Find the distribution of star ratings to the business in the following cities:
i. Avon
SELECT stars,
SUM(review_count) AS count
FROM business
WHERE city == 'Avon'
GROUP BY stars
Copy and Paste the Resulting Table Below (2 columns - star rating and count):
+-------+-------+
| stars | count |
+-------+-------+
| 1.5 | 10 |
| 2.5 | 6 |
| 3.5 | 88 |
| 4.0 | 21 |
| 4.5 | 31 |
| 5.0 | 3 |
+-------+-------+
ii. Beachwood
SELECT stars,
SUM(review_count) AS count
FROM business
WHERE city == 'Beachwood'
GROUP BY stars
Copy and Paste the Resulting Table Below (2 columns - star rating and count):
+-------+-------+
| stars | count |
+-------+-------+
| 2.0 | 8 |
| 2.5 | 3 |
| 3.0 | 11 |
| 3.5 | 6 |
| 4.0 | 69 |
| 4.5 | 17 |
| 5.0 | 23 |
+-------+-------+
SELECT id,
name,
review_count
FROM user
ORDER BY review_count DESC
LIMIT 3
+------------------------+--------+--------------+
| id | name | review_count |
+------------------------+--------+--------------+
| -G7Zkl1wIWBBmD0KRy_sCw | Gerald | 2000 |
| -3s52C4zL_DHRK0ULG6qtg | Sara | 1629 |
| -8lbUNlXVSoXqaRRiHiSNg | Yuri | 1339 |
+------------------------+--------+--------------+
Yes, but also the amount of time that they have been yelping. The longer they
have been yelping and the more reviews they give has a higher fan count.
SELECT id,
name,
review_count,
fans,
yelping_since
FROM user
ORDER BY fans DESC
+------------------------+-----------+--------------+------+---------------------+
| id | name | review_count | fans | yelping_since |
+------------------------+-----------+--------------+------+---------------------+
| -9I98YbNQnLdAmcYfb324Q | Amy | 609 | 503 | 2007-07-19 00:00:00 |
| -8EnCioUmDygAbsYZmTeRQ | Mimi | 968 | 497 | 2011-03-30 00:00:00 |
| --2vR0DIsmQ6WfcSzKWigw | Harald | 1153 | 311 | 2012-11-27 00:00:00 |
| -G7Zkl1wIWBBmD0KRy_sCw | Gerald | 2000 | 253 | 2012-12-16 00:00:00 |
| -0IiMAZI2SsQ7VmyzJjokQ | Christine | 930 | 173 | 2009-07-08 00:00:00 |
| -g3XIcCb2b-BD0QBCcq2Sw | Lisa | 813 | 159 | 2009-10-05 00:00:00 |
| -9bbDysuiWeo2VShFJJtcw | Cat | 377 | 133 | 2009-02-05 00:00:00 |
| -FZBTkAZEXoP7CYvRV2ZwQ | William | 1215 | 126 | 2015-02-19 00:00:00 |
| -9da1xk7zgnnfO1uTVYGkA | Fran | 862 | 124 | 2012-04-05 00:00:00 |
| -lh59ko3dxChBSZ9U7LfUw | Lissa | 834 | 120 | 2007-08-14 00:00:00 |
| -B-QEUESGWHPE_889WJaeg | Mark | 861 | 115 | 2009-05-31 00:00:00 |
| -DmqnhW4Omr3YhmnigaqHg | Tiffany | 408 | 111 | 2008-10-28 00:00:00 |
| -cv9PPT7IHux7XUc9dOpkg | bernice | 255 | 105 | 2007-08-29 00:00:00 |
| -DFCC64NXgqrxlO8aLU5rg | Roanna | 1039 | 104 | 2006-03-28 00:00:00 |
9. Are there more reviews with the word "love" or with the word "hate" in them?
Answer: love has 1780, while hate only has 232 :) 'love prevails'
SELECT COUNT(*)
SELECT COUNT(*)
FROM review
FROM review
WHERE text LIKE "%love%" WHERE text
LIKE "%hate%"
= 1780
SELECT id,
name,
fans
FROM user
ORDER BY fans DESC
LIMIT 10
+------------------------+-----------+------+
| id | name | fans |
+------------------------+-----------+------+
| -9I98YbNQnLdAmcYfb324Q | Amy | 503 |
| -8EnCioUmDygAbsYZmTeRQ | Mimi | 497 |
| --2vR0DIsmQ6WfcSzKWigw | Harald | 311 |
| -G7Zkl1wIWBBmD0KRy_sCw | Gerald | 253 |
| -0IiMAZI2SsQ7VmyzJjokQ | Christine | 173 |
| -g3XIcCb2b-BD0QBCcq2Sw | Lisa | 159 |
| -9bbDysuiWeo2VShFJJtcw | Cat | 133 |
| -FZBTkAZEXoP7CYvRV2ZwQ | William | 126 |
| -9da1xk7zgnnfO1uTVYGkA | Fran | 124 |
| -lh59ko3dxChBSZ9U7LfUw | Lissa | 120 |
+------------------------+-----------+------+
i. Do the two groups you chose to analyze have a different distribution of hours?
The 4-5 star group seems to have shorter hours then the 2-3 star group.
Please note the query returned only three businesses so not a great
sample size.
ii. Do the two groups you chose to analyze have a different number of reviews?
Yes and no, one of the 4-5 star group has a lot more reviews but then the other
4-5 star group has close to the same number of reviews as the 2-3 star group
iii. Are you able to infer anything from the location data provided between these two
groups? Explain.
SELECT B.name,
B.review_count,
H.hours,
postal_code,
CASE
WHEN hours LIKE "%monday%" THEN 1
WHEN hours LIKE "%tuesday%" THEN 2
WHEN hours LIKE "%wednesday%" THEN 3
WHEN hours LIKE "%thursday%" THEN 4
WHEN hours LIKE "%friday%" THEN 5
WHEN hours LIKE "%saturday%" THEN 6
WHEN hours LIKE "%sunday%" THEN 7
END AS ord,
CASE
WHEN B.stars BETWEEN 2 AND 3 THEN '2-3 stars'
WHEN B.stars BETWEEN 4 AND 5 THEN '4-5 stars'
END AS star_rating
FROM business B INNER JOIN hours H
ON B.id = H.business_id
INNER JOIN category C
ON C.business_id = B.id
WHERE (B.city == 'Las Vegas'
AND
C.category LIKE 'shopping')
AND
(B.stars BETWEEN 2 AND 3
OR
B.stars BETWEEN 4 AND 5)
GROUP BY stars,ord
ORDER BY ord,star_rating ASC
2. Group business based on the ones that are open and the ones that are closed. What differences
can you find between the ones that are still open and the ones that are closed? List at least two
differences and the SQL code you used to arrive at your answer.
i. Difference 1:
The businesses that are open tend to have more reviews than ones that
are closed on average.
ii. Difference 2:
The average star rating is higher for businesses that are open than
businesses that are closed.
SELECT COUNT(DISTINCT(id)),
AVG(review_count),
SUM(review_count),
AVG(stars),
is_open
FROM business
GROUP BY is_open
3. For this last part of your analysis, you are going to choose the type of analysis you want to
conduct on the Yelp dataset and are going to prepare the data for analysis.
Ideas for analysis include: Parsing out keywords and business attributes for sentiment analysis,
clustering businesses to find commonalities or anomalies between them, predicting the overall star
rating for a business, predicting the number of fans a user will have, and so on. These are just a
few examples to get you started, so feel free to be creative and come up with your own problem you
want to solve. Provide answers, in-line, to all of the following:
Predicting whether a business will stay open or close. We wish not to explicitly
examine the text of the reviews, but this would be an interesting analysis.
ii. Write 1-2 brief paragraphs on the type of data you will need for your analysis
and why you chose that data:
+------------------------+--------------------------------+-----------------------
------+---------------+-------+-------------+----------+-----------+--------------+-------+-------
-------+---------------+-----------------+----------------+--------------+----------------+-------
-------+------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------
----------------+---------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------
------------------------------------------------+---------+
| id | name | address
| city | state | postal_code | latitude | longitude | review_count | stars | monday_hours
| tuesday_hours | wednesday_hours | thursday_hours | friday_hours | saturday_hours | sunday_hours
| categories
| attributes
| is_open |
+------------------------+--------------------------------+-----------------------
------+---------------+-------+-------------+----------+-----------+--------------+-------+-------
-------+---------------+-----------------+----------------+--------------+----------------+-------
-------+------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------
----------------+---------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------
------------------------------------------------+---------+
| -0DET7VdEQOJVJ_v6klEug | Flaming Kitchen | 3235 York Regional
| BusinessAcceptsCreditCards,RestaurantsPriceRange2,BusinessParking,BikeParking
| 1 |
| 0-aPEeNc2zVb5Gp-i7Ckqg | Buddy's Muffler & Exhaust | 1509 Hickory Grove Rd
| Gastonia | NC | 28056 | 35.2772 | -81.06 | 4 | 5.0 | 8:30-17:00
| 8:30-17:00 | 8:30-17:00 | 8:30-17:00 | 8:30-17:00 | 9:00-15:00 | None
| Automotive,Auto Repair
| BusinessAcceptsCreditCards
| 1 |
| 01xXe2m_z048W5gcBFpoJA | Five Guys | 2641 N 44th St, Ste
100 | Phoenix | AZ | 85008 | 33.478 | -111.986 | 63 | 3.5 |
10:00-22:00 | 10:00-22:00 | 10:00-22:00 | 10:00-22:00 | 10:00-22:00 | 10:00-22:00 |
10:00-22:00 | American (New),Burgers,Fast Food,Restaurants
|
RestaurantsTableService,GoodForMeal,Alcohol,Caters,HasTV,RestaurantsGoodForGroups,NoiseLevel,WiFi,
RestaurantsAttire,RestaurantsReservations,OutdoorSeating,BusinessAcceptsCreditCards,RestaurantsPri
ceRange2,BikeParking,RestaurantsDelivery,Ambience,RestaurantsTakeOut,GoodForKids,DriveThru,Busines
sParking | 1 |
| 06I2r8S3tHP_LwGnnkk6Uw | All Storage - Anthem | 2620 W Horizon Ridge
Pkwy | Henderson | NV | 89052 | 36.0021 | -115.102 | 3 | 3.5 | 9:00-
16:30 | 9:00-16:30 | 9:00-16:30 | 9:00-16:30 | 9:00-16:30 | 9:00-16:30 | None
| Truck Rental,Local Services,Self Storage,Parking,Automotive
| BusinessAcceptsCreditCards,BusinessAcceptsBitcoin
| 1 |
| 07h3mGtTovPJE660nX6E-A | Mood | 1 Greenside Place
| Edinburgh | EDH | EH1 3AA | 55.957 | -3.18502 | 11 | 2.0 | None
| None | None | 22:30-3:00 | 22:00-3:00 | 22:00-3:00 | 22:30-3:00
| Dance Clubs,Nightlife
|
Alcohol,OutdoorSeating,BusinessAcceptsCreditCards,RestaurantsPriceRange2,AgesAllowed,Music,Smoking
,RestaurantsGoodForGroups,WheelchairAccessible
| 0 |
| 0AJF-USLN6K5T4caooDdjw | Starbucks | 4605 E Chandler Blvd,
Ste A | Phoenix | AZ | 85048 | 33.3044 | -111.984 | 52 | 3.0 | 5:00-
20:00 | 5:00-20:00 | 5:00-20:00 | 5:00-20:30 | 5:00-20:00 | 5:00-20:00 | 5:00-
20:00 | Coffee & Tea,Food
|
BusinessParking,Caters,WiFi,OutdoorSeating,BusinessAcceptsCreditCards,RestaurantsPriceRange2,BikeP
arking,RestaurantsTakeOut
| 1 |
| 0B3W6KxkD3o4W4l6cq735w | Big Smoke Burger | 260 Yonge Street
| Toronto | ON | M4B 2L9 | 43.6546 | -79.3805 | 47 | 3.0 | 10:30-21:00
| 10:30-21:00 | 10:30-21:00 | 10:30-21:00 | 10:30-21:00 | 10:30-21:00 | 11:00-19:00
| Poutineries,Burgers,Restaurants
|
RestaurantsTableService,GoodForMeal,Alcohol,Caters,HasTV,RestaurantsGoodForGroups,NoiseLevel,WiFi,
RestaurantsAttire,RestaurantsReservations,OutdoorSeating,BusinessAcceptsCreditCards,RestaurantsPri
ceRange2,WheelchairAccessible,BikeParking,RestaurantsDelivery,Ambience,RestaurantsTakeOut,GoodForK
ids,DriveThru,BusinessParking | 1 |
| 0IySwcfqwJjpHPsYwjpAkg | Subway | 2904 Yorkmont Rd
| Charlotte | NC | 28208 | 35.1903 | -80.9288 | 7 | 3.5 | 6:00-22:00
| 6:00-22:00 | 6:00-22:00 | 6:00-22:00 | 6:00-22:00 | 10:00-21:00 | None
| Fast Food,Restaurants,Sandwiches
| Ambience,RestaurantsPriceRange2,GoodForKids
| 1 |
| 0K2rKvqdBmiOAUTebcUohQ | Red Rock Canyon Visitor Center | 1000 Scenic Loop Dr
| Las Vegas | NV | 89161 | 36.1357 | -115.428 | 32 | 4.5 | 8:00-16:30
| 8:00-16:30 | 8:00-16:30 | 8:00-16:30 | 8:00-16:30 | 8:00-16:30 | 8:00-16:30
| Education,Visitor Centers,Professional Services,Special Education,Local Services,Community
Service/Non-Profit,Hotels & Travel,Travel Services,Gift Shops,Shopping,Parks,Hiking,Flowers &
Gifts,Active Life | BusinessAcceptsCreditCards,GoodForKids
| 1 |
| 0Ni7Stqt4RFWDGjOYRi2Bw | Scent From Above Company | 2501 W Behrend Dr, Ste
67 | Scottsdale | AZ | 85027 | 33.6656 | -112.111 | 14 | 4.5 | 6:00-
16:00 | 6:00-16:00 | 6:00-16:00 | 6:00-16:00 | 6:00-16:00 | None | None
| Home Cleaning,Local Services,Professional Services,Carpet Cleaning,Home Services,Office
Cleaning,Window Washing
| BusinessAcceptsCreditCards,ByAppointmentOnly
| 1 |
| 0WBMEfqXQnEOAIkV-uCW6w | The Charlotte Room | 19 Charlotte Street
iv. Provide the SQL code you used to create your final dataset:
SELECT B.id,
B.name,
B.address,
B.city,
B.state,
B.postal_code,
B.latitude,
B.longitude,
B.review_count,
B.stars,
MAX(CASE
WHEN H.hours LIKE "%monday%" THEN
TRIM(H.hours,'%MondayTuesWednesThursFriSatSun|%')
END) AS monday_hours,
MAX(CASE
WHEN H.hours LIKE "%tuesday%" THEN
TRIM(H.hours,'%MondayTuesWednesThursFriSatSun|%')
END) AS tuesday_hours,
MAX(CASE
WHEN H.hours LIKE "%wednesday%" THEN
TRIM(H.hours,'%MondayTuesWednesThursFriSatSun|%')
END) AS wednesday_hours,
MAX(CASE
WHEN H.hours LIKE "%thursday%" THEN
TRIM(H.hours,'%MondayTuesWednesThursFriSatSun|%')
END) AS thursday_hours,
MAX(CASE
WHEN H.hours LIKE "%friday%" THEN
TRIM(H.hours,'%MondayTuesWednesThursFriSatSun|%')
END) AS friday_hours,
MAX(CASE
WHEN H.hours LIKE "%saturday%" THEN
TRIM(H.hours,'%MondayTuesWednesThursFriSatSun|%')
END) AS saturday_hours,
MAX(CASE
WHEN H.hours LIKE "%sunday%" THEN
TRIM(H.hours,'%MondayTuesWednesThursFriSatSun|%')
END) AS sunday_hours,
GROUP_CONCAT(DISTINCT(C.category)) AS categories,
GROUP_CONCAT(DISTINCT(A.name)) AS attributes,
B.is_open
FROM business B
INNER JOIN hours H
ON B.id = H.business_id
INNER JOIN category C
ON B.id = C.business_id
INNER JOIN attribute A
ON B.id = A.business_id
GROUP BY B.id