Data Filtering in Pandas
Data Filtering in Pandas
Data Filtering in Pandas
yunusemrecevik
TABLE OF
CONTENTS
SINGLE CONDITION 2
MULTIPLE CONDITION 4
IN OPERATOR 6
LIKE OPERATOR 8
IS NULL OPERATOR 11
FILTER DATES 14
BASIC OPERATIONS 17
DATA
TABLE
ID NAME SURNAME COUNTRY AGE SALARY
pandas
1
SINGLE CONDITION
SQL
SELECT * FROM TABLE WHERE ID > 5
PANDAS
METHOD 1: DATAFRAME WAY
2
SINGLE CONDITION (NOT)
SQL
SELECT * FROM TABLE WHERE NOT ID > 5
PANDAS
METHOD 1: DATAFRAME WAY
bracket notation
3
MULTIPLE CONDITION (AND)
SQL
SELECT * FROM TABLE WHERE ID > 5 AND COUNTRY = 'CANADA'
PANDAS
METHOD 1: DATAFRAME WAY
Dot notation is faster to write and clearer to read. You can use whichever you like.
For more details:
https://www.dataschool.io/pandas-dot-notation-vs-brackets/
4
MULTIPLE CONDITION (AND & OR)
SQL
SELECT * FROM TABLE WHERE ID > 5 AND (COUNTRY = 'CANADA'
OR COUNTRY = 'USA')
PANDAS
METHOD 1: DATAFRAME WAY
you can use " & " for and operator you can use " | " for or operator
you can use "and" or "&" for and operator you can use "or" or "|" for and operator
5
IN OPERATOR
SQL
SELECT * FROM TABLE WHERE COUNTRY IN ('GERMANY', 'FRANCE')
PANDAS
6
NOT IN OPERATOR
SQL
SELECT * FROM TABLE WHERE COUNTRY NOT IN ('USA', 'FRANCE')
PANDAS
METHOD 1: DATAFRAME WAY
7
LIKE OPERATOR - startswith
SQL
SELECT * FROM TABLE WHERE SURNAME LIKE "WA%"
PANDAS
METHOD 1: DATAFRAME WAY
LIKE REGEX
str.match() determine if each string matches a regular expression.
For more detail :
https://pandas.pydata.org/docs/reference/api/pandas.Series.str.match.html
8
LIKE OPERATOR - endswith
SQL
SELECT * FROM TABLE WHERE SURNAME LIKE "%YAN"
PANDAS
METHOD 1: DATAFRAME WAY
9
LIKE OPERATOR - contains
SQL
SELECT * FROM TABLE WHERE NAME LIKE "%OH%"
PANDAS
10
IS NULL OPERATOR
SQL
SELECT * FROM TABLE WHERE COUNTRY IS NULL
11
IS NULL OPERATOR
PANDAS
12
IS NOT NULL OPERATOR
SQL
SELECT * FROM TABLE WHERE COUNTRY IS NOT NULL
PANDAS
METHOD 1: DATAFRAME WAY
you can use also notna() method
13
FILTER DATES
PANDAS
14
FILTER STRING DATES
SQL
SELECT * FROM TABLE WHERE MEMBER_DATE > '2015-01-01'
PANDAS
15
FILTER DATATIME DATES
SQL
SELECT * FROM TABLE WHERE YEAR(MEMBER_DATE) < 2005
PANDAS
16
BASIC OPERATIONS
SQL
SELECT * FROM TABLE WHERE SALARY / 20 > 12000
PANDAS
17
BASIC OPERATIONS
SQL
SELECT * FROM TABLE WHERE NAME > "JO"
PANDAS
18
BASIC OPERATIONS
SQL
SELECT * FROM TABLE WHERE SALARY * 2 > @VALUE
PANDAS
METHOD 1: DATAFRAME WAY
19
BASIC OPERATIONS
SQL
SELECT * FROM TABLE WHERE CONCAT(NAME, ' ' ,SURNAME) = 'PAUL
WAGNER'
PANDAS
METHOD 1: DATAFRAME WAY
20
DOWNLOAD PANDAS CODES
https://github.com/egitimplus/helpers/blob/main/notebook/filter.ipynb
FOLLOW ME ON
@yunusemrecevik
@yunusemrecevik
@EGITIMPLUS
21