Regular Expressions in Python
The term Regular Expression is popularly shortened as regex. A regex is a sequence of characters that defines a search pattern, used mainly for performing find and replace operations in search engines and text processors.
Python offers regex capabilities through the re module bundled as a part of the standard library.
Raw strings
Different functions in Python's re module use raw string as an argument. A normal string, when prefixed with 'r' or 'R' becomes a raw string.
rawstr = r'Hello! How are you?'
print(rawstr) #Hello! How are you?The difference between a normal string and a raw string is that the normal string in print() function translates escape characters (such as \n, \t etc.) if any, while those in a raw string are not.
str1 = "Hello!
How are you?"
print("normal string:", str1)
str2 = r"Hello!
How are you?"
print("raw string:",str2)normal string: Hello! How are you? raw string: Hello!\nHow are you?In the above example, \n inside str1 (normal string) has translated as a newline being printed in the next line. But, it is printed as \n in str2 - a raw string.
meta characters
Some characters carry a special meaning when they appear as a part pattern matching string. In Windows or Linux DOS commands, we use * and ? - they are similar to meta characters. Python's re module uses the following characters as meta characters:
. ^ $ * + ? [ ] \ | ( )
When a set of alpha-numeric characters are placed inside square brackets [], the target string is matched with these characters. A range of characters or individual characters can be listed in the square bracket. For example:
| Pattern | Description |
|---|---|
| [abc] | match any of the characters a, b, or c |
| [a-c] | which uses a range to express the same set of characters. |
| [a-z] | match only lowercase letters. |
| [0-9] | match only digits. |
The following specific characters carry certain specific meaning.
| Pattern | Description |
|---|---|
| \d | Matches any decimal digit; this is equivalent to the class [0-9]. |
| \D | Matches any non-digit character |
| \s | Matches any whitespace character |
| \S | Matches any non-whitespace character |
| \w | Matches any alphanumeric character |
| \W | Matches any non-alphanumeric character. |
| . | Matches with any single character except newline ‘\n'. |
| ? | match 0 or 1 occurrence of the pattern to its left |
| + | 1 or more occurrences of the pattern to its left |
| * | 0 or more occurrences of the pattern to its left |
| \b | boundary between word and non-word. /B is opposite of /b |
| [..] | Matches any single character in a square bracket |
| \ | It is used for special meaning characters like . to match a period or + for plus sign. |
| {n,m} | Matches at least n and at most m occurrences of preceding |
| a| b | Matches either a or b |
re.match() function
This function in re module tries to find if the specified pattern is present at the beginning of the given string.
re.match(pattern, string)The function returns None, if the given pattern is not in the beginning, and a match objects if found.
from re import match
mystr = "Welcome to TutorialsTeacher"
obj1 = match("We", mystr)
print(obj1)
obj2 = match("teacher", mystr)
print(obj2)
print("start:", obj1.start(), "end:", obj1.end())The match object has start and end properties.
<re.Match object; span=(0, 2), match='We'> None start: 0 end: 2The following example demonstrates the use of the range of characters to find out if a string starts with 'W' and is followed by an alphabet.
from re import match
strings=["Welcome to TutorialsTeacher", "weather forecast","Winston Churchill", "W.G.Grace","Wonders of India", "Water park"]
for string in strings:
obj = match("W[a-z]",string)
print(obj)<re.Match object; span=(0, 2), match='We'> None <re.Match object; span=(0, 2), match='Wi'> None <re.Match object; span=(0, 2), match='Wo'> <re.Match object; span=(0, 2), match='Wa'>re.search() function
The re.search() function searches for a specified pattern anywhere in the given string and stops the search on the first occurrence.
from re import search
string = "Try to earn while you learn"
obj = search("earn", string)
print(obj)
print(obj.start(), obj.end(), obj.group())<re.Match object; span=(7, 11), match='earn'> 7 11 earnThis function also returns the Match object with start and end attributes. It also gives a group of characters of which the pattern is a part of.
re.findall() Function
As against the search() function, the findall() continues to search for the pattern till the target string is exhausted. The object returns a list of all occurrences.
from re import findall
string = "Try to earn while you learn"
obj = findall("earn", string)
print(obj)['earn', 'earn']This function can be used to get the list of words in a sentence. We shall use \W* pattern for the purpose. We also check which of the words do not have any vowels in them.
from re import findall, search
obj = findall(r"w*", "Fly in the sky.")
print(obj)
for word in obj:
obj = search(r"[aeiou]",word)
if word!='' and obj==None:
print(word)['Fly', '', 'in', '', 'the', '', 'sky', '', ''] Fly skyre.finditer() function
The re.finditer() function returns an iterator object of all matches in the target string. For each matched group, start and end positions can be obtained by span() attribute.
from re import finditer
string = "Try to earn while you learn"
it = finditer("earn", string)
for match in it:
print(match.span())(7, 11) (23, 27)re.split() function
The re.split() function works similar to the split() method of str object in Python. It splits the given string every time a white space is found. In the above example of the findall() to get all words, the list also contains each occurrence of white space as a word. That is eliminated by the split() function in re module.
from re import split
string = "Flat is better than nested. Sparse is better than dense."
words = split(r' ', string)
print(words)['Flat', 'is', 'better', 'than', 'nested.', 'Sparse', 'is', 'better', 'than', 'dense.']re.compile() Function
The re.compile() function returns a pattern object which can be repeatedly used in different regex functions. In the following example, a string 'is' is compiled to get a pattern object and is subjected to the search() method.
from re import *
pattern = compile(r'[aeiou]')
string = "Flat is better than nested. Sparse is better than dense."
words = split(r' ', string)
for word in words:
print(word, pattern.match(word))Flat None is <re.Match object; span=(0, 1), match='i'> better None than None nested. None Sparse None is <re.Match object; span=(0, 1), match='i'> better None than None dense. NoneThe same pattern object can be reused in searching for words having vowels, as shown below.
for word in words:
print(word, pattern.search(word))Flat <re.Match object; span=(2, 3), match='a'> is <re.Match object; span=(0, 1), match='i'> better <re.Match object; span=(1, 2), match='e'> than <re.Match object; span=(2, 3), match='a'> nested. <re.Match object; span=(1, 2), match='e'> Sparse <re.Match object; span=(2, 3), match='a'> is <re.Match object; span=(0, 1), match='i'> better <re.Match object; span=(1, 2), match='e'> than <re.Match object; span=(2, 3), match='a'> dense. <re.Match object; span=(1, 2), match='e'>