Python Pandas - CategoricalDtype

---

---

Pandas CategoricalDtype

In Pandas, CategoricalDtype defines the data type for categorical data, specifying categories and their ordering. This data type can be useful when working with categorical data in Series, DataFrames, and various Pandas operations.

Using CategoricalDtype provides better control over categorical data by explicitly defining categories and their order. This can help reduce memory usage and improve performance when handling large datasets. In this tutorial, we will learn about CategoricalDtype and its structure, and practical examples.

CategoricalDtype Structure

A CategoricalDtype is fully described by −

• categories: A sequence of unique values without missing entries.

• ordered

  : A boolean indicating if the categories have an inherent order.

Creating CategoricalDtype

You can create a CategoricalDtype using the pandas.api.types.CategoricalDtype class. This class defines a custom data type for categorical data, allowing you to control categories and their order explicitly.

Following is the syntax for creating the CategoricalDtype in Pandas −

from pandas.api.types import CategoricalDtype
cat_type = CategoricalDtype(categories=None, ordered=False)

Here,

• categories: This parameter takes a sequence of unique, non-null values defining valid categories. It is stored as a Pandas index and if not provided, the dtype of that data index will be used.

• ordered: It takes a boolean value indicating whether the categories have an order. By default it is set to False.

Example: Applying CategoricalDtype to a Series

The following example demonstrates creating a Pandas Series object with the CategoricalDtype.

import pandas as pd
from pandas.api.types import CategoricalDtype

# Define custom CategoricalDtype
cat_type = CategoricalDtype(categories=["low", "medium", "high"], ordered=True)

# Create a Series with a defined categorical type
s = pd.Series(["low", "high", "medium", "low"], dtype=cat_type)

# Display the Series
print("Categorical Series:")
print(s)

Following is the output of the above code −

Categorical Series:
0       low
1      high
2    medium
3       low
dtype: category
Categories (3, object): ['low' < 'medium' < 'high']

Example: Applying CategoricalDtype to a DataFrame

The following example shows how to apply CategoricalDtype to a DataFrame column.

import pandas as pd
from pandas.api.types import CategoricalDtype

# Define custom CategoricalDtype
cat_type = CategoricalDtype(categories=["small", "medium", "large"], ordered=True)

# Create a DataFrame
df = pd.DataFrame({"Size": ["large", "small", "medium", "large"]})

# Convert column to CategoricalDtype
df["Size"] = df["Size"].astype(cat_type)

# Display the DataFrame
print("DataFrame with Categorical Data:")
print(df['Size'])

When we run above program, it produces following result −

DataFrame with Categorical Data:
0     large
1     small
2    medium
3     large
Name: Size, dtype: category
Categories (3, object): ['small' < 'medium' < 'large']

Usage of CategoricalDtype in Pandas

A CategoricalDtype can be used wherever pandas expects a dtype. such as −

• pandas.read_csv()

• DataFrame.astype()

• pandas.Series() constructor

Example: Using CategoricalDtype with DataFrame.astype()

This example shows using the CategoricalDtype with the Pandas DataFeam.astype() method for specifying the data type of a DataFrame column.

import pandas as pd
from pandas.api.types import CategoricalDtype

# Creating a DataFrame
data = {'col1': ["duck", "wolf", 'cat']}
df = pd.DataFrame(data)

# Convert column to CategoricalDtype
custom_dtype = CategoricalDtype(categories=["duck", "cat", "wolf"], ordered=True)
df['col1'] = df['col1'].astype(custom_dtype)

# Display the DataFrame
print("DataFrame with Categorical Data:")
print(df['col1'])

While executing the above code we get the following output −

DataFrame with Categorical Data:
0    duck
1    wolf
2     cat
Name: col1, dtype: category
Categories (3, object): ['duck' < 'cat' < 'wolf']

Example: Default String Representation

As a shortcut, you can also use the 'category' string representation as the dtype for CategoricalDtype(). This assumes default unordered categories inferred from the data.

This example uses the shortcut 'category' for applying categorical data type to the Pandas Series object.

import pandas as pd
from pandas.api.types import CategoricalDtype

# Create a Series with a defined categorical type
s = pd.Series(["low", "high", "medium", "low"], dtype='category')

# Display the Series
print("Categorical Series:")
print(s)

Following is the output of the above code −

Categorical Series:
0       low
1      high
2    medium
3       low
dtype: category
Categories (3, object): ['high', 'low', 'medium']

Comparing CategoricalDtype Instances

Instances of CategoricalDtype are equal if they have the same categories and order. When categories are unordered, their order does not matter.

Example

This example compares the ordered and unordered CategoricalDtype instance for showing the equality semantics of the categorical data type object.

import pandas as pd
from pandas.api.types import CategoricalDtype

c1 = CategoricalDtype(['a', 'b', 'c'], ordered=False)

# Unordered categories - order does not matter
result1 = (c1 == CategoricalDtype(['b', 'c', 'a'], ordered=False))
print("Equality of two unordered same categories:", result1)

# Ordered categories - different orders considered unequal
result2 = (c1 == CategoricalDtype(['a', 'b', 'c'], ordered=True))
print("Equality of ordered category with an unordered one:", result2)

# Comparison with 'category' shortcut
print(c1 == 'category')

When we run above program, it produces following result −

Equality of two unordered same categories: True
Equality of ordered category with an unordered one: False
True