Pandas apply function to column

Pandas apply function to column

Pandas is a powerful Python library used for data manipulation and analysis. One of its core features is the ability to apply functions to columns of data in a DataFrame. This capability is incredibly useful for data preprocessing, transformation, and analysis. In this article, we will explore various ways to apply functions to columns in a DataFrame using the apply, map, and applymap methods, along with vectorized operations.

Introduction to Pandas DataFrame

A DataFrame is a two-dimensional, size-mutable, potentially heterogeneous tabular data structure with labeled axes (rows and columns). Before diving into applying functions, let’s first understand how to create a DataFrame.

Example 1: Creating a DataFrame

import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)
print(df)

Output:

Applying Functions Using apply()

The apply() function is used to apply a function along an axis of the DataFrame or to a series of values.

Example 2: Applying a Function to a Column

import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

def add_domain(email):
    return email + "@pandasdataframe.com"

df['Email'] = df['Name'].apply(lambda x: add_domain(x.lower()))
print(df)

Output:

Using map() to Transform a Series

The map() function is used to map values of a Series from one value to another.

Example 3: Mapping with a Dictionary

import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

age_map = {25: 'Twenty Five', 30: 'Thirty', 35: 'Thirty Five'}
df['Age Description'] = df['Age'].map(age_map)
print(df)

Output:

Example 4: Using map() with a Function

import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

df['Name Length'] = df['Name'].map(len)
print(df)

Output:

Vectorized String Operations

Pandas provides vectorized string functions, which are very efficient for processing text data.

Example 5: Capitalizing Names

import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

df['Name'] = df['Name'].str.capitalize()
print(df)

Output:

Example 6: Finding Length of Each Name

import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

df['Name Length'] = df['Name'].str.len()
print(df)

Output:

Using applymap() for Element-wise Operations

The applymap() function is used to apply a function to each element of the DataFrame.

Example 7: Adding Suffix to Each Element

import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

df = df.applymap(lambda x: str(x) + "_pandasdataframe.com")
print(df)

Conditional Functions with apply()

Example 8: Applying Conditional Logic

import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

def check_age(age):
    if age > 30:
        return "Over 30"
    else:
        return "Under 30"

df['Age Group'] = df['Age'].apply(check_age)
print(df)

Output:

Example 9: Using np.where

import numpy as np
import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

df['Is Adult'] = np.where(df['Age'] >= 18, 'Yes', 'No')
print(df)

Output:

Performance Considerations

When working with large data sets, performance can become an issue. Vectorized operations are generally more efficient than applying functions using apply().

Example 10: Vectorized Operation vs apply()

import numpy as np
import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

# Vectorized operation
df['Age'] *= 2

# Using apply()
df['Age'] = df['Age'].apply(lambda x: x * 2)
print(df)

Output:

Advanced Function Application

Example 11: Using apply() with Additional Arguments

import numpy as np
import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

def custom_email(name, domain):
    return f"{name.lower()}@{domain}"

df['Custom Email'] = df['Name'].apply(custom_email, domain="pandasdataframe.com")
print(df)

Output:

Example 12: Applying Functions Row-wise

import numpy as np
import pandas as pd

data = {'Name': ['Alice', 'Bob', 'Charlie'],
        'Age': [25, 30, 35],
        'City': ['New York', 'Los Angeles', 'Chicago']}
df = pd.DataFrame(data)

def generate_username(row):
    return row['Name'].lower() + str(row['Age'])

df['Username'] = df.apply(generate_username, axis=1)
print(df)

Output:

Pandas apply function to column Conclusion

Applying functions to columns in Pandas is a versatile way to manipulate and analyze data. Whether you’re performing simple transformations or complex conditional logic, Pandas provides a range of methods to efficiently handle these operations. By mastering these techniques, you can significantly enhance your data analysis workflows.

This guide has covered a variety of methods and examples to help you understand how to apply functions to DataFrame columns effectively. With these tools, you can start to tackle more complex data processing tasks in your projects.