Pandas Append DataFrames

Pandas Append DataFrames

Pandas is a powerful Python library used for data manipulation and analysis. One of the common operations when working with data is appending or combining multiple DataFrame objects into a single DataFrame. This can be particularly useful when dealing with data from multiple sources or when performing batch data processing. In this article, we will explore various ways to append DataFrames using Pandas, along with detailed examples.

Understanding DataFrame Append Operation

Appending a DataFrame involves adding the rows of one DataFrame to another DataFrame. This operation can be performed using the append() method or the concat() function in Pandas. While append() is a specific method for adding rows, concat() provides more flexibility, allowing concatenation along either the rows or the columns axis.

Example 1: Basic DataFrame Append

import pandas as pd

# Create two DataFrames
df1 = pd.DataFrame({
    'URL': ['pandasdataframe.com/1', 'pandasdataframe.com/2'],
    'Data': [10, 20]
})

df2 = pd.DataFrame({
    'URL': ['pandasdataframe.com/3', 'pandasdataframe.com/4'],
    'Data': [30, 40]
})

# Append df2 to df1
result = df1._append(df2, ignore_index=True)
print(result)

Output:

Example 2: Append with Different Columns

import pandas as pd

# Create two DataFrames with different columns
df1 = pd.DataFrame({
    'URL': ['pandasdataframe.com/1', 'pandasdataframe.com/2'],
    'Data1': [10, 20]
})

df2 = pd.DataFrame({
    'URL': ['pandasdataframe.com/3', 'pandasdataframe.com/4'],
    'Data2': [30, 40]
})

# Append df2 to df1
result = df1._append(df2, ignore_index=True, sort=False)
print(result)

Output:

Example 3: Using Concat Instead of Append

import pandas as pd

# Create two DataFrames
df1 = pd.DataFrame({
    'URL': ['pandasdataframe.com/1', 'pandasdataframe.com/2'],
    'Data': [10, 20]
})

df2 = pd.DataFrame({
    'URL': ['pandasdataframe.com/3', 'pandasdataframe.com/4'],
    'Data': [30, 40]
})

# Use concat to append df2 to df1
result = pd.concat([df1, df2], ignore_index=True)
print(result)

Output:

Example 4: Append Multiple DataFrames

import pandas as pd

# Create multiple DataFrames
df1 = pd.DataFrame({
    'URL': ['pandasdataframe.com/1', 'pandasdataframe.com/2'],
    'Data': [10, 20]
})

df2 = pd.DataFrame({
    'URL': ['pandasdataframe.com/3', 'pandasdataframe.com/4'],
    'Data': [30, 40]
})

df3 = pd.DataFrame({
    'URL': ['pandasdataframe.com/5', 'pandasdataframe.com/6'],
    'Data': [50, 60]
})

# Append all DataFrames
result = df1._append([df2, df3], ignore_index=True)
print(result)

Output:

Example 5: Append with Different Indexes

import pandas as pd

# Create two DataFrames with different indexes
df1 = pd.DataFrame({
    'URL': ['pandasdataframe.com/1', 'pandasdataframe.com/2'],
    'Data': [10, 20]
}, index=[1, 2])

df2 = pd.DataFrame({
    'URL': ['pandasdataframe.com/3', 'pandasdataframe.com/4'],
    'Data': [30, 40]
}, index=[3, 4])

# Append df2 to df1 without resetting index
result = df1._append(df2)
print(result)

Output:

Example 6: Append with a Series

import pandas as pd

# Create a DataFrame and a Series
df = pd.DataFrame({
    'URL': ['pandasdataframe.com/1', 'pandasdataframe.com/2'],
    'Data': [10, 20]
})

series = pd.Series(['pandasdataframe.com/3', 30], index=['URL', 'Data'])

# Append the Series as a row to the DataFrame
result = df._append(series, ignore_index=True)
print(result)

Output:

Example 7: Append DataFrames with MultiIndex

import pandas as pd

# Create two Dataframes with MultiIndex
index = pd.MultiIndex.from_tuples([('A', 1), ('A', 2)], names=['Letter', 'Number'])
df1 = pd.DataFrame({
    'URL': ['pandasdataframe.com/1', 'pandasdataframe.com/2'],
    'Data': [10, 20]
}, index=index)

index2 = pd.MultiIndex.from_tuples([('B', 1), ('B', 2)], names=['Letter', 'Number'])
df2 = pd.DataFrame({
    'URL': ['pandasdataframe.com/3', 'pandasdataframe.com/4'],
    'Data': [30, 40]
}, index=index2)

# Append df2 to df1
result = df1._append(df2)
print(result)

Output:

Example 8: Append with Non-Aligning Columns

import pandas as pd

# Create two DataFrames with non-aligning columns
df1 = pd.DataFrame({
    'URL': ['pandasdataframe.com/1', 'pandasdataframe.com/2'],
    'Data1': [10, 20]
})

df2 = pd.DataFrame({
    'URL': ['pandasdataframe.com/3', 'pandasdataframe.com/4'],
    'Data2': [30, 40],
    'Extra': ['A', 'B']
})

# Append df2 to df1 with sorting columns
result = df1._append(df2, ignore_index=True, sort=True)
print(result)

Output:

Example 9: Append with Different DataTypes

import pandas as pd

# Create two DataFrames with different data types
df1 = pd.DataFrame({
    'URL': ['pandasdataframe.com/1', 'pandasdataframe.com/2'],
    'Data': [10, 20]
})

df2 = pd.DataFrame({
    'URL': ['pandasdataframe.com/3', 'pandasdataframe.com/4'],
    'Data': ['Thirty', 'Forty']
})

# Append df2 to df1
result = df1._append(df2, ignore_index=True)
print(result)

Output:

Example 10: Using Append in a Loop

import pandas as pd

# Initialize an empty DataFrame
result = pd.DataFrame()

# Loop to append multiple DataFrames
for i in range(5):
    df = pd.DataFrame({
        'URL': [f'pandasdataframe.com/{i+1}'],
        'Data': [i * 10]
    })
    result = result.append(df, ignore_index=True)

print(result)

Pandas Append DataFrames Conclusion

Appending DataFrames is a fundamental operation in data manipulation with Pandas. Whether you are combining data from different sources, aggregating results, or simply organizing scattered data, understanding how to effectively append DataFrames is crucial. The examples provided in this article demonstrate various scenarios and methods to append DataFrames, giving you a solid foundation for performing these operations in your data projects.