This page includes an interactive code editor. Try modifying and running the examples!

Pandas GroupBy Operations

The GroupBy operation is one of the most powerful features in Pandas, enabling you to split data into groups, apply functions to each group, and combine the results. It's essential for aggregation, transformation, and filtering operations.

1. The GroupBy Process: Split-Apply-Combine

GroupBy follows a three-step process:

Split: Divide the data into groups based on specified criteria
Apply: Apply a function to each group independently
Combine: Combine the results into a new data structure

Visualization: DataFrame → Split by Group → Apply Function → Combine Results

2. Basic GroupBy Syntax and Methods

Method	Description	Example
groupby()	Create GroupBy object	`df.groupby('column')`
sum()	Sum of each group	`grouped.sum()`
mean()	Average of each group	`grouped.mean()`
count()	Count of elements	`grouped.count()`
agg()	Multiple aggregations	`grouped.agg(['sum', 'mean'])`

3. Common Aggregation Functions

sum() - Sum of values
mean() - Arithmetic mean
median() - Median value
std() - Standard deviation
var() - Variance

min() - Minimum value
max() - Maximum value
count() - Count of non-NA values
size() - Size of group
first()/last() - First/last value

Basic GroupBy Examples

Basic GroupBy Operations

import pandas as pd
import numpy as np

# Create sample sales data
data = {
    'Region': ['North', 'South', 'North', 'East', 'West', 'South', 'East', 'West', 'North', 'South'],
    'Product': ['A', 'B', 'A', 'C', 'B', 'A', 'C', 'A', 'B', 'C'],
    'Sales': [1000, 1500, 1200, 800, 2000, 1300, 900, 1100, 1600, 700],
    'Quantity': [10, 15, 12, 8, 20, 13, 9, 11, 16, 7],
    'Month': ['Jan', 'Jan', 'Feb', 'Feb', 'Mar', 'Mar', 'Apr', 'Apr', 'May', 'May']
}

df = pd.DataFrame(data)
print("Original DataFrame:")
print(df)

# Basic GroupBy: Total sales per region
print("\n1. Total Sales by Region:")
region_sales = df.groupby('Region')['Sales'].sum()
print(region_sales)

# Multiple aggregations
print("\n2. Multiple Aggregations by Region:")
region_stats = df.groupby('Region').agg({
    'Sales': ['sum', 'mean', 'max', 'min'],
    'Quantity': ['sum', 'mean']
})
print(region_stats)

# Group by multiple columns
print("\n3. Sales by Region and Product:")
region_product = df.groupby(['Region', 'Product'])['Sales'].sum()
print(region_product)

# Using custom aggregation functions
print("\n4. Custom Aggregations:")
def sales_range(series):
    return series.max() - series.min()

custom_agg = df.groupby('Region').agg({
    'Sales': ['sum', sales_range, 'std'],
    'Quantity': 'mean'
})
print(custom_agg)

4. Advanced GroupBy Techniques

Technique	Description	Use Case
transform()	Return object with group values broadcasted	Adding group-wise statistics to original data
filter()	Filter groups based on conditions	Selecting groups that meet specific criteria
apply()	Apply custom function to each group	Complex group-wise operations
Multiple Columns	Group by multiple columns	Hierarchical grouping analysis

Advanced GroupBy Operations

Advanced GroupBy Techniques

import pandas as pd

# Advanced GroupBy operations
data = {
    'Department': ['IT', 'HR', 'Finance', 'IT', 'HR', 'Finance', 'IT', 'HR'],
    'Employee': ['Alice', 'Bob', 'Charlie', 'David', 'Eve', 'Frank', 'Grace', 'Henry'],
    'Salary': [70000, 50000, 60000, 80000, 55000, 65000, 75000, 52000],
    'Experience': [3, 2, 5, 4, 1, 6, 3, 2],
    'Bonus': [5000, 2000, 4000, 6000, 1500, 5000, 4500, 1800]
}

df = pd.DataFrame(data)
print("Employee DataFrame:")
print(df)

# Transform: Add group-wise statistics to original data
print("\n1. Transform - Add department average salary:")
df['Dept_Avg_Salary'] = df.groupby('Department')['Salary'].transform('mean')
df['Salary_vs_Avg'] = df['Salary'] - df['Dept_Avg_Salary']
print(df)

# Filter: Filter groups based on conditions
print("\n2. Filter - Departments with total salary > 150000:")
def high_salary_dept(group):
    return group['Salary'].sum() > 150000

filtered_depts = df.groupby('Department').filter(high_salary_dept)
print(filtered_depts)

# Apply: Custom function to each group
print("\n3. Apply - Top earner in each department:")
def top_earner(group):
    return group.nlargest(1, 'Salary')

top_earners = df.groupby('Department').apply(top_earner)
print(top_earners)

# Pivot tables vs GroupBy
print("\n4. Pivot Table equivalent of GroupBy:")
pivot_result = pd.pivot_table(df, 
                             values='Salary', 
                             index='Department', 
                             aggfunc=['mean', 'sum', 'count'])
print("Pivot Table Result:")
print(pivot_result)

# GroupBy with multiple functions
print("\n5. Comprehensive department analysis:")
comprehensive = df.groupby('Department').agg({
    'Salary': ['count', 'mean', 'std', 'min', 'max'],
    'Experience': 'mean',
    'Bonus': 'sum'
}).round(2)
print(comprehensive)

Real-World E-commerce Example

Real-World GroupBy Application

import pandas as pd
import numpy as np

# Real-world example: E-commerce data analysis
np.random.seed(42)
dates = pd.date_range('2024-01-01', periods=100, freq='D')
categories = ['Electronics', 'Clothing', 'Books', 'Home', 'Sports']
regions = ['North', 'South', 'East', 'West']

ecommerce_data = {
    'Date': dates,
    'Category': np.random.choice(categories, 100),
    'Region': np.random.choice(regions, 100),
    'Sales_Amount': np.random.randint(50, 500, 100),
    'Units_Sold': np.random.randint(1, 10, 100),
    'Customer_Rating': np.random.uniform(3.0, 5.0, 100).round(1)
}

df = pd.DataFrame(ecommerce_data)
df['Month'] = df['Date'].dt.month_name()
df['Day_Of_Week'] = df['Date'].dt.day_name()

print("E-commerce Data Sample:")
print(df.head(8))

# 1. Monthly sales by category
print("\n1. Monthly Sales by Category:")
monthly_sales = df.groupby(['Month', 'Category'])['Sales_Amount'].sum().unstack()
print(monthly_sales)

# 2. Regional performance analysis
print("\n2. Regional Performance:")
regional_stats = df.groupby('Region').agg({
    'Sales_Amount': ['sum', 'mean', 'count'],
    'Customer_Rating': 'mean',
    'Units_Sold': 'sum'
}).round(2)
print(regional_stats)

# 3. Best performing category by region
print("\n3. Best Category by Region (by total sales):")
def top_category(group):
    return group.groupby('Category')['Sales_Amount'].sum().idxmax()

best_categories = df.groupby('Region').apply(top_category)
print(best_categories)

# 4. Day-wise analysis
print("\n4. Average Sales by Day of Week:")
day_analysis = df.groupby('Day_Of_Week').agg({
    'Sales_Amount': 'mean',
    'Units_Sold': 'mean',
    'Customer_Rating': 'mean'
}).round(2)
print(day_analysis)

# 5. Complex multi-level grouping
print("\n5. Category Performance by Region and Month:")
complex_grouping = df.groupby(['Category', 'Region', 'Month']).agg({
    'Sales_Amount': ['sum', 'mean'],
    'Units_Sold': 'sum',
    'Customer_Rating': 'mean'
}).round(2)
print(complex_grouping.head(10))

GroupBy Best Practices

Use specific columns instead of entire DataFrame
Chain operations for better performance
Use agg() for multiple aggregations
Consider using pd.pivot_table() for simple cases
Reset index after grouping for cleaner DataFrames

Performance Tips

Avoid using apply() when built-in methods exist
Use categorical data for grouping columns
Sort data before grouping if needed
Use as_index=False to keep grouping columns as regular columns
Consider Dask for very large datasets

Important: Remember that GroupBy operations are lazy - they don't compute until you apply an aggregation function. This allows for efficient chaining of operations.

Pro Tip: Use .reset_index() after GroupBy operations to convert the result back to a regular DataFrame with proper column names.

← Data Manipulation Merging Data →

Frontend

JavaScript & Frameworks

Backend

Artificial Intelligence

Database

CSS Frameworks

Data Analytics

Digital Marketing