This page includes an interactive code editor. Try modifying and running the examples!

← Rolling & Window Functions Visualization →

Pandas Time Series Analysis

Time series analysis is a crucial aspect of data analysis that deals with time-indexed data. Pandas provides powerful tools for working with temporal data, including date/time indexing, resampling, and rolling window operations.

1. Creating Time Series Data

Pandas offers several ways to create time series data with proper datetime indexing.

date_range(): Generate sequences of dates
to_datetime(): Convert strings to datetime objects
Timestamp: Individual time points
Period: Time intervals

Frequency	Code	Description
Daily	`freq='D'`	Calendar day frequency
Business Day	`freq='B'`	Business day frequency
Hourly	`freq='H'`	Hourly frequency
Weekly	`freq='W'`	Weekly frequency
Monthly	`freq='M'`	Month end frequency

2. Time Series Indexing and Slicing

Pandas allows intuitive indexing and slicing using datetime objects.

Partial string indexing: df['2023-01']
Date ranges: df['2023-01-01':'2023-01-15']
Time-based selection: df.between_time()

3. Resampling and Frequency Conversion

Change the frequency of your time series data.

Downsampling: Reduce frequency (e.g., daily to weekly)
Upsampling: Increase frequency (e.g., daily to hourly)
Aggregation methods: mean(), sum(), ohlc()

4. Rolling Window Operations

Calculate moving statistics over sliding windows.

Rolling mean: Moving average
Rolling standard deviation: Volatility measurement
Expanding windows: Cumulative statistics
Exponential weighted moving average: Recent values weighted higher

5. Shifting and Differencing

Create lag features and calculate differences for time series analysis.

Shift: Create lagged versions of data
Diff: Calculate differences between periods
Percentage change: pct_change()

6. Time-based Feature Engineering

Extract meaningful features from datetime index.

Temporal features: Day of week, month, quarter, year
Business features: Is weekend, is holiday, business days
Seasonal features: Hour of day, week of year

Time Series Components

A time series typically consists of four components:

Trend

Long-term increase or decrease

Seasonality

Regular periodic fluctuations

Cyclical

Irregular periodic fluctuations

Noise

Random variation

Example: Comprehensive Time Series Analysis

Time Series Analysis Example

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# 1. CREATING TIME SERIES DATA
print("=== CREATING TIME SERIES DATA ===")

# Create date range
dates = pd.date_range('2023-01-01', periods=100, freq='D')
print("Date Range:")
print(dates[:5])  # Show first 5 dates

# Create time series data with trend and seasonality
np.random.seed(42)
trend = np.linspace(0, 10, 100)
seasonality = 5 * np.sin(2 * np.pi * np.arange(100) / 30)
noise = np.random.normal(0, 0.5, 100)
values = trend + seasonality + noise

# Create DataFrame
ts_df = pd.DataFrame({
    'date': dates,
    'value': values,
    'temperature': 20 + 10 * np.sin(2 * np.pi * np.arange(100) / 365) + np.random.normal(0, 2, 100),
    'sales': 100 + 50 * np.sin(2 * np.pi * np.arange(100) / 7) + np.random.normal(0, 10, 100)
})
ts_df = ts_df.set_index('date')
print("\nTime Series DataFrame (first 5 rows):")
print(ts_df.head())

# 2. BASIC TIME SERIES OPERATIONS
print("\n=== BASIC TIME SERIES OPERATIONS ===")

# Selecting data by date
jan_data = ts_df['2023-01']
print("January 2023 Data:")
print(jan_data.head())

# Slicing by date range
q1_data = ts_df['2023-01-01':'2023-03-31']
print("\nQ1 2023 Data Shape:", q1_data.shape)

# 3. RESAMPLING AND FREQUENCY CONVERSION
print("\n=== RESAMPLING ===")

# Downsampling to weekly frequency
weekly_mean = ts_df.resample('W').mean()
weekly_sum = ts_df.resample('W').sum()
print("Weekly Mean (first 4 weeks):")
print(weekly_mean.head(4))

# Upsampling to hourly frequency (with forward fill)
hourly_data = ts_df.resample('H').ffill().head(24)  # First 24 hours
print("\nHourly Data (first 24 hours):")
print(hourly_data.head())

# 4. ROLLING STATISTICS
print("\n=== ROLLING STATISTICS ===")

# Rolling mean (7-day moving average)
ts_df['7_day_avg'] = ts_df['value'].rolling(window=7).mean()
ts_df['30_day_avg'] = ts_df['value'].rolling(window=30).mean()

# Rolling standard deviation
ts_df['7_day_std'] = ts_df['value'].rolling(window=7).std()

print("Data with Rolling Statistics (first 10 rows):")
print(ts_df[['value', '7_day_avg', '30_day_avg', '7_day_std']].head(10))

# 5. TIME SERIES SHIFTING AND DIFFERENCING
print("\n=== SHIFTING AND DIFFERENCING ===")

# Lag features
ts_df['value_lag1'] = ts_df['value'].shift(1)  # Previous day
ts_df['value_lag7'] = ts_df['value'].shift(7)  # Same day last week

# Differences
ts_df['value_diff1'] = ts_df['value'].diff(1)  # Day-to-day difference
ts_df['value_diff7'] = ts_df['value'].diff(7)  # Week-to-week difference

print("Data with Lag and Difference Features:")
print(ts_df[['value', 'value_lag1', 'value_lag7', 'value_diff1', 'value_diff7']].head(10))

# 6. TIME-BASED FEATURE ENGINEERING
print("\n=== TIME-BASED FEATURES ===")

# Extract time components
ts_df['day_of_week'] = ts_df.index.dayofweek
ts_df['day_name'] = ts_df.index.day_name()
ts_df['month'] = ts_df.index.month
ts_df['quarter'] = ts_df.index.quarter
ts_df['is_weekend'] = ts_df.index.dayofweek.isin([5, 6]).astype(int)

print("Data with Time Features:")
print(ts_df[['value', 'day_of_week', 'day_name', 'month', 'is_weekend']].head(10))

# 7. HANDLING MISSING DATES
print("\n=== HANDLING MISSING DATES ===")

# Create data with missing dates
missing_dates = dates.drop([10, 11, 12, 50, 51])  # Remove some dates
ts_missing = pd.DataFrame({
    'value': 100 + np.random.normal(0, 10, len(missing_dates))
}, index=missing_dates)

print("Data with Missing Dates:")
print(ts_missing.head(15))

# Reindex to complete date range
ts_complete = ts_missing.reindex(dates)
print("\nAfter Reindexing (showing missing values):")
print(ts_complete.loc['2023-01-08':'2023-01-15'])  # Show period with missing dates

# Fill missing values
ts_filled = ts_complete.fillna(method='ffill')  # Forward fill
ts_interpolated = ts_complete.interpolate(method='linear')  # Linear interpolation

print("\nAfter Forward Fill:")
print(ts_filled.loc['2023-01-08':'2023-01-15'])

# 8. TIME ZONE HANDLING
print("\n=== TIME ZONE HANDLING ===")

# Create timezone-aware index
ts_tz = ts_df.copy()
ts_tz.index = ts_tz.index.tz_localize('UTC')
print("UTC Timezone:")
print(ts_tz.index.tz)

# Convert to different timezone
ts_tz_est = ts_tz.tz_convert('US/Eastern')
print("\nEST Timezone:")
print(ts_tz_est.index.tz)

# 9. WINDOW FUNCTIONS
print("\n=== WINDOW FUNCTIONS ===")

# Expanding window statistics
ts_df['expanding_mean'] = ts_df['value'].expanding().mean()
ts_df['expanding_std'] = ts_df['value'].expanding().std()

# Exponential weighted moving average
ts_df['ewm_avg'] = ts_df['value'].ewm(span=7).mean()

print("Expanding and EWM Statistics:")
print(ts_df[['value', 'expanding_mean', 'expanding_std', 'ewm_avg']].tail(10))

# 10. PERIODS VS TIMESTAMPS
print("\n=== PERIODS VS TIMESTAMPS ===")

# Create period index
periods = pd.period_range('2023-01', periods=12, freq='M')
period_df = pd.DataFrame({
    'sales': np.random.randint(1000, 5000, 12)
}, index=periods)
print("Period Index DataFrame:")
print(period_df)

# Convert period to timestamp
timestamp_df = period_df.to_timestamp()
print("\nConverted to Timestamp:")
print(timestamp_df.head())

print("\n=== SUMMARY STATISTICS ===")
print("Time Series Summary:")
print(f"Start Date: {ts_df.index.min()}")
print(f"End Date: {ts_df.index.max()}")
print(f"Duration: {ts_df.index.max() - ts_df.index.min()}")
print(f"Number of observations: {len(ts_df)}")
print(f"Mean value: {ts_df['value'].mean():.2f}")
print(f"Standard deviation: {ts_df['value'].std():.2f}")

Common Time Series Methods

resample() - Frequency conversion
rolling() - Moving window calculations
shift() - Create lag features
diff() - Calculate differences
asfreq() - Convert to specific frequency

Key Applications

Financial analysis (stock prices)
Sales forecasting
Weather data analysis
IoT sensor data
Web traffic analytics

Important: Always ensure your datetime index is properly set and sorted before performing time series operations. Use df.sort_index() if necessary.

Performance Tip: For large time series datasets, consider using pd.DataFrame.rolling() with engine='numba' for faster calculations.

← Rolling & Window Functions Visualization →

Frontend

JavaScript & Frameworks

Backend

Artificial Intelligence

Database

CSS Frameworks

Data Analytics

Digital Marketing