Time Series with Pandas (Basics)

• Majority of the data is in date-time series

• Datetime function

  • Python has a built-in date time functionality, and pandas has it’s own aswell
  • dates are very precise, and can go down to minuscule seconds

  from datetime import datetime
  import pandas as pd
  
  my_date_time = datetime(2023, 01, 05, 12,35,49) 
  
  # can access month, year, etc
  my_date_time.month # output is 01
  
  # Pandas datetime columns
  date_time_indx = pd.DatetimeIndex(col_name) 
  
  # In time series analysis, our index is time, and we can do bunch of analysis on it
  
  # Max index
  df.index.argmax()
  
  # Max value
  df.index.max()
  

• Time Resampling

  • We usually aggregate the date data / time to some frequency.. i.e. every month, week, quarter, year, etc. —> Group by does not work here b/c of business logic / time logic is missing sometimes
  • To do this, we do time re-sampling using built in pandas functions
  • First: Set the time as df index
  • then use a rule to resample
  • then we have to add different aggregation (Mean, max, sum … etc)
  • We can also apply our own custom functions

  df.resample(rule='A').mean() # A here represents year end frequency 
  
  df.resample(rule='Q').max() # Quarterly max 
  
  # Make our own customization functions
  def first_day(entry):
  	return entry[0]
  # Apply the function to our resampled df
  df.resample('A').apply(first_day)
  

  

• Time Shifts

  • Shifts the data by the desired number of periods — For example, if you want to shift all rows to move down by 1, or 2, or 30, etc.
    • This allows us to calculate different financial metrics
  • T-shift: Shifting everything on the index to a specific frequeny.
    • For example, if you want to shift all of your dates to monthly, or yearly, or weekly etc

  # Time shift example -- Shift everything down by 1 
  df.shift(periods = 1) 
  
  # To shift backwards, you just set the periods to negative
  df.shift(periods=-1) 
  
  # T shift 
  df.tshift(freq='M') # Shift all index dates to the end of the month
  
  df.tshift(freq='A') # Shift all index dates to the end of the month
  

• Pandas Rolling and Expanding — Creating Simple Moving Averages

  • Create rolling mean, moving average, etc
  • First, get a datetime index
  • now, we can use the build in pandas function (.rolling())

  df.rolling(7).mean()
  # Give us the moving average after every 7 days 
  
  # Can also change it to different days or do it on specific columns
  df['Open'].rolling(30).mean() # Moving average for opening price for 30 days
  

  • Expanding: When you want to do everything from the start of the time-series up to the current point → Use expanding method
    • Just takes into account everything since the start
    • It’s a just a general expanding mean

  df['Close'].expanding(min_periods=1).mean()
  # At each paticular point on the x axis, it just means that it takes
  # into account everything before it
  

• Bullenger Bands — One technical indicator

  • Volatility bands placed below and above the price
  • Use to decide if a particular stocks price is too high or low
  • When the price is above the upperband → bearish, or when the price is below the band → bearish —- It’s one of the indicators (obv just don’t use this)

# Close 20 day moving average stock price
df['Close: 20 Day moving Average'] = df['Close'].rolling(20).mean() 

# upper = 20MA + 2std(20)
df['Upper'] = df['Close: 20 Day moving Average'] + 2*df['Close'].rolling(20).std() 

# lower = 20MA - 2std(20) 
df['Lower'] = df['Close: 20 Day moving Average'] - 2*df['Close'].rolling(20).std() 

# Close 
df['Close']

# plot them all in 1 plot
df[['Close', 'Upper', 'Lower', 'Close: 20 Day moving Average']].plot(figsize(16,6))

Time Series Analysis - Analysis Techniques

Agenda:

1. Time Series Basics
2. Statsmodels
3. ETS Models and Decomposition
4. ARIMA Models
5. EWMA Models (Exponentially Weighted Moving Averages)

Time Series Basics

• Trends → Upward, Horizontal/stationary, Downward (What is the average doing overall)