Derivatives

Why Use Derivatives for ML?

• To optimize functions! To find the maximum or minimum values → Very important in ML, as its used to minimize the loss function, which leads to your model performing at it’s BT

What is a derivative?

• At a given instant, what is the instantaneous rate of change, of a function?

• Suppose you’re driving a car from point A to point B

  • During this process, you may go fast, you may go slow on inner roads, faster on highways, you may have to stop for gas, you may have to go reverse, etc.
  • However, if I was to pin point a single point in time of your entire road trip, and ask what is the instantaneous velocity at that time… That would be your derivative → The function would be distance, with velocity as the derivative, the instantaneous change at that moment

• Velocity = distance traveled / time taken

  

  • Velocity = (202 - 122) metres / 15 - 10 seconds
  • velocity = 16m/s ( The velocity of the car between those two points is 16m/s)
  • Detailed Answer

• So we may not know the velocity at 12.5, but we can take finer and finer intervals.. The better the estimate is. We keep taking tinier and tinier intervals, we will eventually get very close to the actual value → Limit as distance between (x2) - (x1) and (t2) - (t1) approaches 0 → Derivative

  

• Derivative = Instantaneous Rate of Change both measure How fast a relation between two variables is changing at any point

• Derivative of a function at a point is precisely the slope of the tangent at that particular point

Slope, Maxima, and Minima

• Zero Slope → No changes in distance over time → Whenever the tangent is 0 → Slope is 0

  

• The point where the car is not moving, and is the furthest away → Maxima

• The point where the car is not moving, and is the closest → Minima

• So it would be t=10, and t=34 respectively.

• To find the maxima, or minima, it MUST occur at ONE of the points where the derivative = 0 or, the where the tangent line is horizontal

Graphs of Functions and Their Derivatives:

• Piece-wise Linear Function

  

• Non linear Function (notice how the derivative itself is not constant! (It’s a function)