• Probability theory is nothing but common sense reduced to calculation. — Pierre Laplace, 1812 — Pierre Laplace, 1812
• Two interpretations / approaches:
  • Frequentists: Long run frequencies of events that can happen multiple events
  • Bayesian: Quantifies uncertainty by information
    • Can be used to model one-off events, which do not have long term frequencies
  • The basic rules of probability are the same, regardless of the approach you follow

Introduction to Probability Theory

• Probability is how likely is an event to occur ?

• Probability = event of interest / sample space

• For example, suppose there are 10 kids and 3 of them play soccer.

• Hence, P(play soccer) = 3 / 10

• All possible outcomes must be inside of the sample space

  

• Here, the rectangle is the sample space and the kids inside the circle (who play soccer) are an Event

• Experiment → Any process that produces something that is uncertain. I.e. flipping a fair coin would be an experienment as we don’t know whats the outcome (heads or tail?)

• Another example: Suppose you flip 2 fair coins, what are the chances of landing on 2 heads? → Think about all possible set of outcomes that can happen… and then answer.

  

• What if you throw 3 coins? 8 possible outcomes!

  

• If we roll the dice or a coin over 1000+ times, we can see that the probabilities actually converge to it’s theoretical values!

  

Complement of Probability → Recall that P(soccer) = 3/10. Since we know all probabilities add up to one, then we know that P(not soccer) = 1 - P(soccer) = 1 - 0.3 = 0.7!

• More formally, P(A’) = 1 - P(A) is the complement rule

  

• Another example: Remember we flipped a coin three times. What is the probability of not getting 3 heads in a row?

  • P(Not 3 heads) = ?
  • P(Not 3 heads) = 1 - P( 3 heads) = 1 - (1/8) = 7/8

Sum of Probabilities (Disjoint and Joint Events)

• If the events are disjoint/mutually exclusive ( i.e. cannot happen at the same time), then you can just add up their probabilities to get the probability of A or B happening

  

• However, what happens when they are NOT disjoint … ?