From Basic Set Theory to Bayesian Inference
Probability theory is at the core of statistical analysis and machine learning. Mastering it is essential to understanding and developing robust models as a data scientist. This blog will walk you through key concepts in probability theory, from the basics of Set Theory to Advanced Bayesian inference, with detailed explanations and practical examples.
Table of Contents
· Introduction
· Basic Set Theory
· Basic Probability Concepts
· Random Variables and Expectations
· Marginal, Joint, and Conditional Probability
· Rules of Probability: Marginalization and Products
· Bayes’ Theorem
· Probability Distributions
· Using Probability for Learning
· Bayesian Inference
· Implementing Probability Concepts in Python
· Toy Example: Bayesian Inference for Coin Flipping
· Conclusion
· Call to Action
Introduction
Probability theory is the mathematical framework for quantifying uncertainty. It allows us to model and analyze random phenomena, and it is indispensable in statistics, machine learning, and data science. Probability theory helps us make informed decisions, assess risks, and build predictive models.
Basic Set Theory
First, let us define a few key terms.
A Set is a collection of objects. These objects are called elements of the set.
Subset b of a set a is a set whose elements are elements of a, i.e., 𝑏 ⊂ 𝑎.
Space S is the largest set; Thus, all other sets are under consideration 𝑠ᵢ ⊂ 𝑆.
Null Set O is an empty or null set. O contains no elements.
Let us visualize the components of set theory.
The figure above depicts various scenarios we encounter with sets. Let us describe different aspects of set theory. Readers are encouraged to refer to the visual after each subsection to deepen their intuition upon reading the definitions and reviewing mathematical expressions.
Subsets
Subset 𝑏 ⊂ 𝑎, or the set a contains b, 𝑎 ⊃ 𝑏 if all elements of b are also elements of a. That is,
In English: The statement, “If b ⊆ a, and c ⊆ b, then c ⊆ a,” expresses the transitive property of set inclusion. If set b is a subset of set a, and set c is a subset of set b, then c must also be a subset of a. The second item, “The following relationship holds: a ⊆a, 0 ⊆ a, a ⊆ S,” highlights the basic properties of set inclusion. Hence:
- a ⊆ a indicates that every set is a subset of itself.
- 0 ⊆ a indicates that the empty set is a subset of any set a.
- a ⊆ S indicates that any set a is a subset of the universal set S.
Set Operations
Equality: For two sets to be equal, every element of a must be in b, and every component of b must be in a. Mathematically speaking:
Union (Sum): The union of two sets, a and b, is a set consisting of all elements of a or b or both. The union operation satisfies the following properties:
The intersection (Product) of sets a and b consists of all elements common to sets a and b. The intersection operation satisfies the following properties:
Mutually Exclusive Sets
We call two sets a and b mutually exclusive or disjoint if they have no common elements, i.e.,
Complements
The complement a of a set a is defined as a set consisting of all elements of S that are not in a . Complement sets satisfy the following properties:
De Morgan Law
The image illustrates De Morgan’s Laws, which are fundamental rules in set theory and Boolean algebra. These laws describe the relationship between the union and intersection of sets and their complements.
Leave a Reply