Probability and Statistics (Fall 2023)

Topic Introduction

Probability and Statistics are two closely-related but different topics. While both of them study the black box of randomness, they focus on different problems:

  • Probability: I have known the inside mechanism. What are the outside behaviors?
  • Statistical Inference: I have known the outside behaviors. What is the inside mechanism?

Probability (including Random Process) is a very important theoretical mathematical model for many advanced topics, e.g., Advanced Algorithms (including Randomized Algorithms and Approximation Algorithms), Analysis of Algorithms, Combinatorial Mathematics (particularly, The Probabilistic Methods popularized by Paul Erdős[ˈɛrdøːʃ]), etc.

Based on Probability, Statistics is a very important applied mathematical tool to solve many problems. For example, it is the basis of most Machine Learning algorithms. Actually, since most Machine Learning algorithms come from the field of Statistical Learning, it is fair to say that a large part of Machine Learning belongs to Statistics. Besides, if you want to be a great decision maker (e.g., policy maker in government, investor, etc.), Statistics will be essential for you. In my view, Statistics is the core of Data Science.

Seminar Information

Content

In this seminar, we’ll be using Probability and Computing by Michael Mitzenmacher and Eli Upfal as our main material. Hopefully, we will cover at least Chapter 1 to Chapter 7 of this book in this seminar, which are:

  • Events and Probability
  • Discrete Random Variables and Expectation
  • Moments and Deviations
  • Chernoff and Hoeffding Bounds
  • Balls, Bins, and Random Graphs
  • The Probabilistic Methods
  • Markov Chains and Random Walks

Besides, we might also cover some topics beyond these 7 chapters:

  • Probability Theory
    • Continuous Random Variable
    • Law of Large Number & Central Limit Theorem
    • Concentration of Measure
  • Statistics
    • Parameter Estimation
    • Hypothesis Testing
  • Some Randomized Algorithms
  • Some Machine Learning Algorithms and Ideas
  • …

Schedule

The detailed schedule will be updated later.

Topics Time Speaker Material
Introduction to Probability 2023/9/21 Runshuo Xie slides
Random Variable 2023/10/12 Runshuo Xie Prerequisite for L2
Prerequisite for L2 (with notes)
handout
handout (with notes)
slides
Presentations on different distributions 2023/10/19 Yifan Lü
Yao Luo
Jiutao Mao & Xiang Jiang
Zhijie Pan
Shide Liang
Yingming He		 二项分布【吕一帆】
几何分布【罗耀】
泊松分布【毛九弢】 泊松分布的应用【蒋庠】
指数分布的应用【潘智杰】
Normal Distribution【梁世徳】
二项分布【贺英明】(2023/11/16)
Tail Inequalities 2023/11/16 Runshuo Xie slides
handout
handout (with notes)
Introduction to Statistical Inference 2023/11/30 Runshuo Xie slides
Introduction to Markov Chain 2023/12/21 Runshuo Xie slides


Reference

Here are all materials that I’ve referred to when I was preparing for the seminar.

Books

Textbook of Probability and Statistics
  • Probability and Random Processes (4th edition) by Geoffrey R. Grimmett, David R. Stirzaker
  • Understanding Probability (3rd edition) by Henk Tijms
  • Elementary Probability Theory With Stochastic Processes and an Introduction to Mathematical Finance by K. L. Chung, Farid AitSahlia
    This is a really unique and fantastic book on probability, which is friendly enough to novice.
  • A Course in Probability Theory by Kai Lai Chung
  • Probability Theory: The Logic of Science by E. T. Jaynes
  • 概率论与数理统计，陈希孺著
Textbook of Computer Science
  • Randomized Algorithms by Rajeev Motwani, Prabhakar Raghavan
  • Concrete Mathmatics by Ronald L. Graham, Donald E. Knuth, Oren Patashnik
  • Concentration of Measure for the Analysis of Randomized Algorithms by Devdatt P. Dubhashi, Alessandro Panconesi
Others
  • The Seven Pillar of Statistical Wisdom by Stephen M. Stigler
  • 从博弈问题到方法论学科——概率论发展史研究，徐传胜著
  • 从掷骰子到阿尔法狗：趣谈概率，张天蓉著
  • 机会的数学：统计学入门，陈希孺著
  • 大数据时代的统计思想，李勇著

Courses

Websites