Discrete Stochastic Processes | Electrical Engineering and Computer Science | MIT OpenCourseWare

Course Info

Instructor

Prof. Robert Gallager

Departments

Electrical Engineering and Computer Science

As Taught In

Spring 2011

Level

Graduate

Topics

Mathematics
- Discrete Mathematics
- Probability and Statistics

Learning Resource Types

Open Textbooks

Lecture Videos

Problem Sets

Problem Set Solutions

Exams

Exam Solutions

Download Course

Lecture Videos

Lecture 1: Introduction and Probability Review

Lecture 2: More Review; The Bernoulli Process

Lecture 3: Law of Large Numbers, Convergence

Lecture 4: Poisson (The Perfect Arrival Process)

Lecture 5: Poisson Combining and Splitting

Lecture 6: From Poisson to Markov

Lecture 7: Finite-state Markov Chains; The Matrix Approach

Lecture 8: Markov Eigenvalues and Eigenvectors

Lecture 9: Markov Rewards and Dynamic Programming

Lecture 10: Renewals and the Strong Law of Large Numbers

Lecture 11: Renewals: Strong Law and Rewards

Lecture 12: Renewal Rewards, Stopping Trials, and Wald's Inequality

Lecture 13: Little, M/G/1, Ensemble Averages

Lecture 14: Review

Lecture 15: The Last Renewal

Lecture 16: Renewals and Countable-state Markov

Lecture 17: Countable-state Markov Chains

Lecture 18: Countable-state Markov Chains and Processes

Lecture 19: Countable-state Markov Processes

Lecture 20: Markov Processes and Random Walks

Lecture 21: Hypothesis Testing and Random Walks

Lecture 22: Random Walks and Thresholds

Lecture 23: Martingales (Plain, Sub, and Super)

Lecture 24: Martingales: Stopping and Converging

Lecture 25: Putting It All Together

Course Info

Instructor

Prof. Robert Gallager

Departments

Electrical Engineering and Computer Science

As Taught In

Spring 2011

Level

Graduate

Topics

Mathematics
- Discrete Mathematics
- Probability and Statistics

Learning Resource Types

Open Textbooks

Lecture Videos

Problem Sets

Problem Set Solutions

Exams

Exam Solutions

Download Course

MIT Open Learning

Over 2,500 courses & materials

Freely sharing knowledge with learners and educators around the world. Learn more

© 2001–2026 Massachusetts Institute of Technology

Accessibility

Creative Commons License

Terms and Conditions

Proud member of:

© 2001–2026 Massachusetts Institute of Technology