CSC456: Stochastic Processes (Spring 2026)
Course Learning Outcomes:
- To define basic concepts from the theory of Markov chains and present proofs for the most important theorems.
- To compute probabilities of transition between states and return to the initial state after long time intervals in Markov chains.
- To derive differential equations for time continuous Markov processes with a discrete state space.
- To solve differential equations for distributions and expectations in time continuous processes and determine corresponding limit distributions.
Course Contents:
The course covers stochastic processes and their applications
Topics include: Overview; Poisson Processes; Renewal Processes; Discrete-Time Markov Chain; Continuous-Time Markov Chains; Markov Renewal & Semi-Regenerative Processes; Brownian Motion and Diffusion Processes.
Handouts, Quizzes and Assignments
Handouts
- 1. Stochastic Processes & Probability
- 2. Markov Chain
- 4. Chapman Kolmogorov Theorem
- 6. Stationary Distribution and Initial State Vector Method
- 7. Transforming a Process into a Markov Chain
- 8. Simple Random Walk
- 9. Review
- 10. Continuous Time Markov Chain (CTMC)
- 11. Poisson process vs Markov Chain
Assignments & Quizzes
- Assignment 1
- Assignment 2
Please click on View Online to see inside the PDF.
Recommended Books:
- An Introduction to Stochastic Processes, Kao, E. P.C., Dover Publications, 2019.
- Introduction to Stochastic Processes with R, Dobrow, R. P., Wiley, 2016.
Further Reading
- Introduction to probability models, Ross, S. M., Academic press, 2014.