CSC456: Stochastic Processes (Fall 2025)
Course Objectives:
- To define basic concepts from the theory of Markov chains and present proofs for the most important theorems.
- To compute probabilities of transition between states and return to the initial state after long time intervals in Markov chains.
- To derive differential equations for time continuous Markov processes with a discrete state space.
- To solve differential equations for distributions and expectations in time continuous processes and determine corresponding limit distributions.
Course Contents:
The course covers stochastic processes and their applications
Topics include: Overview; Poisson Processes; Renewal Processes; Discrete-Time Markov Chain; Continuous-Time Markov Chains; Markov Renewal & Semi-Regenerative Processes; Brownian Motion and Diffusion Processes.
Handouts, Quizzes and Assignments
Handouts
- Probability Handout
- Markov Chain Handout
- Transforming a Process into a Markov Chain
- Simple Random Walk
- Chapman–Kolmogorov-Equation
- Stationary Distribution in Markov Chain & Initial State Vector Method
- Continuous Time Markov Chains (CTMC)
Assignments & Quizzes
- Assignment 1
- Sample Quiz
- Quiz Competition
- Assignment 2
- Assignment 3
- Assignment 4
- Quiz 1
- Quiz 3
- Quiz 4
Please click on View Online to see inside the PDF.
Recommended Books:
- An Introduction to Stochastic Processes, Kao, E. P.C., Dover Publications, 2019.
- Introduction to Stochastic Processes with R, Dobrow, R. P., Wiley, 2016.
Further Reading
- Introduction to probability models, Ross, S. M., Academic press, 2014.