Description

About The Book

Applied Stochastic Processes presents a concise, graduate-level treatment of the subject, emphasizing applications and practical computation. It also establishes the complete mathematical theory in an accessible way.

After reviewing basic probability, the text covers Poisson processes, renewal processes, discrete- and continuous-time Markov chains, and Brownian motion. It also offers an introduction to stochastic differential equations. While the main applications described are queues, the book also considers other examples, such as the mathematical model of a single stock market.

With exercises in most sections, this book provides a clear, practical introduction for beginning graduate students. The material is presented in a straightforward manner using short, motivating examples. In addition, the author develops the mathematical theory with a strong emphasis on probability intuition.

Table of Contents

Probability and Stochastic Processes
Probability
Random variables and their distributions
Mathematical expectation
Joint distribution and independence
Convergence of random variables
Laplace transform and generating functions
Examples of discrete distributions
Examples of continuous distributions
Stochastic processes
Stopping times
Conditional expectation

Poisson Processes
Introduction to Poisson processes
Arrival and inter-arrival times of Poisson processes
Conditional distribution of arrival times
Poisson processes with different types of events
Compound Poisson processes
Nonhomogeneous Poisson processes

Renewal Processes
An introduction to renewal processes
Renewal reward processes
Queuing systems
Queue lengths, waiting times, and busy periods
Renewal equation
Key renewal theorem
Regenerative processes
Queue length distribution and PASTA

Discrete Time Markov Chains
Markov property and transition probabilities
Examples of discrete time Markov chains
Multi-step transition and reaching probabilities
Classes, recurrence, and transience
Periodicity, class property, and positive recurrence
Expected hitting time and hitting probability
Stationary distribution
Limiting properties

Continuous Time Markov Chain
Markov property and transition probability
Transition rates
Stationary distribution and limiting properties
Birth and death processes
Exponential queuing systems
Time reversibility
Hitting time and phase-type distributions
Queuing systems with time-varying rates

Brownian Motion and Beyond
Brownian motion
Standard Brownian motion and its maximum
Conditional expectation and martingales
Brownian motion with drift
Stochastic integrals
Itô’s formula and stochastic differential equations
A single stock market model

Bibliography

Index

Features

Offers a succinct, accessible account of applied stochastic processes for a first-year graduate course
Provides systematic coverage of queuing applications, unlike similar texts on stochastic processes
Includes an introduction to stochastic differential equations
Contains exercises at the end of most sections and a brief bibliography at the back of the book
Solutions manual and figure slides available upon qualifying course adoption

About The Author

Ming Liao is a professor in the Department of Mathematics and Statistics at Auburn University. He has published 45 research papers and one monograph on probability theory. He received a Ph.D. from Stanford University.