This book introduces the reader to the world of stochastic modeling. It focuses on a clear presentation of both discrete-time and continuous-time models, based on a solid formulation of the mathematical foundations. The concept of Markov chains runs like a common thread through the individual chapters. Markov chains are of interest in the analysis of discrete-time dynamic systems that are subject to random influences. They impress with their clear structure and simplicity in presentation and solution. Additionally, they, along with Poisson processes and their generalizations, are an important building block for understanding continuous-time systems. Lastly, Markov decision processes, which involve optimal control of Markov chains, emphasize the significance of Markov chains as a crucial analytical tool.