Since their first inception, automatic reading systems have evolved substantially, yet the recognition of handwriting remains an open research problem due to its substantial variation in appearance. With the introduction of Markovian models to the field, a promising modeling and recognition paradigm was established for automatic handwriting recognition. However, no standard procedures for building Markov model-based recognizers have yet been established. This text provides a comprehensive overview of the application of Markov models in the field of handwriting recognition, covering both hidden Markov models and Markov-chain or n-gram models. First, the text introduces the typical architecture of a Markov model-based handwriting recognition system, and familiarizes the reader with the essential theoretical concepts behind Markovian models. Then, the text reviews proposed solutions in the literature for open problems in applying Markov model-based approaches to automatic handwriting recognition.