This book lays out the theoretical foundation of the so-called multi-armed bandit (MAB) problems and puts it in the context of resource management in wireless networks. Part I of the book presents the formulations, algorithms and performance of three forms of MAB problems, namely, stochastic, Markov and adversarial. Covering all three forms of MAB problems makes this book unique in the field. Part II of the book provides detailed discussions of representative applications of the sequential learning framework in cognitive radio networks, wireless LANs and wireless mesh networks. Both individuals in industry and those in the wireless research community will benefit from this comprehensive and timely treatment of these topics. Advanced-level students studying communications engineering and networks will also find the content valuable and accessible.