Practical Topics in Optimization
It serves as a foundational resource for students, researchers, and practitioners in mathematics, computer science, and related fields, offering both theoretical depth and practical application, but it is incremental as it synthesizes existing knowledge rather than presenting new research.
This book provides an introductory guide and comprehensive reference on optimization techniques, aiming to demystify algorithms like black-box and stochastic optimizers through formal and intuitive explanations to equip readers with knowledge for applying these methods in fields such as machine learning and operations research.
In an era where data-driven decision-making and computational efficiency are paramount, optimization plays a foundational role in advancing fields such as mathematics, computer science, operations research, machine learning, and beyond. From refining machine learning models to improving resource allocation and designing efficient algorithms, optimization techniques serve as essential tools for tackling complex problems. This book aims to provide both an introductory guide and a comprehensive reference, equipping readers with the necessary knowledge to understand and apply optimization methods within their respective fields. Our primary goal is to demystify the inner workings of optimization algorithms, including black-box and stochastic optimizers, by offering both formal and intuitive explanations. Starting from fundamental mathematical principles, we derive key results to ensure that readers not only learn how these techniques work but also understand when and why to apply them effectively. By striking a careful balance between theoretical depth and practical application, this book serves a broad audience, from students and researchers to practitioners seeking robust optimization strategies.