From a practical standpoint, we define the optimization task as follows: given a system or process, find the best solution to this process within constraints. This task requires the following elements:
• An OBJECTIVE FUNCTION is needed that provides a scalar quantitative performance measure that needs to be minimized or maximized. This can be the system’s cost, yield, profit, etc.
• A PREDICTIVE MODEL is required that describes the behavior of the system. For the optimization problem this translates into a set of equations and inequalities that we term constraints. These constraints comprise a feasible region that defines limits of performance for the system.
• VARIABLES that appear in the predictive model must be adjusted to satisfy the constraints. This can usually be accomplished with multiple instances of variable values, leading to a feasible region that is determined by a subspace of these variables. In many engineering problems, this subspace can be characterized by a set of decision variables that can be interpreted as degrees of freedom in the process.
Optimization is a fundamental and frequently applied task for most engineering activities. However, in many cases, this task is done by trial and error (through case study). To avoid such tedious activities, we take a systematic approach to this task, which is as efficient as possible and also provides some guarantee that a better solution cannot be found.
The systematic determination of optimal solutions leads to a large family of methods and algorithms. Moreover, the literature for optimization is dynamic, with hundreds of papers published every month in dozens of journals. Moreover, research in optimization can be observed at a number of different levels that necessarily need to overlap but are often considered by separate communities:
• At the mathematical programming1 level, research focuses on understanding fundamental properties of optimization problems and algorithms. Key issues include existence of solutions, convergence of algorithms, and related issues such as stability and convergence rates.
• The scientific computing level is strongly influenced by mathematical properties as well as the implementation of the optimization method for efficient and “practical” use. Here research questions include numerical stability, ill-conditioning of algorithmic steps, and computational complexity and performance.
• At the level of operations research, attention is focused on formulation of the optimization problem and development of solution strategies, often by using well-established solution methods. Many of the problems encountered at this level consider well structured models with linear and discrete elements.
• At the engineering level, optimization strategies are applied to challenging, and often poorly defined, real-world problems. Knowledge of optimization at this level is engaged with the efficiency and reliability of applicable methods, analysis of the solution, and diagnosis and recovery from failure of the solution method.
From the above description of optimization research, it is clear that successful development of an optimization strategy within a given level requires a working knowledge of the preceding levels. For instance, while it is important at the mathematical programming level to develop the “right” optimization algorithm, at the engineering level it is even more important to solve the “right” optimization problem formulation. On the other hand, as engineers need to consider optimization tasks on a regular basis, a systematic approach with a fundamental knowledge of optimization formulations and algorithms is essential. It should be noted that this requires not only knowledge of existing software, which may have limited application to particularly difficult problems, but also knowledge of the underlying algorithmic principles that allow challenging applications to be addressed.
• An OBJECTIVE FUNCTION is needed that provides a scalar quantitative performance measure that needs to be minimized or maximized. This can be the system’s cost, yield, profit, etc.
• A PREDICTIVE MODEL is required that describes the behavior of the system. For the optimization problem this translates into a set of equations and inequalities that we term constraints. These constraints comprise a feasible region that defines limits of performance for the system.
• VARIABLES that appear in the predictive model must be adjusted to satisfy the constraints. This can usually be accomplished with multiple instances of variable values, leading to a feasible region that is determined by a subspace of these variables. In many engineering problems, this subspace can be characterized by a set of decision variables that can be interpreted as degrees of freedom in the process.
Optimization is a fundamental and frequently applied task for most engineering activities. However, in many cases, this task is done by trial and error (through case study). To avoid such tedious activities, we take a systematic approach to this task, which is as efficient as possible and also provides some guarantee that a better solution cannot be found.
The systematic determination of optimal solutions leads to a large family of methods and algorithms. Moreover, the literature for optimization is dynamic, with hundreds of papers published every month in dozens of journals. Moreover, research in optimization can be observed at a number of different levels that necessarily need to overlap but are often considered by separate communities:
• At the mathematical programming1 level, research focuses on understanding fundamental properties of optimization problems and algorithms. Key issues include existence of solutions, convergence of algorithms, and related issues such as stability and convergence rates.
• The scientific computing level is strongly influenced by mathematical properties as well as the implementation of the optimization method for efficient and “practical” use. Here research questions include numerical stability, ill-conditioning of algorithmic steps, and computational complexity and performance.
• At the level of operations research, attention is focused on formulation of the optimization problem and development of solution strategies, often by using well-established solution methods. Many of the problems encountered at this level consider well structured models with linear and discrete elements.
• At the engineering level, optimization strategies are applied to challenging, and often poorly defined, real-world problems. Knowledge of optimization at this level is engaged with the efficiency and reliability of applicable methods, analysis of the solution, and diagnosis and recovery from failure of the solution method.
From the above description of optimization research, it is clear that successful development of an optimization strategy within a given level requires a working knowledge of the preceding levels. For instance, while it is important at the mathematical programming level to develop the “right” optimization algorithm, at the engineering level it is even more important to solve the “right” optimization problem formulation. On the other hand, as engineers need to consider optimization tasks on a regular basis, a systematic approach with a fundamental knowledge of optimization formulations and algorithms is essential. It should be noted that this requires not only knowledge of existing software, which may have limited application to particularly difficult problems, but also knowledge of the underlying algorithmic principles that allow challenging applications to be addressed.