1. George Cummings >
2. Artificial Intelligence >
3. Analytical Decision Making >
4. A. Optimization

A. Optimization

Fri, Sep 16, 2022 The Master’s of Management of Artificial Intelligence (MMAI) programme at Queen’s University starts its students technical training with MMAI861. It seems to serve two purposes: Presenting management tools to assist decision-making and back up decisions Giving a soft entry to mind-boggling mathematic concepts that hit the non-mathematician like yours truly later in the programme.

---

---

The Master’s of Management of Artificial Intelligence (MMAI) programme at Queen’s University starts its students technical training with MMAI861. It seems to serve two purposes:

• Presenting management tools to assist decision-making and back up decisions
• Giving a soft entry to mind-boggling mathematic concepts that hit the non-mathematician like yours truly later in the programme. It is not machine learning by any means but it demonstrates the power of calculation to assist in business decisions.

The main points of the course are:

• Types of decision process
• Optimization problems: for example, how much of each product to manufacturer.
• Simulations
• Decision analytics.

These posts are my aide-memoires. They do not replace the course slides¹, but present some additional notes that may help understand the theory behind the techniques. For example, System 1 and System 2 decision processes do not need any explanation. Biases and fallacies are likewise easy to grasp.

What isn’t easy…

A. Optimization

A.1. Optimization Terms

• Stochastic vs Deterministic: Deterministic means that, for a particular set of inputs, a model will always give the same results. Stochastic, on the other hand, has uncertainty built into the model and will give a range of answers or estimations. I suppose the stochastic nature of real-life data is the reason why understanding statistics is vital to the ML student.
• Static Policy vs Dynamic Policy: Static choices are a predetermined set of answers, like multiple choice. Dynamic choices are sets of answers that change according to events, including previous choices made by the decision-maker.²
• LP: Linear Programming, aka Linear Optimization. Basically, calculating with a simple linear equation such as m = 4x + y + 3.
• Continuous vs Discrete: Discrete numbers can be counted in a finite amount of time³. You can count the number of student in MMAI861. Discrete numbers cannot be measured (e.g. weight is not discrete)⁴. Continuous numbers cannot be counted in a finite amount of time, but they can be measured. No such thing as 95.3 students, but you can have 0.5 cups of sugar.
• Constrained vs unconstrained. Constraints are a set of conditions that a calculation must satisfy. Basically, an answer must fall within a range. Unconstrained answers could be anything within -∞ to ∞.

A.2. Optimization: LP Model for Business

• Objective Function: In Excel, an objective cell. The function whose value (i.e. answer) is to be maximized or minimized, e.g. profit or loss. It is a linear equation.
• Decision Variables: In Excel, a variable cell. The arguments to the objective function. The point of the optimization exericise: they are unknown, so the computer must come up with the appropriate values. Hours of labour per product, amounts of ingredient x, quantity of part y.
• Constraints: The limitations on the decision variables. For example, the amount of sugar cubes in an optimized cup of coffee cannot be negative.

Take the linear equation Z = Ax + By + C. The objective function is the whole statement. x, y, and z are the decision variables. A, B and C are the constants, coefficients that determine the strength of each decision variable. Constraints are max or min function applied to each component. In a facetious, coffee optimization problem: if x is the number of sugar cubes in an series of cups of coffee, then two constraints could be, “you cannot have a negative number of sugar cubes per cup, and there are only 12000 cubes in the inventory.”

Part of optimizing is identifying the objective function, decision variables, and constraints verbally from a business problem. By the way, later on when dealing with loss functions in ML, know that objective functions and loss functions are the same thing.

A.2.1 LP Suitability

Linear Programming assumes the problem is (moderately) simple: a single objective, data certainty, proportionality, additivity and divisibility. If any of the above characteristics are violated, then LP cannot be used.

• Proportionality: decision variables cannot have exponents, for example.
• Additivity: each decision variable is independent.
• Divisibility: outside of deliberate contraints, non-integers are permitted.

A.2.2 LP Graphical Solutions

Demonstrates how Excel (for example) go about setting the decision variables.

The visual method of optimization is restricted to LP models with two (maybe three) decision variables. Obviously, non-visual may use more than three.

• The feasible region is bounded by constraints.
• The algorithm (or analyst) moves the objective through the feasible region.
• The optimal solution is always found at a vertex of the feasible region.
• There may be multiple optimal solutions (i.e., different vertices).

Excel Solver uses the Simplex Method to search through the feasible region. One Python equivalent could be scipy.optimize.linprog()

Some folks have pretty well had it with Excel. Python has several LP solver packages available. See How to create your first linear programming solver in Python.

A.3. Sensitivity Analysis for LP

Once the decision variables have been optimized, one runs a sensitivity analysis to explore further options for the business.

According to Investopedia, Sensitivity Analysis determines how different values of a decision variable affect the objective function under a given set of assumptions. It is used for “what-if” conversations. Messing around with the constraints is fun!

Definitions:

• Shadow Price: the marginal improvement in the object value function caused by relaxing one (and only one) constraint by one unit.

A.4. Integer Programming

Integer Programming is LP with the constraint that one or more decision variables are restricted to integers, including binary {0,1} integers. A factory may need to deliver a number of boxes of product to several outlets. A machine at the factory may be used or unused (1,0) – a fixed-cost model.

See Joshua Emmanuel’s YouTube video for examples of integer programming with binary constraints.

Of note: binary constraints are a mathematical version of an “if” statement.

Next: Math for simulations 1