Quadratic assignment problem

Author: Thomas Kueny, Eric Miller, Natasha Rice, Joseph Szczerba, David Wittmann (SysEn 5800 Fall 2020)

Introduction

The Quadratic Assignment Problem, discovered by Koopmans and Beckmann in 1957, is a mathematical optimization module created to describe the location of invisible economic activities. An NP-Complete problem, this model can be applied to many other optimization problems outside of the field of economics. It has been used to optimize backboards, inter-plant transportation, hospital transportation, exam scheduling, along with many other applications not described within this page.

Theory, Methodology, and/or Algorithmic Discussions

Koopmans-Beckman Mathematical Formulation

Economists Koopmans and Beckman began their investigation of the Quadratic Assignment Problem (QAP) to ascertain the optimal method of locating important economic resources in a given area. The Koopmans-Beckman formulation of the Quadratic Assignment Problem aims to achieve the objective of assigning facilities to locations in order to minimize the overall cost. Below is the Koopmans-Beckman formulation of the QAP as described by neos-guide.org.

Quadratic Assignment Problem Formulation

Parameters

$F=(F_{ij})$ is an n x n matrix where $F_{ij}$ the required flow between facilities $i$ and $j$. $D=(D_{ij})$ is an n x n matrix where $D_{ij}$ the distance between facilities $i$ and $j$ $i$ and $j$ . Intuitively, the product of distance and flow represents cost, and the objective is to minimize this cost. Ideally, two facilities with lots of flow (more product or supplies) between the two should be placed closer together than two facilities that rarely interact (low flow).

Sets

Consider the set of numbers $N={\big \{}1,2,...n{\big \}}$. The group of bijections between $N$ and itself is defined as $S_{n}=\{\phi :N\rightarrow N\}$. Each member $\phi \in S_{n}$ represents a particular arrangement of the $n$ elements. As an example, the permutation $\phi =(321)$ represents facility 3 being placed at location 1, facility 1 at location 3 and facility 2 at location 2. Each permutation can be assigned a permutation matrix. The permutation matrix of $\phi$ can be written as

$X_{\phi }={\begin{bmatrix}0&0&1\\0&1&0\\1&0&0\end{bmatrix}}$

The operation $X_{\phi }DX_{\phi }^{T}$ permutes the values of the distance matrix $D$ so that $(X_{\phi }DX_{\phi }^{T})_{i,j}$ represents the distance between facilities $i$ and $j$ if the facilities are arranged according to the permutation $\phi$. The cost function between facilities $i$ and $j$, after being permuted to their location, is $F_{i,j}$$(X_{\phi }DX_{\phi }^{T})_{i,j}$ . To find the total cost, sum over each $i$ and $j$ for the facilities arranged according to $\phi$.

Optimization Problem

$min\sum _{i=1}^{n}\sum _{j=1}^{n}F_{ij}D_{ij}*d{_{\phi (i))\phi (j))}}^{}$

Computational Complexity

QAP belongs to the classification of problems known as NP-complete, thus being a computationally complex problem. QAP’s NP-completeness was proven by Sahni and Gonzalez in 1976, who states that of all combinatorial optimization problems, QAP is the “hardest of the hard”.[1]

Algorithmic Discussions

While an algorithm that can solve QAP in polynomial time is unlikely to exist, there are three primary methods for acquiring the optimal solution to a QAP problem: 1. Dynamic Program 2. Cutting Plane 3. Branch and Bound Procedures The third method has been proven to be the most effective in solving QAP, although when n > 15, QAP begins to become virtually unsolvable.

Branch and Bound Procedures

The Branch and Bound method was first proposed by Ailsa Land and Alison Doig in 1960 and is the most commonly used tool for solving NP-hard optimization problems.

A branch-and-bound algorithm consists of a systematic enumeration of candidate solutions by means of state space search: the set of candidate solutions is thought of as forming a rooted tree with the full set at the root. The algorithm explores branches of this tree, which represent subsets of the solution set. Before one lists all of the candidate solutions of a branch, the branch is checked against upper and lower estimated bounds on the optimal solution, and the branch is eliminated if it cannot produce a better solution than the best one found so far by the algorithm.

Linearizations

The first attempts to solve the QAP eliminated the quadratic term in the objective function of

$min\sum _{i=1}^{n}\sum _{j=1}^{n}c{_{\phi (i)\phi (j)}}+\sum _{i=1}^{n}b{_{\phi (i)}}$

in order to transform the problem into a (mixed) 0-1 linear program. The linearization of the objective function is usually achieved by introducing new variables and new linear (and binary) constraints. Then existing methods for (mixed) linear integer programming (MILP) can be applied. The very large number of new variables and constraints, however, usually poses an obstacle for efficiently solving the resulting linear integer programs. MILP formulations provide moreover LP relaxations of the problem which can be used to compute lower bounds.

Numerical Example

Inner Product

The inner product of two matrices $A,B$ is defined as $<A,B>=\sum _{i=1}^{n}\sum _{j=1}^{n}a_{i,j}b_{i,j}$. The inner product $\langle F,X_{\phi }DX_{\phi }^{T}\rangle =\sum _{i=1}^{n}\sum _{j=1}^{n}F_{i,j}(X_{\phi }DX_{\phi }^{T})_{i,j}$ thus describes the total cost of assigning facilities to locations according to $\phi$. The objective function to minimize cost can be written as $\langle F,XDX^{T}\rangle$.

The true objective cost function only uses the entries above the diagonal in the matrix comprised of elements $F_{i,j}$$(X_{\phi }DX_{\phi }^{T})_{i,j}$.Since this matrix is symmetric with zeroes on the diagonal, dividing by 2 removes the double count of each element to give the correct cost value.

QAP with 3 Facilities

Consider the QAP of placing 3 facilities in 3 locations. Let the distance and flow matrices be defined respectively as $D={\begin{bmatrix}0&5&6\\5&0&3.6\\6&3.6&0\end{bmatrix}}$, ${\textstyle F={\begin{bmatrix}0&10&3\\10&0&6.5\\3&6.5&0\end{bmatrix}}}$.

Let $P$ be the set of permutation matrices for $S_{3}$. The problem can be stated as follows:

$\min _{X\in P}\langle F,XDX^{T}\rangle$.

The table lists the resulting cost function to place the facilities according to each permutation $\phi \in S_{3}$. Note that $\phi =()$ is the identity mapping where the number of the facility is the location number. Based on these results, $\phi _{4}=(13)$ is the arrangement of facilities that minimizes the cost function with a value of 86.5. Facility 1 should be placed at location 3, Facility 2 remains at location 2 and Facility 3 is placed at location 1.

Cost for Each Permutation in $S_{3}$

Permutation	Cost
$\phi _{1}=$ ( )	91.4
$\phi _{2}=(12)$	99.8
$\phi _{3}=(23)$	98.4
$\phi _{4}=(13)$	86.5
$\phi _{5}=(123)$	103.3
$\phi _{6}=(132)$	90

For the optimal solution $\phi _{4}=(13)$, the calculation is as follows:

$XDX^{T}={\begin{bmatrix}0&0&1\\0&1&0\\1&0&0\end{bmatrix}}{\begin{bmatrix}0&5&6\\5&0&3.6\\6&3.6&0\end{bmatrix}}{\begin{bmatrix}0&0&1\\0&1&0\\1&0&0\end{bmatrix}}={\begin{bmatrix}0&3.6&6\\3.6&0&5\\6&5&0\end{bmatrix}}$

$F_{i,j}(X_{\phi }DX_{\phi }^{T})_{i,j}={\begin{bmatrix}0&36&18\\36&0&32.5\\18&32.5&0\end{bmatrix}}$

Looking at the figure of the optimal solution, the flow$\cdot$distance values are repeated in this matrix which is why the sum of all elements must be divided by 2. This gives $\langle F,X_{\phi }DX_{\phi }^{T}\rangle =173$ so the minimum cost is 86.5.

Optimal Solution for the example with facility/location pairings listed. Line thickness is representative of flow. Note that the closest locations (2,3) are assigned facilities (2,1) with the greatest flow between them. The two farthest locations (1,3) are assigned facilities (3,1) that have the lowest flow. Overall, this results in a minimized cost function.

Applications

Inter-plant Transportation Problem

The quadratic assignment problem was first introduced by Koopmans and Beckmann to address how economic decisions could be made to optimize the transportation costs of goods between both manufacturing plants and locations. Factoring in the location of each of the manufacturing plants as well as the volume of goods between locations to maximize revenue is what distinguishes this from other linear programming assignment problems like the Knapsack Problem.

The Backboard Wiring Problem

As the quadratic assignment problem is focused on minimizing the cost of traveling from one location to another, it is an ideal approach to determining the placement of components in many modern electronics. Leon Steinberg proposed a QAP solution to optimize the layout of elements on a blackboard by minimizing the total amount of wiring required.

When defining the problem Steinberg states that we have a set of n elements

$E=\left\{E_{1},E_{2},...,E_{n}\right\}$

as well as a set of r points

$P_{1},P_{2},...,P_{r}$ where $r\geq n$

In his paper he derives the below formula:

$min\sum _{1\leq i\leq j\leq n}^{}C_{ij}(d_{s(i)s(j))})$

$C_{ij}$ represents the number of wires connecting components $E_{i}$ and $E_{i}$

$d_{s(i)s(j))}$ is the length of wire needed to connect tow components if one is placed at $P_{\alpha }$ and the other is placed at $P_{\beta }$. If $P_{\alpha }$ has coordinates $(x_{\alpha },y_{\alpha },z_{\alpha })$ then $d_{s(i)s(j))}$ can be calculated by

$d_{s(i)s(j))}=\surd ({\bigl (}x_{\alpha }-x_{\beta }{\bigr )}^{2}+{\bigl (}y_{\alpha }-y_{\beta }{\bigr )}^{2}+{\bigl (}z_{\alpha }-z_{\beta }{\bigr )}^{2})$

$s(x)$ represents a particular placement of a component.

In his paper Steinberg a backboard with a 9 by 4 array, allowing for 36 potential positions for the 34 components that needed to be placed on the backboard. For the calculation, he selected a random initial placement of s1 and chose a random family of 25 unconnected sets.

The initial placement of components is shown below:

The initial placement of components in the backboard from Steinberg's

After the initial placement of elements, it took an additional 35 iterations to get us to our final optimized backboard layout. Leading to a total of 59 iterations and a final wire length of 4,969.440.

Final component position in Steinberg's Backboard optimization problem

Hospital Layout

Building new hospitals was a common event in 1977 when Alealid N Elshafei wrote his paper on "Hospital Layouts as a Quadratic Assignment Problem".With the high initial cost to construct the hospital and to staff it, it is important to ensure that it is operating as efficiently as possible. Elshafei's paper was commissioned to create an optimization formula to locate clinics within a building in such a way that minimizes the total distance that a patient travels within the hospital throughout the year. When doing a study of a major hospital in Cairo he determined that the Outpatient ward was acting as a bottleneck in the hospital and focused his efforts on optimizing the 17 departments there.

Elshafei identified the following QAP to determine where clinics should be placed:

$min\sum _{i,j}\sum _{k,q}f_{ik}d_{jq}y_{ij}y_{kq}$

Such that $\sum _{j\epsilon J}y_{ij}=1\forall i\epsilon I$

$\sum _{i\epsilon I}y_{ij}=1\forall j\epsilon J$

$y_{ij}{\begin{Bmatrix}=1\quad {\mbox{ , if facility i is located at j}}\\=0\quad {\mbox{ , otherwise}}\end{Bmatrix}}$

$f_{ik}$ is the yearly flow between facilities i and k

$d_{jq}$ is the distance between the possible facility locations j and q

$I$ is the initial set of facilities

$J$ is the initial set of all locations

It can take a large amount of computational power to calculate the solution to a QAP problem, necessitating the use of a heuristic solution rather than using an optimizing algorithm. This approach is composed of two parts(Parts A and Part B). Part A develops the initial layout of the hospital, this is done by the following two strategies. Strategy 1 where we use a rank $L_{i}$ that designates the number of interactions facility $i$ has with with the other facilities. As well as ranking them in terms of $R_{j}$, which designates the sum of distances from j to all other locations. then we go through the list and find the facility with the greatest $L_{i}$ and the least $R_{j}$. Strategy 2 says that at any step in the process we next place an unassigned facility that has the largest number of interactions with the most recently placed facility. Part B covers improving the initial solution developed in Part A, here at any step in the process we test whether the assignment is best for all of the facilities that are involved and make swaps where necessary until there are no more swaps to be made.

For the Cairo hospital with 17 clinics, and one receiving and recording room bringing us to a total of 18 facilities. By running the above optimization Elshafei was able to get the total distance per year down to 11,281,887 from a distance of 13,973,298 based on the original hospital layout.

Exam Scheduling System

The scheduling system uses matrices for Exams, Time Slots, and Rooms with the goal of reducing the rate of schedule conflicts. To accomplish this goal, the “examination with the highest cross faculty student is been prioritized in the schedule after which the examination with the highest number of cross-program is considered and finally with the highest number of repeating student, at each stage group with the highest number of student are prioritized.”